(Suggestion) A callback for Runner

[pshrm]

Hi Peter,
We have created a small framework on top of Karate. One of the requirement we had that we needed to send test results (reports) to a central ELK instance and reporting has to be realtime i.e., as soon as a feature is executed, result should be updated in ELK.
We achieved it by modifying the karate code. We realized this is not maintainable as upgrading karate version sometimes becomes time consuming due to our changes.
May I suggest a generic approach.

I am thinking of providing a callback interface in Runner. Something like this.

public interface RunnerCallback { void afterFeature(FeatureResult result); }
And in the Runner the parallel method can accept the implentation of this interface.

public static Results parallel(String tagSelector, List<Resource> resources, String scenarioName, Collection<ExecutionHook> hooks, int threadCount, String reportDir, RunnerCallback callback) {}
And when a feature is executed Runner would call the method afterFeatture() to give back the control back to calling method. Here we can perform another tasks i.e., sending results to ELK
// call the call back. if (callback != null) callback.afterFeature(result); latch.countDown();
What do you think about it? Please let me know. If it make sense I can send you a Pull Request.

Regards
Prashant

[ptrthomas]

@pshrm first can you explore this: https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/core/ExecutionHook.java

this has already evolved to become an un-documented interceptor. it may be missing feature hooks which we can add if you need them. I'd like to take time to see some examples so yes - do submit a draft PR first.

[ptrthomas]

@pshrm sorry my bad, you are already aware of ExecutionHook. I'll take a look. plan is to create a "builder" of some sort for the runner, it is long overdue :)

[ptrthomas]

@pshrm and I'm proposing that the ExecutionHook can be updated to add beforeFeature and afterFeature and heck why not add a beforeSuite and afterSuite while we are at it.

I'd like to do this right and perhaps align other infra in Karate into this - so I hope you can be patient and maybe come up with some ideas in the PR

[pshrm]

@ptrthomas Great! thanks a lot. beforeSuite and afterSuite would be useful too. i.e., if one wants to clean up reports dir (in our case we do) after the suite is executed. I will definitely explore the code and try to propose a PR. Keep up the good work.

[pshrm]

@ptrthomas I gave it an attempt on Runner builder and ExecutionHook for beforeFeature and afterFeature.
Please refer to attached zip. I'm not sure if I can modify Runner.java as it may be breaking changes for other users hence created builder class RunnerBuilder.java.
This is just to give some ideas and may not be perfect (or production like code).
What do you think?
karate-core.zip

[ptrthomas]

@pshrm it will be easier to review if you follow the standard git / PR approach, but I'll try take a look later

[pshrm]

@ptrthomas no worries. I will send you a standard git/PR.

[pshrm]

@ptrthomas opened a draft PR #865. Please have a look.

[ptrthomas]

@pshrm I decided to re-work some of it - can you review this commit ? f848c0b

[pshrm]

@ptrthomas looks good. I hope you will add feature hooks as well later. I've created Junit parallel runner which notifies the IDE with test results. if you wish I can send another PR. Please let me know.

[ptrthomas]

@pshrm yes, I will take care of the feature hooks. I will be happy to look at your JUnit parallel work. as you have seen I tend to do things "my way" but your work certainly helps give me ideas and push me to do the stuff needed ^_^

[pshrm]

Hi @ptrthomas, here is PR for Junit Parallel runner. I just did it for Junit4 - #871.
Please review and let me know your thoughts.

[ptrthomas]

released 0.9.5