Duplicated test execution while using cucumber.features property

[fslevoaca-ionos]

It seems tests are executed twice via the cucumber.features property.

Project to reproduce the issue: https://github.com/fslev/cucumber-showcase

Steps to reproduce the behavior:

1. Run Cucumber test scenarios using the cucumber.features property

mvn clean test -Dcucumber.features=classpath:io/example/Feature1.feature:3

Actual:

Running scenario: Feature 1 - Scenario 1_1
Running scenario: Feature 1 - Scenario 1_1

Scenario 1_1 is executed twice.

While using tags:

mvn clean verify -Dcucumber.filter.tags="@1"

... scenario is executed correctly, only once.

Thanks

[mpkorstanje]

In your project Cucumber gets two test discovery/execution requests. One when Maven makes a discovery request. And a second one through the JUnit Platform Suite engine. Normally the request made by Maven fails to discover any tests because Maven only requests the discovery of classes. However by setting the cucumber.features it will discover something.

So when using cucumber.features you must ensure that there is only a single execution of the Cucumber Engine.

Ultimately this comes down to a lack of proper support for the JUnit Platform in Maven (junit-team/junit5#2849) and IDEA. If that existed you would not have to use the JUnit Platform Suite Engine to run Cucumber.

Though I'm surprised you didn't see this warning:

cucumber-jvm/cucumber-junit-platform-engine/src/main/java/io/cucumber/junit/platform/engine/DiscoverySelectorResolver.java

Lines 39 to 44 in b53204e

	log.warn(
	() -> "Discovering tests using the " + FEATURES_PROPERTY_NAME
	+ " property. Other discovery selectors are ignored!\n" +
	"Please request/upvote/sponsor/ect better support for JUnit 5 discovery selectors.\n" +
	"See: https://github.com/cucumber/cucumber-jvm/pull/2498");
	}

Duplicate of #2567, #2498.