Create tests to verify that Startup and Shutdown events are fired

[tevans78]

I can not find any CDI 4.0.0 TCK tests that actually validate that the new Startup and Shutdown events are correctly fired.
The events were added to the API and spec here: jakartaee/cdi#543

At the moment in 4.0.0, there is a single reference to Startup in a BCE test but I do not think that test checks the event was fired, only that an observer is present
https://github.com/eclipse-ee4j/cdi-tck/blob/d78a11a2dabac738f9bfa1bc3a82a6ae1f3df52c/impl/src/main/java/org/jboss/cdi/tck/tests/build/compatible/extensions/registration/MyServiceFoo.java#L14

[Ladicek]

I vaguely recall we debated that for a bit and thought it's probably one of those things that are really hard to test. At least in case of Shutdown. I don't recall why we didn't add a test for Startup -- we couldn't really verify when exactly it was fired, but we should be able to verify that it was fired. @manovotn do you perhaps remember?

[manovotn]

Hmm, I don't remember. Maybe this was simply forgotten it the barrage of API and TCK changes? I really can't imagine why we wouldn't add any test whatsoever.

We should be able to test both event for CDI SE.
However, outside of that (and therefore for just Lite), we can only assert Startup unless we create an observer for Shutdown
that would blow up the test and we assert that exception.

Any volunteers to write that test? We can add it as part of the next release (some SP1 or however we call those).

[starksm64]

I have assigned this to myself and will look at adding some tests.

[starksm64]

I added a Startup event observer to the org.jboss.cdi.tck.tests.event.observer.priority.contextLifecycleEvent.ApplicationScopedObserver class and updated the ApplicationContextLifecycleEventObserverOrderingTest to verify that a Startup event was seen, and it is not in Weld 5.0.0.CR1. I checked out the 5.0.0.CR1 tag, and updated the jboss-tck-runner pom.xml to use the 4.0.1-SNAPSHOT cdi-tck-core-impl tests, and ran against a startup-suite.xml file containing only the org.jboss.cdi.tck.tests.event.observer.priority.contextLifecycleEvent.* package:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="CDI TCK" verbose="0" configfailurepolicy="continue">

  <listeners>
    <!-- Required - avoid randomly mixed test method execution -->
    <listener class-name="org.jboss.cdi.tck.impl.testng.SingleTestClassMethodInterceptor" />
    <!-- Optional - intended for debug purpose only -->
    <listener class-name="org.jboss.cdi.tck.impl.testng.ConfigurationLoggingListener"/>
    <listener class-name="org.jboss.cdi.tck.impl.testng.ProgressLoggingTestListener"/>
    <!-- Optional - it's recommended to disable the default JUnit XML reporter -->
    <listener class-name="org.testng.reporters.SuiteHTMLReporter"/>
    <listener class-name="org.testng.reporters.FailedReporter"/>
    <listener class-name="org.testng.reporters.XMLReporter"/>
    <listener class-name="org.testng.reporters.EmailableReporter"/>
    <listener class-name="org.testng.reporters.TestHTMLReporter" />
  </listeners>

  <test name="CDI TCK Startup Event">

    <packages>
      <package name="org.jboss.cdi.tck.tests.event.observer.priority.contextLifecycleEvent.*" />
    </packages>
  </test>
</suite>

Running this produces:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.959 s <<< FAILURE! - in TestSuite
[ERROR] org.jboss.cdi.tck.tests.event.observer.priority.contextLifecycleEvent.ApplicationContextLifecycleEventObserverOrderingTest.testContextLifecycleEventOrdering  Time elapsed: 0.023 s  <<< FAILURE!
java.lang.AssertionError: expected [ABCDSu] but found [ABCD]
        at org.jboss.cdi.tck.tests.event.observer.priority.contextLifecycleEvent.ApplicationContextLifecycleEventObserverOrderingTest.testContextLifecycleEventOrdering(ApplicationContextLifecycleEventObserverOrderingTest.java:45)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ApplicationContextLifecycleEventObserverOrderingTest>Arquillian.run:138->testContextLifecycleEventOrdering:45 expected [ABCDSu] but found [ABCD]
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

[starksm64]

So if Weld 5.0.0.CR1 really does not support the Startup/Shutdown events and we have no tests in the TCK, we really should respin the 4.0.0 release and restart the current ballot.

[manovotn]

@starksm64 I am aware of it, see https://issues.redhat.com/browse/WELD-2709
I will include it in the Final release of Weld so we will have it covered there.

[Emily-Jiang]

So if Weld 5.0.0.CR1 really does not support the Startup/Shutdown events and we have no tests in the TCK, we really should respin the 4.0.0 release and restart the current ballot.

Even though Weld 5.0.0.CR1 supports the event, we still have no tests, which is enough to void the current ballot. I think I will need to go ahead to cancel the ballot @starksm64. Agree?
plus #352 needs to be addressed.

[starksm64]

I think we should reset with a Weld 5.0.0.CR2 and new 4.0.0 release. I can't make the next CDI meeting if we want to discuss there as I have an all day engineering meeting.