[GitHub] [pulsar] lhotari edited a comment on pull request #9672: [Testing] Improve integration test base classes

[GitBox <gi...@apache.org>]

lhotari edited a comment on pull request #9672:
URL: https://github.com/apache/pulsar/pull/9672#issuecomment-784122330


   @merlimat It seems that the original tests are invalid. 
   
   In this case, the TestNG test suite is defined this way in `tests/integration/src/test/resources/pulsar-backwards-compatibility.xml`:
   ```
   <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
   <suite name="Pulsar Messaging Backwards Compatibility Tests" verbose="2" annotations="JDK">
       <test name="messaging-backwards-compatibility-suite" preserve-order="true">
           <classes>
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.SmokeTest2_2" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.SmokeTest2_3" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.SmokeTest2_4" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.SmokeTest2_5" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.ClientTest2_2" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.ClientTest2_3" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.ClientTest2_4" />
               <class name="org.apache.pulsar.tests.integration.backwardscompatibility.ClientTest2_5" />
           </classes>
       </test>
   </suite>
   ```
   
   What happens in the original code (before this PR) is that the containers get initialized for all test classes up-front. Since each test class uses BeforeSuite to start the containers and the container is held in a static field, this results in the last container to be initialized to be used for all test runs.
   
   I verified this behavior locally. Here is the output of `docker ps` when the first test starts executing:
   ```
   ❯ docker ps
   CONTAINER ID   IMAGE                       COMMAND                  CREATED          STATUS          PORTS                                              NAMES
   294c5cf14f6e   apachepulsar/pulsar:2.5.0   "bin/pulsar standalo…"   14 seconds ago   Up 14 seconds   0.0.0.0:49259->6650/tcp, 0.0.0.0:49258->8080/tcp   immhznye-standalone
   d83abdf6798b   apachepulsar/pulsar:2.4.0   "bin/pulsar standalo…"   24 seconds ago   Up 24 seconds   0.0.0.0:49257->6650/tcp, 0.0.0.0:49256->8080/tcp   ntjmaswt-standalone
   1135a3fdc63a   apachepulsar/pulsar:2.3.0   "bin/pulsar standalo…"   34 seconds ago   Up 34 seconds   0.0.0.0:49255->6650/tcp, 0.0.0.0:49254->8080/tcp   ehkeslmc-standalone
   28d87c681157   apachepulsar/pulsar:2.2.0   "bin/pulsar standalo…"   44 seconds ago   Up 44 seconds   0.0.0.0:49253->6650/tcp, 0.0.0.0:49252->8080/tcp   idpnqdfo-standalone
   f0404f5eb71c   testcontainers/ryuk:0.3.0   "/app"                   45 seconds ago   Up 44 seconds   0.0.0.0:49251->8080/tcp                            testcontainers-ryuk-23a90d7b-ca18-49c7-ace2-f86a719d1311
   ```
   
   Therefore, I think that deleting the failing test (SmokeTest2_2.testBatchIndexAckDisabled) is the way to resolve this. 
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org