You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "mattcasters (via GitHub)" <gi...@apache.org> on 2026/07/13 16:33:07 UTC

[I] [Bug]: Fix failing integration tests (hop)

mattcasters opened a new issue, #7516:
URL: https://github.com/apache/hop/issues/7516

   ### Apache Hop version?
   
   2.19.0-SNAPSHOT
   
   ### Java version?
   
   OpenJDK 21
   
   ### Operating system
   
   Linux
   
   ### What happened?
   
   19 failed tests for a variety of reasons:
   
   https://ci-builds.apache.org/job/Hop/job/Hop-integration-tests/lastCompletedBuild/testReport/
   
   
   ### Issue Priority
   
   Priority: 1
   
   ### Issue Component
   
   Component: Other


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: Fix failing integration tests (hop)

Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters commented on issue #7516:
URL: https://github.com/apache/hop/issues/7516#issuecomment-4960308131

   ## Plan to fix the 19 failing ITs (#2236)
   
   Daily cycle is ~2h54m (down from ~4h15m) after #7506 (single-JVM suite runner + Surefire Report Output + WorkflowExecutor keeps input fields on the execution-result stream). Most new failures are age 2 since that landed.
   
   ### Failure groups
   
   | Group | Count | Symptom | Likely cause |
   | --- | --- | --- | --- |
   | **A. HTTP** | 11 | `localhost:80` connection refused | Nested runner: `HOSTNAME` falls back to parameter default instead of env `http` |
   | **B. Kafka** | 3 | empty / unresolvable `bootstrap.servers` | Nested runner: `BOOTSTRAP_SERVERS` not reaching child workflows |
   | **C. AES** | 1 | decrypt yields empty plaintext | #7497 re-binds encoder from hop-config (`Hop`) over docker AES2 env |
   | **D. WorkflowExecutor** | 1 (`transforms.0083`) | output fields include input `f1` | #7506 intentionally prepends input fields on execution-result stream |
   | **E. Beam** | 1 (`suite_startup`) | runner runs as Beam; missing `filename` | Single-JVM runner incompatible with Beam project run config |
   | **F. Env override** | 1 (`parameters…0012`) | `TEST_0012` stays default | Env vars not winning under nested/single-JVM path |
   | **G. Samples** | 1 (age 41) | chronic 30m suite | Pre-existing; partially addressed by #7508 |
   
   ### Approach (prefer fix runner/contracts; keep single-JVM speedup)
   
   1. **Confirm** with docker ITs (`CLIENT_UNZIP=false` against the built client):
      ```bash
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh http
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh kafka
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh aes
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh transforms
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh parameters_and_variables
      CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh beam_directrunner
      ```
      Check whether `HOP_IT_PER_TEST_JVM=true` clears each failure (runner nesting vs product bug).
   
   2. **Fix variable/parameter plumbing** for A/B/F: extend `run-tests.sh` / `run-workflow.hwf` so env hosts and suite params reach nested `main-*.hwf` (HOSTNAME, BOOTSTRAP_SERVERS, CASTLEMOCK_HOSTNAME, etc.).
   
   3. **Update 0083** expected field list for the new WorkflowExecutor schema (D) — do not revert #7506.
   
   4. **Beam (E):** force native local for the suite runner, or use per-test JVM only for Beam projects.
   
   5. **AES (C):** put `HOP_PASSWORD_ENCODER_PLUGIN=AES2` + `HOP_AES_ENCODER_KEY` in the aes project/env config so project enable does not wipe docker env.
   
   6. **Samples (G):** find first failing sample in the long log; ignore external-only samples or fix product as needed.
   
   ### Suggested PR split
   
   - **PR1:** runner plumbing (A/B/F) + 0083 (D)
   - **PR2:** Beam runner isolation (E)
   - **PR3:** AES project encoder config (C)
   - **PR4:** samples (G)
   
   Escape hatch: `HOP_IT_PER_TEST_JVM=true` only where a suite truly cannot share a JVM (e.g. Beam) — avoid a global flip that would undo the ~1h savings.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: Fix failing integration tests (hop)

Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters closed issue #7516: [Bug]: Fix failing integration tests
URL: https://github.com/apache/hop/issues/7516


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org