[I] [BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources [gluten]

["LuciferYang (via GitHub)" <gi...@apache.org>]

LuciferYang opened a new issue, #12933:
URL: https://github.com/apache/gluten/issues/12933

   ### What happens
   
   Both profiles in the root pom declare a `build-helper-maven-plugin` execution with the id `add-iceberg-test-sources`: `pom.xml:1861` under `iceberg`, `pom.xml:1912` under `iceberg-test`. Maven merges plugin executions by id, so when both profiles are active only one `<sources>` list survives.
   
   `help:effective-pom` with `-Pbackends-clickhouse -Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test` resolves that execution to:
   
   ```
   add-iceberg-test-sources
       src-iceberg-spark35/test/scala
       src-iceberg-spark35/test/java
   ```
   
   The `iceberg` profile's four entries (`src-iceberg/test/{scala,java}` and `src-iceberg${iceberg.binary.version}/test/{scala,java}`) are gone. No warning is printed; the sources are simply not compiled.
   
   ### Effect on ClickHouse: the combination cannot build
   
   `ClickHouseIcebergHiveTableSupport` lives in `backends-clickhouse/src-iceberg/test/scala`, and both Java tests under `src-iceberg-spark35/test/java` instantiate it. The class therefore leaves the source path at exactly the moment the tests join it:
   
   ```
   $ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut -Pdelta \
         -Piceberg -Piceberg-test -DskipTests test-compile -pl backends-clickhouse
   [ERROR] .../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18] cannot find symbol
   [ERROR]   symbol:   class ClickHouseIcebergHiveTableSupport
   ```
   
   Giving the `iceberg-test` execution a distinct id (`add-iceberg-spark-test-sources`) and changing nothing else makes the same command exit 0 and emit both `ClickHouseIcebergHiveTableSupport.class` and `TestPositionDeletesTableGluten.class`.
   
   ### Effect on Velox: harmless today, which is why nobody noticed
   
   Nothing in `backends-velox` needs both directories at once. `src-iceberg/test` holds `VeloxIcebergSuite` and `VeloxTPCHIcebergSuite`, which run in the jobs that pass `-Piceberg` without `-Piceberg-test` (`velox_backend_x86.yml:941` and `:1013`, whose `wildcardSuites` includes `org.apache.gluten`), plus a `@EnhancedFeaturesTest` variant that `velox_backend_enhanced.yml` covers, a JUnit test surefire picks up, and one helper object. The 52 files under `src-iceberg-spark34/test` run in the three groups at `:817`, `:821`, `:825` that pass both profiles.
   
   ### Suggested fix
   
   Rename the `iceberg-test` profile's execution so the two lists no longer overwrite each other. Both profiles are always passed together in every in-repo invocation, so the merged list is what the plugin was meant to produce.
   
   Two things a reviewer will want to know about the blast radius. The three x86 groups and the 4.0 `test-compile` at `:1277` would newly compile `src-iceberg/test/**`, so those suites run twice across the matrix; expect longer jobs rather than new failures. And `src-iceberg${iceberg.binary.version}/test` is empty in every module, so nothing else appears on the source path.
   
   Only `backends-velox` and `backends-clickhouse` hold files under `src-iceberg*/test`, so the change is confined to those two modules.
   
   This blocks #12902's follow-up work on the ClickHouse Iceberg Java tests, tracked separately.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources [gluten]

["jackylee-ch (via GitHub)" <gi...@apache.org>]

jackylee-ch closed issue #12933: [BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources
URL: https://github.com/apache/gluten/issues/12933


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org