(texera) 01/01: test(amber): move live-catalog iceberg specs to the integration suite (#6045)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6045-baca3f9ed5d2fda7dbf047aa6641b62605a61d87
in repository https://gitbox.apache.org/repos/asf/texera.git

commit 11837ed502e42af91363d676e500d63862c410b2
Author: Meng Wang <me...@uci.edu>
AuthorDate: Wed Jul 1 12:16:28 2026 -0700

    test(amber): move live-catalog iceberg specs to the integration suite (#6045)
    
    ### What changes were proposed in this PR?
    
    Three Iceberg specs — `IcebergDocumentSpec`,
    `IcebergDocumentConsoleMessagesSpec`, and `IcebergTableStatsSpec` —
    resolve a **live** Iceberg catalog in their per-test setup
    (`beforeEach`/`beforeAll` create documents through `DocumentFactory` /
    `IcebergCatalogInstance.getInstance()`), so none of their cases can run
    without a catalog. Today the amber unit job runs them against the
    postgres catalog it provisions; once that job stops provisioning one
    they would fail.
    
    This PR moves them from `common/workflow-core/src/test/scala/...` to
    `amber/src/test/integration/...` — where the `@IntegrationTest` filter
    (`AMBER_TEST_FILTER`) already applies and
    `IcebergRestCatalogIntegrationSpec` already lives — and tags the three
    concrete specs `@IntegrationTest`.
    
    Tagging in place is not sufficient: `common/workflow-core/build.sbt`
    does not read `AMBER_TEST_FILTER`, so `@IntegrationTest` would be
    ignored there.
    
    The shared `VirtualDocumentSpec` base (extended by the first two specs)
    moves with them, into `amber/src/test/scala` — it is a generic
    `AnyFlatSpec` harness rather than an integration test, so it lives in
    amber's regular test sources, not the integration set. (Keeping it in
    `common` and exposing it via `WorkflowCore % "test->test"` was tried but
    reverted: WorkflowCore's test scope pulls Hadoop → `com.sun.jersey`
    1.19, whose bundled JAX-RS 1.x `Response` shadows amber's
    `javax.ws.rs-api:2.1.1` and breaks `HuggingFaceModelResourceSpec`.
    Longer term, generalizing `amber-integration` into a shared
    `integration` CI job would let the specs live in `common` next to the
    code they exercise.)
    
    **`IcebergUtilSpec` intentionally stays in `common/workflow-core`.**
    Unlike the three above it `extends AnyFlatSpec` with no catalog setup
    and never calls `getInstance()`; its cases are pure Texera↔Iceberg
    type/schema/tuple conversions plus one REST-endpoint failure check whose
    own comment documents it as passing "with or without Lakekeeper
    reachable." It is a unit test and must keep running in the amber unit
    job.
    
    ### Any related issues, documentation, discussions?
    
    Part of #6034 (Task 1 of 3). Does not close it — Tasks 2 (flip the
    `storage.conf` catalog default to `rest`) and 3 (mirror prod in the CI
    integration suite) follow, and Task 1 must land first.
    
    ### How was this PR tested?
    
    Test-file relocation with no production-code change, so verification is
    at the compile/placement level:
    
    - `sbt WorkflowExecutionService/Test/compile` and `sbt
    WorkflowCore/Test/compile` pass locally (the specs move cleanly and
    amber's test scope compiles).
    - `sbt scalafmtCheckAll` passes.
    - `build / amber` and `build / amber-integration` (ubuntu + macos) are
    green on CI, compiling the integration source set and running the specs
    against a live catalog.
    - Behavior is unchanged; the three specs continue to run in the
    `amber-integration` job.
    
    ### Was this PR authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (claude-opus-4-8)
---
 .../storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala     | 2 ++
 .../texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala       | 2 ++
 .../texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala     | 2 ++
 .../apache/texera/amber/core/storage/model/VirtualDocumentSpec.scala    | 0
 4 files changed, 6 insertions(+)

diff --git a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala
similarity index 97%
rename from common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala
rename to amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala
index 4a1827b091..55a743255a 100644
--- a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala
+++ b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentConsoleMessagesSpec.scala
@@ -24,6 +24,7 @@ import org.apache.texera.amber.core.storage.result.ResultSchema
 import org.apache.texera.amber.core.storage.{DocumentFactory, VFSURIFactory}
 import org.apache.texera.amber.core.tuple.{Schema, Tuple}
 import org.apache.texera.amber.core.virtualidentity._
+import org.apache.texera.amber.tags.IntegrationTest
 import org.scalatest.BeforeAndAfterAll
 import org.scalatest.flatspec.AnyFlatSpec
 import org.scalatest.matchers.should.Matchers
@@ -32,6 +33,7 @@ import java.net.URI
 import scala.util.Using.Releasable
 import scala.util.{Try, Using}
 
+@IntegrationTest
 class IcebergDocumentConsoleMessagesSpec
     extends AnyFlatSpec
     with Matchers
diff --git a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala
similarity index 99%
rename from common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala
rename to amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala
index d21644f6e6..78999c7ea3 100644
--- a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala
+++ b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergDocumentSpec.scala
@@ -32,6 +32,7 @@ import org.apache.texera.amber.core.virtualidentity.{
   WorkflowIdentity
 }
 import org.apache.texera.amber.core.workflow.{GlobalPortIdentity, PortIdentity}
+import org.apache.texera.amber.tags.IntegrationTest
 import org.apache.texera.amber.util.IcebergUtil
 import org.apache.iceberg.catalog.Catalog
 import org.apache.iceberg.data.Record
@@ -44,6 +45,7 @@ import java.sql.Timestamp
 import java.util.UUID
 import java.util.concurrent.atomic.AtomicInteger
 
+@IntegrationTest
 class IcebergDocumentSpec extends VirtualDocumentSpec[Tuple] with BeforeAndAfterAll {
 
   var amberSchema: Schema = _
diff --git a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala
similarity index 99%
rename from common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala
rename to amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala
index 956e9f42b9..868574abb9 100644
--- a/common/workflow-core/src/test/scala/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala
+++ b/amber/src/test/integration/org/apache/texera/amber/storage/result/iceberg/IcebergTableStatsSpec.scala
@@ -29,6 +29,7 @@ import org.apache.texera.amber.core.virtualidentity.{
   WorkflowIdentity
 }
 import org.apache.texera.amber.core.workflow.{GlobalPortIdentity, PortIdentity}
+import org.apache.texera.amber.tags.IntegrationTest
 import org.apache.texera.amber.util.IcebergUtil
 import org.apache.iceberg.catalog.Catalog
 import org.apache.iceberg.data.Record
@@ -42,6 +43,7 @@ import java.time.format.DateTimeFormatter
 import java.time.{LocalDate, ZoneId}
 import java.util.UUID
 
+@IntegrationTest
 class IcebergTableStatsSpec extends AnyFlatSpec with BeforeAndAfterAll with Suite {
 
   var amberSchema: Schema = _
diff --git a/common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/model/VirtualDocumentSpec.scala b/amber/src/test/scala/org/apache/texera/amber/core/storage/model/VirtualDocumentSpec.scala
similarity index 100%
rename from common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/model/VirtualDocumentSpec.scala
rename to amber/src/test/scala/org/apache/texera/amber/core/storage/model/VirtualDocumentSpec.scala