[GitHub] [iceberg] wypoon commented on a change in pull request #3415: Spark: Add separate modules for Spark 3.1

[GitBox <gi...@apache.org>]

wypoon commented on a change in pull request #3415:
URL: https://github.com/apache/iceberg/pull/3415#discussion_r739581775



##########
File path: .github/workflows/spark-ci.yml
##########
@@ -99,6 +99,33 @@ jobs:
         path: |
           **/build/testlogs
 
+  spark-31-tests:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        jvm: [8, 11]
+        spark: ['3.1']
+    env:
+      SPARK_LOCAL_IP: localhost
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-java@v1
+        with:
+          java-version: ${{ matrix.jvm }}
+      - uses: actions/cache@v2
+        with:
+          path: ~/.gradle/caches
+          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
+          restore-keys: ${{ runner.os }}-gradle
+      - run: echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
+      - run: ./gradlew -DsparkVersions=${{ matrix.spark }} -DhiveVersions= -DflinkVersions= :iceberg-spark:iceberg-spark-3.1:check :iceberg-spark:iceberg-spark-3.1-extensions:check :iceberg-spark:iceberg-spark-3.1-runtime:check -Pquick=true -x javadoc
+      - uses: actions/upload-artifact@v2
+        if: failure()
+        with:
+          name: test logs
+          path: |
+            **/build/testlogs

Review comment:
       Ok, I'll try that. I'm not familiar with the CI. What is not clear to me is:
   Suppose we have two values in
   ```
           spark: ['3.1', '3.2']
   ```
   instead of the one value
   ```
           spark: ['3.1']
   ```
   then in the
   ```
         - run: ./gradlew -DsparkVersions=${{ matrix.spark }} ...
   ```
   does that get run *twice*, each time with one of the values? Or is that run once, with two values in `sparkValues` (because sparkVersions can take a comma-separated list of versions)?
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org