Re: [PR] fix(infra/testing): Enable iceberg spark interops integration tests on ci pipeline [iceberg-go]

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #493:
URL: https://github.com/apache/iceberg-go/pull/493#discussion_r2255115031


##########
internal/recipe/Dockerfile:
##########
@@ -20,7 +20,7 @@ RUN pip3 install pyiceberg[s3fs,hive]
 RUN pip3 install pyarrow
 
 COPY provision.py .
-COPY run_spark_count_sql.py .
+COPY validation.py .
 
 ENTRYPOINT ["./entrypoint.sh"]
 CMD ["notebook"]

Review Comment:
   add empty newline



##########
internal/recipe/local_spark.go:
##########
@@ -19,18 +19,24 @@ package recipe
 
 import (
 	"bytes"
-	_ "embed"

Review Comment:
   why rearranging the imports?



##########
internal/recipe/local_spark.go:
##########
@@ -62,3 +68,68 @@ func Start(t *testing.T) (*compose.DockerCompose, error) {
 
 	return stack, nil
 }
+
+func ExecuteSpark(t *testing.T, scriptPath string, args ...string) (string, error) {
+	cli, err := client.NewClientWithOpts(
+		client.FromEnv,
+		client.WithVersion("1.48"), // Use explicit version to avoid compatibility issues

Review Comment:
   Should this be an env var?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org