You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2025/11/15 15:13:41 UTC
[I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
manuzhang opened a new issue, #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788
### Describe the bug
As in https://github.com/apache/iceberg/pull/14591, build in JDK 11 failed
```
> Task :iceberg-spark:iceberg-spark-3.5_2.12:compileScala
javac exited with exit code 1
Error: /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:23:32: error: cannot access AbstractColumnReader
Error: /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:24:32: error: cannot access ColumnReader
Error: /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:25:32: error: cannot access TypeUtil
Error: /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:26:32: error: cannot access Utils
```
It looks to be broken by https://github.com/apache/datafusion-comet/pull/2514
cc @parthchandra @andygrove
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3559650577
Tried with jdk 11 and you're right the build does fail. It succeeds with jdk 17. I don't see an obvious reason for the failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang closed issue #2788: iceberg failed to build with datafusion-comet 0.11.0 in JDK 11
URL: https://github.com/apache/datafusion-comet/issues/2788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3554130313
I don't think we can upgrade to Comet 0.11.0 unless https://github.com/apache/iceberg/pull/13786 is merged.
FWIW, iceberg with the aforementioned PR builds successfully with Comet 0.11.0 for all versions of Spark in my local build. I did use JDK 17 though because Spark 4 support requires it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3555397029
TBH, I think it is better if we do not update Comet in Iceberg until the PR-13786 is merged. Without it, Comet-Iceberg is broken and updating Comet version simply gives the impression that all is fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3782015911
Iceberg build can [pass with datafusion-comet 0.12.0](https://github.com/apache/iceberg/pull/15105) with removal of JDK 11.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3555122553
@parthchandra The error was thrown when building iceberg-spark-3.5 and iceberg-spark-4.0 is built with JDK17+. I see https://github.com/apache/iceberg/pull/13786 is required, but the build should pass at least.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] iceberg failed to build with datafusion-comet 0.11.0 in JDK 11 [datafusion-comet]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #2788:
URL: https://github.com/apache/datafusion-comet/issues/2788#issuecomment-3555473707
@parthchandra That makes sense. I'm resolving and asking about the build issue. I will rebase the PR after [apache/iceberg#13786](https://github.com/apache/iceberg/pull/13786) is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org