You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/03/17 19:25:38 UTC
[I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
andygrove opened a new issue, #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542
### Describe the bug
This issue is to track Spark SQL test failures in native_iceberg_compat mode.
- Comet tries to read JSON files with Parquet reader
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2744657118
Let me look into this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2769901472
```
catalyst: Passed: Total 6925, Failed 0, Errors 0, Passed 6925, Ignored 5, Canceled 1
core 1: Failed: Total 8686, Failed 47, Errors 0, Passed 8639, Ignored 277, Canceled 3
core 2: Failed: Total 2045, Failed 106, Errors 0, Passed 1939, Ignored 360
core 3: Failed: Total 1394, Failed 24, Errors 0, Passed 1370, Ignored 119, Canceled 15
hive 1: Failed: Total 2144, Failed 9, Errors 0, Passed 2135, Ignored 38, Canceled 4
hive 2: Error: Total 19, Failed 0, Errors 1, Passed 18, Ignored 1, Canceled 4
hive 3: Passed: Total 1044, Failed 0, Errors 0, Passed 1044, Ignored 13, Canceled 4
```
Counts from https://github.com/apache/datafusion-comet/pull/1541 today. Will track here as we keep updating.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2744045497
I'm looking into the core3 `row index generation` errors. At least one of them is failing with NPE in Comet code:
```
Caused by: java.lang.NullPointerException
at org.apache.comet.parquet.NativeBatchReader.nextBatch(NativeBatchReader.java:427)
at org.apache.comet.parquet.NativeBatchReader.nextKeyValue(NativeBatchReader.java:373)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2898017079
Failure count:
```
core-1: Failed: Total 9216, Failed 17, Errors 0, Passed 9199, Ignored 214, Canceled 6
core-2: Failed: Total 2655, Failed 19, Errors 0, Passed 2636, Ignored 387
hive-1: Failed: Total 2174, Failed 3, Errors 0, Passed 2171, Ignored 14, Canceled 4
```
Total failures: 39 (down from 48)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2730625465
@parthchandra @mbutrovich fyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "parthchandra (via GitHub)" <gi...@apache.org>.
parthchandra commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2845276570
Failure count :
```
Core 1: Tests: succeeded 9113, failed 25, canceled 6, ignored 292, pending 0
Core 2: Tests: succeeded 2636, failed 19, canceled 0, ignored 387, pending 0
Hive 1: Tests: succeeded 2144, failed 4, canceled 4, ignored 40, pending 0
```
Total failures: 48 (down from 176)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #1542: Spark SQL test failures in native_iceberg_compat mode
URL: https://github.com/apache/datafusion-comet/issues/1542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1542:
URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2744057076
@parthchandra I made some notes about on the `core3` failures. Do you know how we should handle missing columns in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org