You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "comphead (via GitHub)" <gi...@apache.org> on 2026/01/16 20:12:03 UTC
[I] Run Spark tests with Comet native runner [datafusion-comet]
comphead opened a new issue, #3209:
URL: https://github.com/apache/datafusion-comet/issues/3209
### What is the problem the feature request solves?
Comet native writer works for basic ops, however to move forward and keep compatibility with Spark we need to run Spark writer and partition discovery tests with Comet writer enabled first to make sure we have a solid reliable base for Comet native writer changes
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Run Spark tests with Comet native writer [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #3209:
URL: https://github.com/apache/datafusion-comet/issues/3209#issuecomment-3832043007
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Run Spark tests with Comet native writer [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #3209:
URL: https://github.com/apache/datafusion-comet/issues/3209#issuecomment-3824912397
Updated Spark 4 diff file (ignoring tests) .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Run Spark tests with Comet native writer [datafusion-comet]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #3209: Run Spark tests with Comet native writer
URL: https://github.com/apache/datafusion-comet/issues/3209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Run Spark tests with Comet native runner [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #3209:
URL: https://github.com/apache/datafusion-comet/issues/3209#issuecomment-3796142279
Looking into this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Run Spark tests with Comet native writer [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #3209:
URL: https://github.com/apache/datafusion-comet/issues/3209#issuecomment-3839294292
Ignored Tests for Comet Native Parquet Writer
```
Common across all Spark versions (3.4, 3.5, 4.0) - 12 tests:
1. empty file should be skipped while write to file
2. INSERT INTO TABLE - complex type but different names
3. Insert overwrite table command should output correct schema: basic
4. parquet timestamp conversion
5. SPARK-29174 Support LOCAL in INSERT OVERWRITE DIRECTORY to data source
6. SPARK-33901: ctas should should not change table's schema
7. SPARK-37160: CREATE TABLE AS SELECT with CHAR_AS_VARCHAR
8. SPARK-38336 INSERT INTO statements with tables with default columns: positive tests
9. SPARK-38811 INSERT INTO on columns added with ALTER TABLE ADD COLUMNS: Positive tests
10. SPARK-43071: INSERT INTO from queries whose final operators are not projections
11. write path implements onTaskCommit API correctly
12. Write Spark version into Parquet metadata
Spark 4.0 only - 2 additional tests:
1. ctas with union
2. SPARK-48817: test multi inserts
Total: 12 tests (Spark 3.4/3.5) | 14 tests (Spark 4.0)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org