You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "baibaichen (via GitHub)" <gi...@apache.org> on 2026/03/02 15:04:55 UTC
[I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
baibaichen opened a new issue, #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550
Please leave a comment and mark on the TODO tests in this issue if you want to contribute
## Backend
VL(Velox)
## Bug Description
These suites were commented out in https://github.com/apache/incubator-gluten/pull/11512 with TODO markers indicating they need to be fixed and re-enabled.
## Context
The test suites below have been disabled due to various failures. Each table shows the status for both Spark 4.0 and 4.1:
- 🔴 = Suite is commented out (disabled)
- 🟢 = Suite is enabled
The failure count (when available) is shown in the status column.
---
### org.apache.spark.sql
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenDataFrameSubquerySuite` | 🟢 | 🔴 (1 failures) | | Disabled in Spark 4.1 |
| `GlutenExplainSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @Surbhi-Vijay | |
| `GlutenLogQuerySuite` | 🔴 (2 failures) | 🔴 (2 failures) | @kapilks | |
| `GlutenSetCommandSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @Surbhi-Vijay | |
| `GlutenSingleLevelAggregateHashMapSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @kapilks | |
| `GlutenSparkSessionJobTaggingAndCancellationSuite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
| `GlutenTwoLevelAggregateHashMapSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @kapilks | |
| `GlutenTwoLevelAggregateHashMapWithVectorizedMapSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @kapilks | |
| `GlutenVariantEndToEndSuite` | 🔴 (3 failures) | 🔴 (3 failures) | | |
| `GlutenVariantShreddingSuite` | 🔴 (8 failures) | 🔴 (8 failures) | | |
| `GlutenXmlFunctionsSuite` | 🔴 (10 failures) | 🔴 (10 failures) | | |
### catalyst.expressions
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenXmlExpressionsSuite` | 🟢 | 🟢 | @baibaichen | Fixed in #11580 |
### connector
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenGroupBasedUpdateTableSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @Surbhi-Vijay | |
| `GlutenMergeIntoDataFrameSuite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
### execution
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenInsertSortForLimitAndOffsetSuite` | 🔴 (6 failures) | 🔴 (6 failures) | | |
| `GlutenPlannerSuite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
| `GlutenRemoveRedundantSortsSuite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
| `GlutenSparkPlanSuite` | 🔴 (1 failures) | 🔴 (1 failures) | @Surbhi-Vijay | |
| `GlutenWholeStageCodegenSparkSubmitSuite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
| `GlutenWholeStageCodegenSuite` | 🔴 (24 failures) | 🔴 (24 failures) | @Surbhi-Vijay | |
### execution.joins
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenSingleJoinSuite` | 🟢 | 🟢 | @baibaichen | Fixed in #11577 |
### execution.datasources.parquet
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenParquetTypeWideningSuite` | 🔴 (74 failures) | 🔴 (74 failures) | | |
| `GlutenParquetVariantShreddingSuite` | 🟢 | 🔴 (1 failures) | | Disabled in Spark 4.1 |
### execution.datasources.text
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenWholeTextFileV1Suite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
| `GlutenWholeTextFileV2Suite` | 🔴 (1 failures) | 🔴 (1 failures) | | |
### execution.python
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenPythonDataSourceSuite` | 🔴 (1 failures) | 🔴 | | |
| `GlutenPythonUDFSuite` | 🔴 (1 failures) | 🔴 | | |
| `GlutenPythonUDTFSuite` | 🔴 | 🔴 | | |
| `GlutenRowQueueSuite` | 🟢 | 🔴 | | Disabled in Spark 4.1 |
### sources
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenBucketedReadWithHiveSupportSuite` | 🔴 (2 failures) | 🔴 (2 failures) | @Surbhi-Vijay | |
| `GlutenBucketedWriteWithHiveSupportSuite` | 🔴 (1 failures) | 🔴 (2 failures) | | Failure count changed: 1 → 2 |
| `GlutenCommitFailureTestRelationSuite` | 🔴 (2 failures) | 🔴 (2 failures) | | |
| `GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite` | 🔴 (2 failures) | 🔴 (2 failures) | @Surbhi-Vijay | |
| `GlutenJsonHadoopFsRelationSuite` | 🔴 (2 failures) | 🔴 (2 failures) | | |
| `GlutenParquetHadoopFsRelationSuite` | 🔴 (2 failures) | 🔴 (2 failures) | | |
| `GlutenSimpleTextHadoopFsRelationSuite` | 🔴 (2 failures) | 🔴 (2 failures) | | |
### streaming
> Most of failtures are same, it seems should be fixed in spark
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenEventTimeWatermarkSuite` | 🔴 | 🔴 | | |
| `GlutenFileStreamSourceSuite` | 🔴 | 🔴 | | |
| `GlutenFlatMapGroupsWithStateDistributionSuite` | 🔴 | 🔴 | | |
| `GlutenFlatMapGroupsWithStateSuite` | 🔴 | 🔴 | | |
| `GlutenRocksDBStateStoreFlatMapGroupsWithStateSuite` | 🔴 | 🔴 | | |
| `GlutenRocksDBStateStoreStreamingAggregationSuite` | 🔴 | 🔴 | | |
| `GlutenRocksDBStateStoreStreamingDeduplicationSuite` | 🔴 | 🔴 | | |
| `GlutenStreamSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingAggregationDistributionSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingAggregationSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingDeduplicationDistributionSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingDeduplicationSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingInnerJoinSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingOuterJoinSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingSessionWindowDistributionSuite` | 🔴 | 🔴 | | |
| `GlutenStreamingStateStoreFormatCompatibilitySuite` | 🔴 | 🔴 | | |
---
## Summary Statistics
- **Total suites disabled in Spark 4.0**: 48
- **Total suites disabled in Spark 4.1**: 51
- **Unique suites across both versions**: 51
This issue was generated based on commit 4e3df671.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen closed issue #11550: Spark 4.x: Tracking disabled test suites
URL: https://github.com/apache/gluten/issues/11550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3850658557
> Can work on `GlutenLogQuerySuite`
Thanks @kapilks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "kapilks (via GitHub)" <gi...@apache.org>.
kapilks commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3848535086
Can work on `GlutenLogQuerySuite`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen closed issue #11550: Spark 4.x: Tracking disabled test suites
URL: https://github.com/apache/incubator-gluten/issues/11550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "kapilks (via GitHub)" <gi...@apache.org>.
kapilks commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3853743554
I can take these:
GlutenSingleLevelAggregateHashMapSuite
GlutenTwoLevelAggregateHashMapSuite
GlutenTwoLevelAggregateHashMapWithVectorizedMapSuite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3858926006
Claiming `GlutenSingleJoinSuite` - fixed in PR #11577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on issue #11550:
URL: https://github.com/apache/gluten/issues/11550#issuecomment-4152603214
Update: PR #11847 (draft) adds `GlutenTestSetWithSystemPropertyTrait` and enables 6 suites (`GlutenSQLExecutionSuite`, `GlutenSQLJsonProtocolSuite`, `GlutenShufflePartitionsUtilSuite`, `GlutenExternalAppendOnlyUnsafeRowArraySuite`, `GlutenUnsafeRowSerializerSuite`, `GlutenSparkSessionJobTaggingAndCancellationSuite`). Also removes 7 TestHiveSingleton suites that cannot load GlutenPlugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-4022645728
`GlutenDataFrameSubquerySuite` (Spark 4.1) fixed in #11727
| Suite | Spark 4.0 | Spark 4.1 | Owner | Comments |
|-------|-----------|-----------|-------|----------|
| `GlutenDataFrameSubquerySuite` | 🟢 | 🟢 | @baibaichen | Fixed in #11727 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "kapilks (via GitHub)" <gi...@apache.org>.
kapilks commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3881712081
Will take GlutenInsertSortForLimitAndOffsetSuite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "Surbhi-Vijay (via GitHub)" <gi...@apache.org>.
Surbhi-Vijay commented on issue #11550:
URL: https://github.com/apache/incubator-gluten/issues/11550#issuecomment-3851148228
I'll take:
GlutenExplainSuite
GlutenSetCommandSuite
GlutenGroupBasedUpdateTableSuite
GlutenSparkPlanSuite
GlutenWholeStageCodegenSuite
GlutenBucketedReadWithHiveSupportSuite
GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen closed issue #11550: Spark 4.x: Tracking disabled test suites
URL: https://github.com/apache/incubator-gluten/issues/11550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Spark 4.x: Tracking disabled test suites [gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on issue #11550:
URL: https://github.com/apache/gluten/issues/11550#issuecomment-4229854784
## Triage update (2026-04-11)
Completed triage of all remaining TODO suites. Changes:
**Failure count corrections:**
- GlutenExplainSuite: 1 → 4 failures
- GlutenPlannerSuite: 1 → 15 failures
- GlutenRemoveRedundantSortsSuite: 1 → 5 failures
**Owner assignments (sub-issues created):**
- 20 Streaming suites → #11911
- GlutenGroupBasedUpdateTableSuite, GlutenSimpleSQLViewSuite → #11912 (JNI exception handling)
- GlutenCollationRegexpExpressionsSuite → #11913 (Velox split limit)
- GlutenColumnarRulesSuite → #11920 (dual-mode ColumnarToRow, feature)
- GlutenRandomDataGeneratorSuite, GlutenOrderingSuite, GlutenHiveResultSuite → #11919 (TimeType, feature)
- GlutenToPrettyStringSuite → #11918 (CastTransformer timezone)
- GlutenCastWithAnsiOnSuite → #10134 (ANSI mode)
- GlutenMultiStatefulOperatorsSuite → #11911 (Streaming)
**Released for community (RC):**
- GlutenExplainSuite, GlutenPlannerSuite, GlutenProjectedOrderingAndPartitioningSuite, GlutenRemoveRedundantProjectsSuite, GlutenRemoveRedundantSortsSuite — owner cleared, available for contribution
**Streaming table moved to #11911** — 20 disabled suites now tracked there.
**New feature suites tracked under #11910** (Spark 4.x feature tracking):
- TimeType (#11919), dual-mode ColumnarToRow (#11920), NullType Parquet (#11921), memory shuffle spill (#11922)
- Variant (#11371), ANSI mode (#10134) also linked
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org