You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "anew (via GitHub)" <gi...@apache.org> on 2026/06/10 16:57:39 UTC
[PR] [CONNECT][SDP][SPARK-57375] Fix flaky AutoCdcScd1SchemaEvolutionSuite [spark]
anew opened a new pull request, #56434:
URL: https://github.com/apache/spark/pull/56434
### What changes were proposed in this pull request?
All AutoCDC E2E test suites share a single static
SharedTablesInMemoryRowLevelOperationTableCatalog.sharedTables map and previously used the same namespace "ns1". When SBT runs suites in parallel within the same JVM (parallelExecution = true), one suite's afterEach calling reset() (which clears the entire map) destroys tables that another concurrent suite is actively using, causing flaky failures where stale data survives full refreshes or auxiliary state persists.
Fix:
- Give each suite a unique namespace derived from its class name so table Identifier keys never collide across suites.
- Replace the global reset() in afterEach with a targeted clearNamespace() that removes only this suite's entries from the shared map.
- Compute the ExpectedContext stop position dynamically in AutoCdcScd1SchemaEvolutionSuite so it adapts to the longer namespace.
### Why are the changes needed?
To fix a flaky test that often fails the CI
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
It fixes a test
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Opus 4.6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [CONNECT][SDP][SPARK-57375] Fix flaky AutoCdcScd1SchemaEvolutionSuite [spark]
Posted by "anew (via GitHub)" <gi...@apache.org>.
anew closed pull request #56434: [CONNECT][SDP][SPARK-57375] Fix flaky AutoCdcScd1SchemaEvolutionSuite
URL: https://github.com/apache/spark/pull/56434
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [CONNECT][SDP][SPARK-57375] Fix flaky AutoCdcScd1SchemaEvolutionSuite [spark]
Posted by "anew (via GitHub)" <gi...@apache.org>.
anew commented on PR #56434:
URL: https://github.com/apache/spark/pull/56434#issuecomment-4674657805
This (AI-suggested) fix did not work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org