You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "baibaichen (via GitHub)" <gi...@apache.org> on 2025/01/17 00:14:01 UTC

[PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

baibaichen opened a new pull request, #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555

   ## What changes were proposed in this pull request?
   
   (Please fill in changes proposed in this fix)
   
   (Fixes: \#6067)
   
   ## How was this patch tested?
   
   (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
   
   
   (If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2631533085

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2636171201

   @CodiumAI-Agent /review 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2631376464

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2651313808

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "lgbo-ustc (via GitHub)" <gi...@apache.org>.
lgbo-ustc commented on code in PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#discussion_r1951825085


##########
gluten-ut/common/src/test/scala/org/apache/gluten/utils/BackendTestSettings.scala:
##########
@@ -98,6 +98,13 @@ abstract class BackendTestSettings {
       exclusion.add(Exclude(testNames: _*))
       this
     }
+    def includeCH(testNames: String*): SuiteSettings = {
+      this
+    }

Review Comment:
   Nothing happen?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2632578804

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on code in PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#discussion_r1951850101


##########
gluten-ut/common/src/test/scala/org/apache/gluten/utils/BackendTestSettings.scala:
##########
@@ -98,6 +98,13 @@ abstract class BackendTestSettings {
       exclusion.add(Exclude(testNames: _*))
       this
     }
+    def includeCH(testNames: String*): SuiteSettings = {
+      this
+    }

Review Comment:
   yes，it's used to indicate this case is supported by ch backend 



##########
gluten-ut/common/src/test/scala/org/apache/gluten/utils/BackendTestSettings.scala:
##########
@@ -98,6 +98,13 @@ abstract class BackendTestSettings {
       exclusion.add(Exclude(testNames: _*))
       this
     }
+    def includeCH(testNames: String*): SuiteSettings = {
+      this
+    }

Review Comment:
   yes，it's used to indicate this case is supported by ch backend 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2597165865

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2651335344

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2597165634

   https://github.com/apache/incubator-gluten/issues/6067


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2636172723

   ## PR Reviewer Guide 🔍
   
   Here are some key observations to aid the review process:
   
   <table>
   <tr><td>
   
   **🎫 Ticket compliance analysis 🔶**
   
   
   
   **[6067](https://github.com/apache/incubator-gluten/issues/6067) - Partially compliant**
   
   Compliant requirements:
   
   - Upgrade Spark version to 3.5 and ensure compilation passes.
   - Upgrade Delta version to 3.2 and ensure compilation passes.
   - Support Scala 2.13.
   
   Non-compliant requirements:
   
   - Support native write.
   - Ensure CH backend unit tests pass (currently only MergeTree + Delta UT passed).
   - Address specific test failures and fallback scenarios.
   
   Requires further human verification:
   
   - Ensure CH backend unit tests pass (currently only MergeTree + Delta UT passed).
   - Address specific test failures and fallback scenarios.
   
   
   
   </td></tr>
   <tr><td>⏱️&nbsp;<strong>Estimated effort to review</strong>: 3 🔵🔵🔵⚪⚪</td></tr>
   <tr><td>🧪&nbsp;<strong>No relevant tests</strong></td></tr>
   <tr><td>🔒&nbsp;<strong>No security concerns identified</strong></td></tr>
   <tr><td>⚡&nbsp;<strong>Recommended focus areas for review</strong><br><br>
   
   <details><summary><a href='https://github.com/apache/incubator-gluten/pull/8555/files#diff-686011a5fa8eeae88230023b9078ee06c5d8257607418ca6bc9fb0f3d49e183cR37-R42'><strong>Possible Issue</strong></a>
   
   The `enableSuite` method no longer throws an exception for duplicate suite names, which could lead to unexpected behavior or silent overwrites. This change should be reviewed for correctness and potential unintended consequences.</summary>
   
   ```scala
   if (!enabledSuites.containsKey(suiteName)) {
     val suiteSettings = new SuiteSettings
     enabledSuites.put(suiteName, suiteSettings)
     enabledSuites.get(suiteName)
   }
   enabledSuites.get(suiteName)
   ```
   
   </details>
   
   </td></tr>
   </table>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2650071504

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2628965248

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen merged PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2649977800

   **[Persistent review](https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2636172723)** updated to latest commit https://github.com/apache/incubator-gluten/commit/590892c28cf278b1e3bc0cb893b7ac8300f5f8b9


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2649976760

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2631315265

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2636163365

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "lgbo-ustc (via GitHub)" <gi...@apache.org>.
lgbo-ustc commented on code in PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#discussion_r1951876740


##########
gluten-ut/common/src/test/scala/org/apache/gluten/utils/BackendTestSettings.scala:
##########
@@ -98,6 +98,13 @@ abstract class BackendTestSettings {
       exclusion.add(Exclude(testNames: _*))
       this
     }
+    def includeCH(testNames: String*): SuiteSettings = {
+      this
+    }

Review Comment:
   I just consider the method names are not general for backends, They are only used in CH test settings. How about this ?
   ```scala
   def includeBackend(backendName: String, testNames: String*)
   def excludeBackend(backendName: String, testNames: String*)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2652535757

   we now prepare to enable spark35 and disable spark 32
   ![image](https://github.com/user-attachments/assets/edf72ece-a33f-4f8f-99fb-514d0725c182)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] Open spark 35 ut [incubator-gluten]

Posted by "lgbo-ustc (via GitHub)" <gi...@apache.org>.
lgbo-ustc commented on code in PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#discussion_r1951876740


##########
gluten-ut/common/src/test/scala/org/apache/gluten/utils/BackendTestSettings.scala:
##########
@@ -98,6 +98,13 @@ abstract class BackendTestSettings {
       exclusion.add(Exclude(testNames: _*))
       this
     }
+    def includeCH(testNames: String*): SuiteSettings = {
+      this
+    }

Review Comment:
   I just consider the method names are not general for backends, They are only used in CH test settings. How about this ?
   ```scala
   def includeBackend(backendName, String, testNames: String*)
   def excludeBackend(backendName, String, testNames: String*)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [PR] [GLUTEN-6067] [WIP] open spark 35 ut [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8555:
URL: https://github.com/apache/incubator-gluten/pull/8555#issuecomment-2628965353

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org