You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zzcclp (via GitHub)" <gi...@apache.org> on 2024/07/16 06:59:10 UTC

[PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

zzcclp opened a new pull request, #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473

   ## What changes were proposed in this pull request?
   
   When inserting mergetree data, it occurs the Task not serializable error in some cases.
   
   RC:
   In the Delta, the options of the `DeltaOptions` is a `CaseInsensitiveMap`, if calling the api `filterKeys()` of the `CaseInsensitiveMap`, it may become not serializable, so lead to this error.
   
   Close #6470.
   
   (Fixes: #6470)
   
   ## How was this patch tested?
   
   (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
   
   
   (If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230723740

   Run Gluten Clickhouse CI


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230386915

   Run Gluten Clickhouse CI


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on code in PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#discussion_r1680238045


##########
gluten-core/src/main/scala/org/apache/spark/softaffinity/SoftAffinity.scala:
##########
@@ -39,7 +39,7 @@ abstract class Affinity(val manager: AffinityManager) extends LogLevelUtil with
       filePaths: Array[String],
       preferredLocations: Array[String]): Array[String] = {
     if (shouldUseSoftAffinity(filePaths, preferredLocations)) {
-      internalGetHostLocations(filePaths.min)
+      internalGetHostLocations(filePaths(0))

Review Comment:
   not related



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen merged PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on code in PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#discussion_r1680229762


##########
gluten-core/src/main/scala/org/apache/spark/softaffinity/SoftAffinity.scala:
##########
@@ -39,7 +39,7 @@ abstract class Affinity(val manager: AffinityManager) extends LogLevelUtil with
       filePaths: Array[String],
       preferredLocations: Array[String]): Array[String] = {
     if (shouldUseSoftAffinity(filePaths, preferredLocations)) {
-      internalGetHostLocations(filePaths.min)
+      internalGetHostLocations(filePaths(0))

Review Comment:
   Is this modification related to this issue?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230164277

   Run Gluten Clickhouse CI


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230163932

   https://github.com/apache/incubator-gluten/issues/6470


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org