You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zzcclp (via GitHub)" <gi...@apache.org> on 2024/07/16 06:59:10 UTC
[PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
zzcclp opened a new pull request, #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473
## What changes were proposed in this pull request?
When inserting mergetree data, it occurs the Task not serializable error in some cases.
RC:
In the Delta, the options of the `DeltaOptions` is a `CaseInsensitiveMap`, if calling the api `filterKeys()` of the `CaseInsensitiveMap`, it may become not serializable, so lead to this error.
Close #6470.
(Fixes: #6470)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230723740
Run Gluten Clickhouse CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230386915
Run Gluten Clickhouse CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on code in PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#discussion_r1680238045
##########
gluten-core/src/main/scala/org/apache/spark/softaffinity/SoftAffinity.scala:
##########
@@ -39,7 +39,7 @@ abstract class Affinity(val manager: AffinityManager) extends LogLevelUtil with
filePaths: Array[String],
preferredLocations: Array[String]): Array[String] = {
if (shouldUseSoftAffinity(filePaths, preferredLocations)) {
- internalGetHostLocations(filePaths.min)
+ internalGetHostLocations(filePaths(0))
Review Comment:
not related
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen merged PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen commented on code in PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#discussion_r1680229762
##########
gluten-core/src/main/scala/org/apache/spark/softaffinity/SoftAffinity.scala:
##########
@@ -39,7 +39,7 @@ abstract class Affinity(val manager: AffinityManager) extends LogLevelUtil with
filePaths: Array[String],
preferredLocations: Array[String]): Array[String] = {
if (shouldUseSoftAffinity(filePaths, preferredLocations)) {
- internalGetHostLocations(filePaths.min)
+ internalGetHostLocations(filePaths(0))
Review Comment:
Is this modification related to this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230164277
Run Gluten Clickhouse CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6470][CH]Fix Task not serializable error when inserting mergetree data [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6473:
URL: https://github.com/apache/incubator-gluten/pull/6473#issuecomment-2230163932
https://github.com/apache/incubator-gluten/issues/6470
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org