You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "shuai-xu (via GitHub)" <gi...@apache.org> on 2024/09/24 03:54:48 UTC
[I] [CH] read result may diff with spark for json format [incubator-gluten]
shuai-xu opened a new issue, #7325:
URL: https://github.com/apache/incubator-gluten/issues/7325
### Backend
CH (ClickHouse)
### Bug description
When read json table, valina spark will first deserialize the line to json, but gluten will not. So for a line like `{"udid":"N9DszceOcjnJIMkW","user-agent":"imoAndroidBeta 2024.07.2052","apps_new":{"2d50eba8bb6c1c877acb513e53570650":{"is_system":true,"flags":814267973},"b15fcb737f5460065eb66ce6c885a5a2":{"is_system":false,"flags":819478084,"name":"12b38c33f160fe0999e1a7fc4e46a675"}}}`, select apps_new spark will get `{"2d50eba8bb6c1c877acb513e53570650":{"is_system":true,"flags":"814267973"},"b15fcb737f5460065eb66ce6c885a5a2":{"is_system":false,"flags":"819478084","name":"12b38c33f160fe0999e1a7fc4e46a675"}` while gluten get `{"2d50eba8bb6c1c877acb513e53570650":{"is_system":true,"flags":814267973},"b15fcb737f5460065eb66ce6c885a5a2":{"is_system":false,"flags":819478084,"name":"12b38c33f160fe0999e1a7fc4e46a675"}`.
### Spark version
Spark-3.3.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] read result may diff with spark for json format [incubator-gluten]
Posted by "baibaichen (via GitHub)" <gi...@apache.org>.
baibaichen closed issue #7325: [CH] read result may diff with spark for json format
URL: https://github.com/apache/incubator-gluten/issues/7325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org