You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/09/29 10:48:51 UTC

[GitHub] [incubator-doris] yangzhg opened a new issue #6781: Optimize cpu consumption when importing parquet files

yangzhg opened a new issue #6781:
URL: https://github.com/apache/incubator-doris/issues/6781


   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Version
   
   master
   
   ### What's Wrong?
   
   When importing parquet type files, we use arrow to parse parquet. Arrow has a common abstraction for columns: array. We need to convert array to the type we need. Here we use dynamic_cast, but we did not do anything about the conversion failure. Processing, there is no place to catch bad_cast exceptions, these conversions consume about 10% of the cpu
   
   ### What You Expected?
   
   fix
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] morningman closed issue #6781: Optimize cpu consumption when importing parquet files

Posted by GitBox <gi...@apache.org>.
morningman closed issue #6781:
URL: https://github.com/apache/incubator-doris/issues/6781


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org