You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@arrow.apache.org by "Anson Qian (JIRA)" <ji...@apache.org> on 2019/03/01 19:43:00 UTC

[jira] [Created] (ARROW-4736) [Go] Optimize memory usage for CSV writer

Anson Qian created ARROW-4736:
---------------------------------

             Summary: [Go] Optimize memory usage for CSV writer
                 Key: ARROW-4736
                 URL: https://issues.apache.org/jira/browse/ARROW-4736
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Anson Qian


perhaps not for this PR, but, depending on the number of rows and cols this record contains, this may be a very large allocation, and very big memory chunk.
it could be more interesting performance wise to write n rows instead of everything in one big chunk.

also, to reduce the memory pressure on the GC, we should probably (re)use this slice-of-slice of strings.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)