[jira] [Closed] (ARROW-15878) [C++] Optimize csv writer for string with quotes

["Yibo Cai (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yibo Cai closed ARROW-15878.
----------------------------
    Resolution: Won't Do

20% improvement for best case (with more memory consumption) looks not very useful in practice.

> [C++] Optimize csv writer for string with quotes
> ------------------------------------------------
>
>                 Key: ARROW-15878
>                 URL: https://issues.apache.org/jira/browse/ARROW-15878
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Yibo Cai
>            Assignee: Yibo Cai
>            Priority: Major
>         Attachments: 0001-ARROW-15878-improve-csv-writer-for-string-with-quote.patch, wip.patch
>
>
> Escaping a string with quotes (put an extra quote before a quote) is the hotspot of csv writer [1]. This can probably be improved, possible approaches:
>  - Find the next quote with memchr, then memcpy blocks without quotes.
>  - Check if there are quotes with simd in 8 bytes or 16 bytes, do memcpy if no, otherwise go slow path.
> Should make sure the method doesn't decrease performance too much for strings with many quotes. And should be similar or better performance for short strings, which is common case.
> [1] [https://github.com/apache/arrow/blob/master/cpp/src/arrow/csv/writer.cc#L139]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)