[jira] [Commented] (ARROW-10440) [C++][Dataset][Python] Add a callback to visit file writers just before Finish()

["Ben Kietzman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366795#comment-17366795 ] 

Ben Kietzman commented on ARROW-10440:
--------------------------------------

Sorry for the late reply.

[~ldacey][~westonpace] Yes, I think it's totally reasonable to provide {{const std::string& FileWriter::path()}}. The caveat is that currently FIleWriter is agnostic about what kind of stream it's writing into; for example it may be writing into a Buffer rather than to disk which would clearly not be associated with any path. The {{path}} property could be added to {{FileFormat::MakeWriter}} and supplied alongside the output stream whenever appropriate.

[~jorisvandenbossche] we could also expose this in Python, in which case the general callback could be used to provide simpler functionality when appropriate

> [C++][Dataset][Python] Add a callback to visit file writers just before Finish()
> --------------------------------------------------------------------------------
>
>                 Key: ARROW-10440
>                 URL: https://issues.apache.org/jira/browse/ARROW-10440
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>    Affects Versions: 2.0.0
>            Reporter: Ben Kietzman
>            Assignee: Ben Kietzman
>            Priority: Major
>             Fix For: 5.0.0
>
>
> This will fill the role of (for example) {{metadata_collector}} or allow stats to be embedded in IPC file footer metadata.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)