You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Eric Erhardt (Jira)" <ji...@apache.org> on 2020/12/16 03:43:00 UTC

[jira] [Resolved] (ARROW-10719) [C#] ArrowStreamWriter doesn't write schema metadata

     [ https://issues.apache.org/jira/browse/ARROW-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Erhardt resolved ARROW-10719.
----------------------------------
    Fix Version/s: 3.0.0
       Resolution: Fixed

Issue resolved by pull request 8902
[https://github.com/apache/arrow/pull/8902]

> [C#] ArrowStreamWriter doesn't write schema metadata
> ----------------------------------------------------
>
>                 Key: ARROW-10719
>                 URL: https://issues.apache.org/jira/browse/ARROW-10719
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C#
>            Reporter: Eric Erhardt
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.0.0
>
>   Original Estimate: 48h
>          Time Spent: 1h 20m
>  Remaining Estimate: 46h 40m
>
> See this TODO in the code:
> [https://github.com/apache/arrow/blob/master/csharp/src/Apache.Arrow/Ipc/ArrowStreamWriter.cs#L445]
>  
> A simple test case:
>  
> {code:java}
> // Build schema
> var schemaBuilder = new Schema.Builder();
> schemaBuilder.Field(f => f
>     .Name("Ints")
>     .Nullable(true)
>     .DataType(Int32Type.Default)
>     .Metadata(new Dictionary<string, string> { { "packed", packed ? "true" : "false" } })
> );
> var indexMetadata = new Dictionary<string, string> { { "index", JsonConvert.SerializeObject(new object()) } };
> var schema = schemaBuilder.Metadata(indexMetadata).Build();
> Int32Array a = new Int32Array.Builder().Append(5).Append(4).Build();
> // Create record batch
> var recordBatch = new RecordBatch(schema, new IArrowArray[] { a }, a.Length);// Write recordbatch to file
> using (var stream = File.Create("visits.arrow"))
> using (var writer = new ArrowFileWriter(stream, recordBatch.Schema))
> {
>     writer.WriteRecordBatch(recordBatch);
>     writer.WriteEnd();
> }using (var stream = File.OpenRead("visits.arrow"))
> using (var reader = new ArrowFileReader(stream))
> {
>     var batch = reader.ReadNextRecordBatch();
>     Console.WriteLine(batch.Schema.HasMetadata);
> }
> {code}
> The above should print `true`, but it prints `false` for HasMetadata.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)