You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Eric Erhardt (Jira)" <ji...@apache.org> on 2020/12/16 03:43:00 UTC
[jira] [Resolved] (ARROW-10719) [C#] ArrowStreamWriter doesn't
write schema metadata
[ https://issues.apache.org/jira/browse/ARROW-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Erhardt resolved ARROW-10719.
----------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 8902
[https://github.com/apache/arrow/pull/8902]
> [C#] ArrowStreamWriter doesn't write schema metadata
> ----------------------------------------------------
>
> Key: ARROW-10719
> URL: https://issues.apache.org/jira/browse/ARROW-10719
> Project: Apache Arrow
> Issue Type: Bug
> Components: C#
> Reporter: Eric Erhardt
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Original Estimate: 48h
> Time Spent: 1h 20m
> Remaining Estimate: 46h 40m
>
> See this TODO in the code:
> [https://github.com/apache/arrow/blob/master/csharp/src/Apache.Arrow/Ipc/ArrowStreamWriter.cs#L445]
>
> A simple test case:
>
> {code:java}
> // Build schema
> var schemaBuilder = new Schema.Builder();
> schemaBuilder.Field(f => f
> .Name("Ints")
> .Nullable(true)
> .DataType(Int32Type.Default)
> .Metadata(new Dictionary<string, string> { { "packed", packed ? "true" : "false" } })
> );
> var indexMetadata = new Dictionary<string, string> { { "index", JsonConvert.SerializeObject(new object()) } };
> var schema = schemaBuilder.Metadata(indexMetadata).Build();
> Int32Array a = new Int32Array.Builder().Append(5).Append(4).Build();
> // Create record batch
> var recordBatch = new RecordBatch(schema, new IArrowArray[] { a }, a.Length);// Write recordbatch to file
> using (var stream = File.Create("visits.arrow"))
> using (var writer = new ArrowFileWriter(stream, recordBatch.Schema))
> {
> writer.WriteRecordBatch(recordBatch);
> writer.WriteEnd();
> }using (var stream = File.OpenRead("visits.arrow"))
> using (var reader = new ArrowFileReader(stream))
> {
> var batch = reader.ReadNextRecordBatch();
> Console.WriteLine(batch.Schema.HasMetadata);
> }
> {code}
> The above should print `true`, but it prints `false` for HasMetadata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)