You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Matthew Topol (Jira)" <ji...@apache.org> on 2022/09/06 22:13:00 UTC

[jira] [Commented] (ARROW-17627) [GO][Parquet] Unable to pass metadata without StoreSchema

    [ https://issues.apache.org/jira/browse/ARROW-17627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601031#comment-17601031 ] 

Matthew Topol commented on ARROW-17627:
---------------------------------------

Hey [~zhouyan1014] thanks for the proposal. This looks solid to me, would you be interested in making a PR for it? (and adding a test in the PR?)

Otherwise, I'll try to get around to adding this when I have a chance.

> [GO][Parquet] Unable to pass metadata without StoreSchema
> ---------------------------------------------------------
>
>                 Key: ARROW-17627
>                 URL: https://issues.apache.org/jira/browse/ARROW-17627
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Go, Parquet
>    Affects Versions: 9.0.0
>            Reporter: Yan Zhou
>            Priority: Major
>
> [https://github.com/apache/arrow/blob/master/go/parquet/pqarrow/file_writer.go#L75|https://github.com/apache/arrow/blob/master/go/parquet/pqarrow/file_writer.go#L75,]
>  
> Right now the only way to pass in KV metadata is to enable StoreSchema and pass it as part of the schema. In otherwise we can't pass through metadata without also store ARROW:schema
>  
> I suggest change it to the following
>  
> {code:go}
> 	meta := make(metadata.KeyValueMetadata, 0)
> 	for i := 0; i < arrschema.Metadata().Len(); i++ {
> 		meta.Append(arrschema.Metadata().Keys()[i], arrschema.Metadata().Values()[i])
> 	}
> 	if arrprops.storeSchema {
> 		serializedSchema := flight.SerializeSchema(arrschema, props.Allocator())
> 		meta.Append("ARROW:schema", base64.StdEncoding.EncodeToString(serializedSchema))
> 	}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)