You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Michael A. Smith (Jira)" <ji...@apache.org> on 2022/04/02 21:08:00 UTC
[jira] [Updated] (AVRO-3261) s3 byte range for just schema or row count
[ https://issues.apache.org/jira/browse/AVRO-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael A. Smith updated AVRO-3261:
-----------------------------------
Description: boto3 python library has [s3 get_object|https://boto3.amazonaws.com/v1/documentation/api/1.21.32/reference/services/s3.html#S3.Client.get_object] that accepts a byte range (to avoid downloading a whole file but just download selected byte range). if my Avro file is 100mb, can Avro library do some byte range seek to only download the s3 object partially (ie the parts that contain header.schema metadata and row count)? (was: boto3 python library has s3 get_object that accepts a byte range (to avoid downloading a whole file but just download selected byte range). if my Avro file is 100mb, can Avro library do some byte range seek to only download the s3 object partially (ie the parts that contain header.schema metadata and row count)?)
> s3 byte range for just schema or row count
> ------------------------------------------
>
> Key: AVRO-3261
> URL: https://issues.apache.org/jira/browse/AVRO-3261
> Project: Apache Avro
> Issue Type: Wish
> Components: community, misc, python, tools
> Affects Versions: 1.11.0
> Reporter: t oo
> Priority: Major
>
> boto3 python library has [s3 get_object|https://boto3.amazonaws.com/v1/documentation/api/1.21.32/reference/services/s3.html#S3.Client.get_object] that accepts a byte range (to avoid downloading a whole file but just download selected byte range). if my Avro file is 100mb, can Avro library do some byte range seek to only download the s3 object partially (ie the parts that contain header.schema metadata and row count)?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)