[jira] [Resolved] (MAPREDUCE-7182) MapReduce input format/record readers to support S3 select queries

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved MAPREDUCE-7182.
---------------------------------------
    Resolution: Won't Fix

> MapReduce input format/record readers to support S3 select queries
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7182
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7182
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2
>    Affects Versions: 3.3.0
>            Reporter: Steve Loughran
>            Priority: Major
>
> HADOOP-15229 adds S3 select through the (new) async openFile API, but the classic RecordReader &c can't handle it because
> # the files are shorter than they are in a getFileStatus, and the readers assume that an EOFException is an error in that situation
> # everything assumes plain text is splitable
> # if a file has a gz extension, the gunzip codec should be used. So breaks transcoded/uncompressed data
> to handle s3 select data sources  we need to be able to address them, either through changes to the existing code (danger?) or some new readers



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org