You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Cheng Lian <li...@gmail.com> on 2015/04/04 06:02:54 UTC
Re: Parquet sync'up notes
On 4/1/15 2:01 AM, Julien Le Dem wrote:
> Release 1.6:
> - mark as blocker of PARQUET-211 remaining tickets to merge before the
> release
> - in next few days merge those remaining PR
> - make sure the IPMC comments about the last parquet-format release are
> addressed.
> - Ryan: next Tuesday will cut out a RC and send it out for a vote.
> Once the release is official:
> - Julien/Alex/Tianshuo: publish artifacts one last time to
> com.twitter.parquet
> - Ryan: rename packages to org.apache
> - Ryan: vote on a 1.7 release
Did you mean vote on a 1.6 release?
> - merge the ByteBuffer related PRs
> - merge PARQUET-212: Implement nested type read rules in parquet-thrift
> Work on the semantic versioning improvements towards 2.0
>
> Logical types defined in Parquet are being integrated in Avro so that types
> can be more uniform across models.
>
> We might want to create a Parquet UNION logical type to make Parquet schema
> aware of the associated constraints.
>
> Parquet-99: Some records with very large values (MBs) cause problems.
> Possibly we should have safe guards about page buffers getting huge because
> of that (and causing OOM).
>
> Vectorized read path making progress: Zhengxiao
> https://issues.apache.org/jira/browse/PARQUET-131
> Hive integration POC by Dong Chen:
> https://issues.apache.org/jira/browse/HIVE-8128
>
> next sync up in 3 weeks
>
Re: Parquet sync'up notes
Posted by Cheng Lian <li...@gmail.com>.
I see. Thanks for the explanation.
Cheng
On 4/5/15 1:43 AM, Ryan Blue wrote:
> The 1.7 comment wasn't a typo. We are going to release 1.7 just after
> 1.6. It will be just a rename of the packages and artifacts so that
> transition to the org.apache namespace will go smoothly. Users can
> update to 1.6, verify that everything works, and then move to 1.7 with
> a rename knowing that it should work just like 1.6 did.
>
> rb
>
> On 04/03/2015 09:02 PM, Cheng Lian wrote:
>>
>>
>> On 4/1/15 2:01 AM, Julien Le Dem wrote:
>>> Release 1.6:
>>> - mark as blocker of PARQUET-211 remaining tickets to merge before
>>> the
>>> release
>>> - in next few days merge those remaining PR
>>> - make sure the IPMC comments about the last parquet-format
>>> release are
>>> addressed.
>>> - Ryan: next Tuesday will cut out a RC and send it out for a vote.
>>> Once the release is official:
>>> - Julien/Alex/Tianshuo: publish artifacts one last time to
>>> com.twitter.parquet
>>> - Ryan: rename packages to org.apache
>>> - Ryan: vote on a 1.7 release
>> Did you mean vote on a 1.6 release?
>>> - merge the ByteBuffer related PRs
>>> - merge PARQUET-212: Implement nested type read rules in
>>> parquet-thrift
>>> Work on the semantic versioning improvements towards 2.0
>>>
>>> Logical types defined in Parquet are being integrated in Avro so that
>>> types
>>> can be more uniform across models.
>>>
>>> We might want to create a Parquet UNION logical type to make Parquet
>>> schema
>>> aware of the associated constraints.
>>>
>>> Parquet-99: Some records with very large values (MBs) cause problems.
>>> Possibly we should have safe guards about page buffers getting huge
>>> because
>>> of that (and causing OOM).
>>>
>>> Vectorized read path making progress: Zhengxiao
>>> https://issues.apache.org/jira/browse/PARQUET-131
>>> Hive integration POC by Dong Chen:
>>> https://issues.apache.org/jira/browse/HIVE-8128
>>>
>>> next sync up in 3 weeks
>>>
>>
>
>
Re: Parquet sync'up notes
Posted by Ryan Blue <bl...@cloudera.com>.
The 1.7 comment wasn't a typo. We are going to release 1.7 just after
1.6. It will be just a rename of the packages and artifacts so that
transition to the org.apache namespace will go smoothly. Users can
update to 1.6, verify that everything works, and then move to 1.7 with a
rename knowing that it should work just like 1.6 did.
rb
On 04/03/2015 09:02 PM, Cheng Lian wrote:
>
>
> On 4/1/15 2:01 AM, Julien Le Dem wrote:
>> Release 1.6:
>> - mark as blocker of PARQUET-211 remaining tickets to merge before the
>> release
>> - in next few days merge those remaining PR
>> - make sure the IPMC comments about the last parquet-format release are
>> addressed.
>> - Ryan: next Tuesday will cut out a RC and send it out for a vote.
>> Once the release is official:
>> - Julien/Alex/Tianshuo: publish artifacts one last time to
>> com.twitter.parquet
>> - Ryan: rename packages to org.apache
>> - Ryan: vote on a 1.7 release
> Did you mean vote on a 1.6 release?
>> - merge the ByteBuffer related PRs
>> - merge PARQUET-212: Implement nested type read rules in parquet-thrift
>> Work on the semantic versioning improvements towards 2.0
>>
>> Logical types defined in Parquet are being integrated in Avro so that
>> types
>> can be more uniform across models.
>>
>> We might want to create a Parquet UNION logical type to make Parquet
>> schema
>> aware of the associated constraints.
>>
>> Parquet-99: Some records with very large values (MBs) cause problems.
>> Possibly we should have safe guards about page buffers getting huge
>> because
>> of that (and causing OOM).
>>
>> Vectorized read path making progress: Zhengxiao
>> https://issues.apache.org/jira/browse/PARQUET-131
>> Hive integration POC by Dong Chen:
>> https://issues.apache.org/jira/browse/HIVE-8128
>>
>> next sync up in 3 weeks
>>
>
--
Ryan Blue
Software Engineer
Cloudera, Inc.