You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Ryan Blue <rb...@netflix.com.INVALID> on 2018/04/05 21:28:17 UTC
Re: parquet-mr next release with PARQUET-1217?
I just sent a vote for this. Took longer than expected because I had to fix
all of the javadoc warnings for java 8. Please test it out and vote.
On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue <rb...@netflix.com> wrote:
> I have no plan for 1.9.1.
>
> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson <he...@cloudera.com>
> wrote:
>
>> Great! Do you know of any plans to do a 1.9.1?
>>
>> On 30 March 2018 at 09:35, Ryan Blue <rb...@netflix.com.invalid> wrote:
>>
>>> I'm planning on getting a 1.10.0 rc out today, if I don't find problems
>>> with the stats changes.
>>>
>>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson <he...@apache.org>
>>> wrote:
>>>
>>> > Hi all -
>>> >
>>> > While using Spark, I got hit by PARQUET-1217 today on some data
>>> written by
>>> > Impala. This is a pretty nasty bug, and one that affects Apache Spark
>>> right
>>> > now because, AFAICT, there's no release to move to that contains the
>>> fix,
>>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's
>>> expensive
>>> > in terms of lost performance.
>>> >
>>> > I'm new to the community, so wanted to see if there was a plan to make
>>> a
>>> > release (1.9.1?) in the near future. I'd rather that than have to build
>>> > short-term workarounds into Spark.
>>> >
>>> > Best,
>>> > Henry
>>> >
>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Netflix
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679 <(415)%20994-6679>
>>
>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
--
Ryan Blue
Software Engineer
Netflix
Re: parquet-mr next release with PARQUET-1217?
Posted by Henry Robinson <he...@apache.org>.
Yeah, Spark would (I think) prefer a 1.8.3. I'd be happy to drive the
release, but not clear if it's restricted to PMC members or committers to
do (particularly, tagging in the repository probably requires write
access).
On 12 April 2018 at 14:48, Ryan Blue <rb...@netflix.com.invalid> wrote:
> I think it would be better to do a 1.8.2 (or is it 1.8.3?) because Spark is
> on the 1.8 line. I doubt Spark is going to move to 1.9 now that 1.10 is
> out.
>
> On Thu, Apr 12, 2018 at 2:43 PM, Julien Le Dem <ju...@gmail.com>
> wrote:
>
> > If someone wants a 1.9.1 it can be done, we'll need someone to own the
> > release process though.
> >
> > On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson <he...@apache.org>
> wrote:
> >
> >> Thanks! Sorry to miss the vote - was AFK for a few days. I look forward
> to
> >> testing it out anyhow.
> >>
> >> On 5 April 2018 at 14:28, Ryan Blue <rb...@netflix.com> wrote:
> >>
> >> > I just sent a vote for this. Took longer than expected because I had
> to
> >> > fix all of the javadoc warnings for java 8. Please test it out and
> vote.
> >> >
> >> > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue <rb...@netflix.com>
> wrote:
> >> >
> >> >> I have no plan for 1.9.1.
> >> >>
> >> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson <henry@cloudera.com
> >
> >> >> wrote:
> >> >>
> >> >>> Great! Do you know of any plans to do a 1.9.1?
> >> >>>
> >> >>> On 30 March 2018 at 09:35, Ryan Blue <rb...@netflix.com.invalid>
> >> wrote:
> >> >>>
> >> >>>> I'm planning on getting a 1.10.0 rc out today, if I don't find
> >> problems
> >> >>>> with the stats changes.
> >> >>>>
> >> >>>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson <he...@apache.org>
> >> >>>> wrote:
> >> >>>>
> >> >>>> > Hi all -
> >> >>>> >
> >> >>>> > While using Spark, I got hit by PARQUET-1217 today on some data
> >> >>>> written by
> >> >>>> > Impala. This is a pretty nasty bug, and one that affects Apache
> >> Spark
> >> >>>> right
> >> >>>> > now because, AFAICT, there's no release to move to that contains
> >> the
> >> >>>> fix,
> >> >>>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's
> >> >>>> expensive
> >> >>>> > in terms of lost performance.
> >> >>>> >
> >> >>>> > I'm new to the community, so wanted to see if there was a plan to
> >> >>>> make a
> >> >>>> > release (1.9.1?) in the near future. I'd rather that than have to
> >> >>>> build
> >> >>>> > short-term workarounds into Spark.
> >> >>>> >
> >> >>>> > Best,
> >> >>>> > Henry
> >> >>>> >
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> Ryan Blue
> >> >>>> Software Engineer
> >> >>>> Netflix
> >> >>>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Henry Robinson
> >> >>> Software Engineer
> >> >>> Cloudera
> >> >>> 415-994-6679 <(415)%20994-6679>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Ryan Blue
> >> >> Software Engineer
> >> >> Netflix
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Ryan Blue
> >> > Software Engineer
> >> > Netflix
> >> >
> >>
> >
> >
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
Re: parquet-mr next release with PARQUET-1217?
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
I think it would be better to do a 1.8.2 (or is it 1.8.3?) because Spark is
on the 1.8 line. I doubt Spark is going to move to 1.9 now that 1.10 is out.
On Thu, Apr 12, 2018 at 2:43 PM, Julien Le Dem <ju...@gmail.com>
wrote:
> If someone wants a 1.9.1 it can be done, we'll need someone to own the
> release process though.
>
> On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson <he...@apache.org> wrote:
>
>> Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to
>> testing it out anyhow.
>>
>> On 5 April 2018 at 14:28, Ryan Blue <rb...@netflix.com> wrote:
>>
>> > I just sent a vote for this. Took longer than expected because I had to
>> > fix all of the javadoc warnings for java 8. Please test it out and vote.
>> >
>> > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue <rb...@netflix.com> wrote:
>> >
>> >> I have no plan for 1.9.1.
>> >>
>> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson <he...@cloudera.com>
>> >> wrote:
>> >>
>> >>> Great! Do you know of any plans to do a 1.9.1?
>> >>>
>> >>> On 30 March 2018 at 09:35, Ryan Blue <rb...@netflix.com.invalid>
>> wrote:
>> >>>
>> >>>> I'm planning on getting a 1.10.0 rc out today, if I don't find
>> problems
>> >>>> with the stats changes.
>> >>>>
>> >>>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson <he...@apache.org>
>> >>>> wrote:
>> >>>>
>> >>>> > Hi all -
>> >>>> >
>> >>>> > While using Spark, I got hit by PARQUET-1217 today on some data
>> >>>> written by
>> >>>> > Impala. This is a pretty nasty bug, and one that affects Apache
>> Spark
>> >>>> right
>> >>>> > now because, AFAICT, there's no release to move to that contains
>> the
>> >>>> fix,
>> >>>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's
>> >>>> expensive
>> >>>> > in terms of lost performance.
>> >>>> >
>> >>>> > I'm new to the community, so wanted to see if there was a plan to
>> >>>> make a
>> >>>> > release (1.9.1?) in the near future. I'd rather that than have to
>> >>>> build
>> >>>> > short-term workarounds into Spark.
>> >>>> >
>> >>>> > Best,
>> >>>> > Henry
>> >>>> >
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Ryan Blue
>> >>>> Software Engineer
>> >>>> Netflix
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Henry Robinson
>> >>> Software Engineer
>> >>> Cloudera
>> >>> 415-994-6679 <(415)%20994-6679>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Ryan Blue
>> >> Software Engineer
>> >> Netflix
>> >>
>> >
>> >
>> >
>> > --
>> > Ryan Blue
>> > Software Engineer
>> > Netflix
>> >
>>
>
>
--
Ryan Blue
Software Engineer
Netflix
Re: parquet-mr next release with PARQUET-1217?
Posted by Julien Le Dem <ju...@gmail.com>.
If someone wants a 1.9.1 it can be done, we'll need someone to own the
release process though.
On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson <he...@apache.org> wrote:
> Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to
> testing it out anyhow.
>
> On 5 April 2018 at 14:28, Ryan Blue <rb...@netflix.com> wrote:
>
> > I just sent a vote for this. Took longer than expected because I had to
> > fix all of the javadoc warnings for java 8. Please test it out and vote.
> >
> > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue <rb...@netflix.com> wrote:
> >
> >> I have no plan for 1.9.1.
> >>
> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson <he...@cloudera.com>
> >> wrote:
> >>
> >>> Great! Do you know of any plans to do a 1.9.1?
> >>>
> >>> On 30 March 2018 at 09:35, Ryan Blue <rb...@netflix.com.invalid>
> wrote:
> >>>
> >>>> I'm planning on getting a 1.10.0 rc out today, if I don't find
> problems
> >>>> with the stats changes.
> >>>>
> >>>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson <he...@apache.org>
> >>>> wrote:
> >>>>
> >>>> > Hi all -
> >>>> >
> >>>> > While using Spark, I got hit by PARQUET-1217 today on some data
> >>>> written by
> >>>> > Impala. This is a pretty nasty bug, and one that affects Apache
> Spark
> >>>> right
> >>>> > now because, AFAICT, there's no release to move to that contains the
> >>>> fix,
> >>>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's
> >>>> expensive
> >>>> > in terms of lost performance.
> >>>> >
> >>>> > I'm new to the community, so wanted to see if there was a plan to
> >>>> make a
> >>>> > release (1.9.1?) in the near future. I'd rather that than have to
> >>>> build
> >>>> > short-term workarounds into Spark.
> >>>> >
> >>>> > Best,
> >>>> > Henry
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Ryan Blue
> >>>> Software Engineer
> >>>> Netflix
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Henry Robinson
> >>> Software Engineer
> >>> Cloudera
> >>> 415-994-6679 <(415)%20994-6679>
> >>>
> >>
> >>
> >>
> >> --
> >> Ryan Blue
> >> Software Engineer
> >> Netflix
> >>
> >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>
Re: parquet-mr next release with PARQUET-1217?
Posted by Henry Robinson <he...@apache.org>.
Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to
testing it out anyhow.
On 5 April 2018 at 14:28, Ryan Blue <rb...@netflix.com> wrote:
> I just sent a vote for this. Took longer than expected because I had to
> fix all of the javadoc warnings for java 8. Please test it out and vote.
>
> On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue <rb...@netflix.com> wrote:
>
>> I have no plan for 1.9.1.
>>
>> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson <he...@cloudera.com>
>> wrote:
>>
>>> Great! Do you know of any plans to do a 1.9.1?
>>>
>>> On 30 March 2018 at 09:35, Ryan Blue <rb...@netflix.com.invalid> wrote:
>>>
>>>> I'm planning on getting a 1.10.0 rc out today, if I don't find problems
>>>> with the stats changes.
>>>>
>>>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson <he...@apache.org>
>>>> wrote:
>>>>
>>>> > Hi all -
>>>> >
>>>> > While using Spark, I got hit by PARQUET-1217 today on some data
>>>> written by
>>>> > Impala. This is a pretty nasty bug, and one that affects Apache Spark
>>>> right
>>>> > now because, AFAICT, there's no release to move to that contains the
>>>> fix,
>>>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's
>>>> expensive
>>>> > in terms of lost performance.
>>>> >
>>>> > I'm new to the community, so wanted to see if there was a plan to
>>>> make a
>>>> > release (1.9.1?) in the near future. I'd rather that than have to
>>>> build
>>>> > short-term workarounds into Spark.
>>>> >
>>>> > Best,
>>>> > Henry
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>> Software Engineer
>>>> Netflix
>>>>
>>>
>>>
>>>
>>> --
>>> Henry Robinson
>>> Software Engineer
>>> Cloudera
>>> 415-994-6679 <(415)%20994-6679>
>>>
>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>