You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Olga Natkovich <ol...@yahoo-inc.com> on 2011/06/23 02:56:42 UTC
[VOTE] Release Pig 0.9.0 (candidate 0)
I have created a candidate build for Pig 0.9.0 release. This release introduces control structures, changes query parser, and performs semantic cleanup.
The rat report showed no issues in Java files outside of build directory.
Keys used to sign the release are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
Please try it out:
http://people.apache.org/~olga/pig-0.9.0-candidate-0/
Build is also available in maven: https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
Should we release this? Vote closes on Monday, June 27.
Olga
Re: [DISCUSS] Release Pig 0.9.0 (candidate 0)
Posted by Jonathan Coveney <jc...@gmail.com>.
Just to weigh in, fixing the hardcoded hdfs would be excellent. Making it so
that pig could work easily with other DFS is going to be more and more
important until the trunk hadoop becomes the standard (if ever!)
2011/6/23 Dmitriy Ryaboy <dv...@gmail.com>
> FWIW I just committed the fix for PIG-2137 to both trunk and the 0.9
> branch.
> Thanks Thejas for testing & improvements.
>
> D
>
> On Thu, Jun 23, 2011 at 3:21 PM, Dmitriy Ryaboy <dv...@gmail.com>
> wrote:
>
> > Moving discussion to a discuss thread so votes can stay on the vote
> thread.
> >
> > Richa, I don't feel those two issues you point out are blockers for a
> > release, but a patch for 0.10 would be welcome to fix the hardcoding
> issue
> > (gosh I can't imagine how you found that.. :)).
> >
> > D
> >
> > On Thu, Jun 23, 2011 at 3:04 PM, Richa Khandelwal <richak@maprtech.com
> >wrote:
> >
> >> Hi Guys,
> >>
> >> I would vote +1 to remove the hadoop jar from the pig.jar. That would
> >> definitely add to the flexibility of using Pig with any version or
> variant
> >> of hadoop.
> >>
> >> Also, there is "hdfs://" hardcoded at a few places in the pig source. I
> >> can
> >> provide a patch which removes the hard-coding and makes it possible to
> use
> >> pig on any distributed filesystem that can be used with hadoop, based on
> >> pig.properties configuration file. Please let me know.
> >>
> >> Thanks,
> >> Richa
> >>
> >> On Thu, Jun 23, 2011 at 2:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
> >> wrote:
> >>
> >> > My experience with sample command is that people use it for some quick
> >> > testing and debugging not for production problems. This issue has been
> >> there
> >> > since the initial introduction of sample. I still don't see a strong
> >> need to
> >> > hold the beta release for it.
> >> >
> >> > Olga
> >> >
> >> > -----Original Message-----
> >> > From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> >> > Sent: Thursday, June 23, 2011 10:14 AM
> >> > To: dev@pig.apache.org
> >> > Cc: dev@pig.apache.org
> >> > Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
> >> >
> >> > There is a workaround - turning off optimizations, or at least
> >> > pushUpFilter. I haven't checked 8 with the new logical plan disabled,
> >> that
> >> > may work too.
> >> > If this failed execution, I'd let it pass since there is a workaround,
> >> but
> >> > as is it silently returns incorrect data, with no way for an analyst
> to
> >> know
> >> > her statistics are now fundamentally wrong. Silent and subtle data
> >> > corruption is just about the wrist big we can have.
> >> > I can't really block the release, you guys can outvote me. But I'd
> >> rather
> >> > you didn't; we can patch this problem today.
> >> >
> >> > On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> >> >
> >> > > Are you referring to PIG-2137? I have a few of questions on that
> >> before
> >> > I vote for this release candidate or to reroll.
> >> > >
> >> > > Is this a new issue introduced in 0.9?
> >> > >
> >> > > Is there a workaround for this?
> >> > >
> >> > > We have already discussed that 0.9.0 will be beta quality, and a
> >> follow
> >> > up release will be needed as users find bugs. As sample is not a
> >> heavily
> >> > used feature I am inclined to view this bug as ok. You feel this is
> >> serious
> >> > enough to block a beta release?
> >> > >
> >> > > Alan.
> >> > >
> >> > > On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
> >> > >
> >> > >> -1
> >> > >>
> >> > >> I discovered a critical bug in how SAMPLE is treated; I don't think
> >> we
> >> > >> should release until it's fixed (Thejas is testing the fix).
> >> > >>
> >> > >> D
> >> > >>
> >> > >> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <
> olgan@yahoo-inc.com
> >> >
> >> > wrote:
> >> > >>
> >> > >>> I have created a candidate build for Pig 0.9.0 release. This
> release
> >> > >>> introduces control structures, changes query parser, and performs
> >> > semantic
> >> > >>> cleanup.
> >> > >>>
> >> > >>> The rat report showed no issues in Java files outside of build
> >> > directory.
> >> > >>>
> >> > >>>
> >> > >>> Keys used to sign the release are available at
> >> > >>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> Please try it out:
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> Build is also available in maven:
> >> > >>>
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> Should we release this? Vote closes on Monday, June 27.
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> Olga
> >> > >>>
> >> > >>>
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] Release Pig 0.9.0 (candidate 0)
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
FWIW I just committed the fix for PIG-2137 to both trunk and the 0.9 branch.
Thanks Thejas for testing & improvements.
D
On Thu, Jun 23, 2011 at 3:21 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Moving discussion to a discuss thread so votes can stay on the vote thread.
>
> Richa, I don't feel those two issues you point out are blockers for a
> release, but a patch for 0.10 would be welcome to fix the hardcoding issue
> (gosh I can't imagine how you found that.. :)).
>
> D
>
> On Thu, Jun 23, 2011 at 3:04 PM, Richa Khandelwal <ri...@maprtech.com>wrote:
>
>> Hi Guys,
>>
>> I would vote +1 to remove the hadoop jar from the pig.jar. That would
>> definitely add to the flexibility of using Pig with any version or variant
>> of hadoop.
>>
>> Also, there is "hdfs://" hardcoded at a few places in the pig source. I
>> can
>> provide a patch which removes the hard-coding and makes it possible to use
>> pig on any distributed filesystem that can be used with hadoop, based on
>> pig.properties configuration file. Please let me know.
>>
>> Thanks,
>> Richa
>>
>> On Thu, Jun 23, 2011 at 2:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
>> wrote:
>>
>> > My experience with sample command is that people use it for some quick
>> > testing and debugging not for production problems. This issue has been
>> there
>> > since the initial introduction of sample. I still don't see a strong
>> need to
>> > hold the beta release for it.
>> >
>> > Olga
>> >
>> > -----Original Message-----
>> > From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>> > Sent: Thursday, June 23, 2011 10:14 AM
>> > To: dev@pig.apache.org
>> > Cc: dev@pig.apache.org
>> > Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
>> >
>> > There is a workaround - turning off optimizations, or at least
>> > pushUpFilter. I haven't checked 8 with the new logical plan disabled,
>> that
>> > may work too.
>> > If this failed execution, I'd let it pass since there is a workaround,
>> but
>> > as is it silently returns incorrect data, with no way for an analyst to
>> know
>> > her statistics are now fundamentally wrong. Silent and subtle data
>> > corruption is just about the wrist big we can have.
>> > I can't really block the release, you guys can outvote me. But I'd
>> rather
>> > you didn't; we can patch this problem today.
>> >
>> > On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>> >
>> > > Are you referring to PIG-2137? I have a few of questions on that
>> before
>> > I vote for this release candidate or to reroll.
>> > >
>> > > Is this a new issue introduced in 0.9?
>> > >
>> > > Is there a workaround for this?
>> > >
>> > > We have already discussed that 0.9.0 will be beta quality, and a
>> follow
>> > up release will be needed as users find bugs. As sample is not a
>> heavily
>> > used feature I am inclined to view this bug as ok. You feel this is
>> serious
>> > enough to block a beta release?
>> > >
>> > > Alan.
>> > >
>> > > On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
>> > >
>> > >> -1
>> > >>
>> > >> I discovered a critical bug in how SAMPLE is treated; I don't think
>> we
>> > >> should release until it's fixed (Thejas is testing the fix).
>> > >>
>> > >> D
>> > >>
>> > >> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <olgan@yahoo-inc.com
>> >
>> > wrote:
>> > >>
>> > >>> I have created a candidate build for Pig 0.9.0 release. This release
>> > >>> introduces control structures, changes query parser, and performs
>> > semantic
>> > >>> cleanup.
>> > >>>
>> > >>> The rat report showed no issues in Java files outside of build
>> > directory.
>> > >>>
>> > >>>
>> > >>> Keys used to sign the release are available at
>> > >>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>> > >>>
>> > >>>
>> > >>>
>> > >>> Please try it out:
>> > >>>
>> > >>>
>> > >>>
>> > >>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>> > >>>
>> > >>>
>> > >>>
>> > >>> Build is also available in maven:
>> > >>>
>> >
>> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>> > >>>
>> > >>>
>> > >>>
>> > >>> Should we release this? Vote closes on Monday, June 27.
>> > >>>
>> > >>>
>> > >>>
>> > >>> Olga
>> > >>>
>> > >>>
>> > >
>> >
>>
>
>
Re: [DISCUSS] Release Pig 0.9.0 (candidate 0)
Posted by Alan Gates <ga...@yahoo-inc.com>.
Agreed, same with including a pig-withouthadoop jar. We do publish
such a jar already in maven. This is a non-critical (by which I mean
it doesn't cause failure or wrong results, not that it isn't
important) issue and there is not agreement yet on whether the without
jar should be included in the tar ball. So we shouldn't hold a
release for it.
Alan.
On Jun 23, 2011, at 3:21 PM, Dmitriy Ryaboy wrote:
> Moving discussion to a discuss thread so votes can stay on the vote
> thread.
>
> Richa, I don't feel those two issues you point out are blockers for a
> release, but a patch for 0.10 would be welcome to fix the hardcoding
> issue
> (gosh I can't imagine how you found that.. :)).
>
> D
>
> On Thu, Jun 23, 2011 at 3:04 PM, Richa Khandelwal
> <ri...@maprtech.com>wrote:
>
>> Hi Guys,
>>
>> I would vote +1 to remove the hadoop jar from the pig.jar. That would
>> definitely add to the flexibility of using Pig with any version or
>> variant
>> of hadoop.
>>
>> Also, there is "hdfs://" hardcoded at a few places in the pig
>> source. I can
>> provide a patch which removes the hard-coding and makes it possible
>> to use
>> pig on any distributed filesystem that can be used with hadoop,
>> based on
>> pig.properties configuration file. Please let me know.
>>
>> Thanks,
>> Richa
>>
>> On Thu, Jun 23, 2011 at 2:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
>> wrote:
>>
>>> My experience with sample command is that people use it for some
>>> quick
>>> testing and debugging not for production problems. This issue has
>>> been
>> there
>>> since the initial introduction of sample. I still don't see a
>>> strong need
>> to
>>> hold the beta release for it.
>>>
>>> Olga
>>>
>>> -----Original Message-----
>>> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>>> Sent: Thursday, June 23, 2011 10:14 AM
>>> To: dev@pig.apache.org
>>> Cc: dev@pig.apache.org
>>> Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
>>>
>>> There is a workaround - turning off optimizations, or at least
>>> pushUpFilter. I haven't checked 8 with the new logical plan
>>> disabled,
>> that
>>> may work too.
>>> If this failed execution, I'd let it pass since there is a
>>> workaround,
>> but
>>> as is it silently returns incorrect data, with no way for an
>>> analyst to
>> know
>>> her statistics are now fundamentally wrong. Silent and subtle data
>>> corruption is just about the wrist big we can have.
>>> I can't really block the release, you guys can outvote me. But I'd
>>> rather
>>> you didn't; we can patch this problem today.
>>>
>>> On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>>>
>>>> Are you referring to PIG-2137? I have a few of questions on that
>> before
>>> I vote for this release candidate or to reroll.
>>>>
>>>> Is this a new issue introduced in 0.9?
>>>>
>>>> Is there a workaround for this?
>>>>
>>>> We have already discussed that 0.9.0 will be beta quality, and a
>>>> follow
>>> up release will be needed as users find bugs. As sample is not a
>>> heavily
>>> used feature I am inclined to view this bug as ok. You feel this is
>> serious
>>> enough to block a beta release?
>>>>
>>>> Alan.
>>>>
>>>> On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
>>>>
>>>>> -1
>>>>>
>>>>> I discovered a critical bug in how SAMPLE is treated; I don't
>>>>> think we
>>>>> should release until it's fixed (Thejas is testing the fix).
>>>>>
>>>>> D
>>>>>
>>>>> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <olgan@yahoo-inc.com
>>>>> >
>>> wrote:
>>>>>
>>>>>> I have created a candidate build for Pig 0.9.0 release. This
>>>>>> release
>>>>>> introduces control structures, changes query parser, and performs
>>> semantic
>>>>>> cleanup.
>>>>>>
>>>>>> The rat report showed no issues in Java files outside of build
>>> directory.
>>>>>>
>>>>>>
>>>>>> Keys used to sign the release are available at
>>>>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please try it out:
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>>>>>>
>>>>>>
>>>>>>
>>>>>> Build is also available in maven:
>>>>>>
>>>
>> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>>>>>>
>>>>>>
>>>>>>
>>>>>> Should we release this? Vote closes on Monday, June 27.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Olga
>>>>>>
>>>>>>
>>>>
>>>
>>
Re: [DISCUSS] Release Pig 0.9.0 (candidate 0)
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Moving discussion to a discuss thread so votes can stay on the vote thread.
Richa, I don't feel those two issues you point out are blockers for a
release, but a patch for 0.10 would be welcome to fix the hardcoding issue
(gosh I can't imagine how you found that.. :)).
D
On Thu, Jun 23, 2011 at 3:04 PM, Richa Khandelwal <ri...@maprtech.com>wrote:
> Hi Guys,
>
> I would vote +1 to remove the hadoop jar from the pig.jar. That would
> definitely add to the flexibility of using Pig with any version or variant
> of hadoop.
>
> Also, there is "hdfs://" hardcoded at a few places in the pig source. I can
> provide a patch which removes the hard-coding and makes it possible to use
> pig on any distributed filesystem that can be used with hadoop, based on
> pig.properties configuration file. Please let me know.
>
> Thanks,
> Richa
>
> On Thu, Jun 23, 2011 at 2:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
> wrote:
>
> > My experience with sample command is that people use it for some quick
> > testing and debugging not for production problems. This issue has been
> there
> > since the initial introduction of sample. I still don't see a strong need
> to
> > hold the beta release for it.
> >
> > Olga
> >
> > -----Original Message-----
> > From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> > Sent: Thursday, June 23, 2011 10:14 AM
> > To: dev@pig.apache.org
> > Cc: dev@pig.apache.org
> > Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
> >
> > There is a workaround - turning off optimizations, or at least
> > pushUpFilter. I haven't checked 8 with the new logical plan disabled,
> that
> > may work too.
> > If this failed execution, I'd let it pass since there is a workaround,
> but
> > as is it silently returns incorrect data, with no way for an analyst to
> know
> > her statistics are now fundamentally wrong. Silent and subtle data
> > corruption is just about the wrist big we can have.
> > I can't really block the release, you guys can outvote me. But I'd rather
> > you didn't; we can patch this problem today.
> >
> > On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> >
> > > Are you referring to PIG-2137? I have a few of questions on that
> before
> > I vote for this release candidate or to reroll.
> > >
> > > Is this a new issue introduced in 0.9?
> > >
> > > Is there a workaround for this?
> > >
> > > We have already discussed that 0.9.0 will be beta quality, and a follow
> > up release will be needed as users find bugs. As sample is not a heavily
> > used feature I am inclined to view this bug as ok. You feel this is
> serious
> > enough to block a beta release?
> > >
> > > Alan.
> > >
> > > On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
> > >
> > >> -1
> > >>
> > >> I discovered a critical bug in how SAMPLE is treated; I don't think we
> > >> should release until it's fixed (Thejas is testing the fix).
> > >>
> > >> D
> > >>
> > >> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
> > wrote:
> > >>
> > >>> I have created a candidate build for Pig 0.9.0 release. This release
> > >>> introduces control structures, changes query parser, and performs
> > semantic
> > >>> cleanup.
> > >>>
> > >>> The rat report showed no issues in Java files outside of build
> > directory.
> > >>>
> > >>>
> > >>> Keys used to sign the release are available at
> > >>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
> > >>>
> > >>>
> > >>>
> > >>> Please try it out:
> > >>>
> > >>>
> > >>>
> > >>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
> > >>>
> > >>>
> > >>>
> > >>> Build is also available in maven:
> > >>>
> >
> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
> > >>>
> > >>>
> > >>>
> > >>> Should we release this? Vote closes on Monday, June 27.
> > >>>
> > >>>
> > >>>
> > >>> Olga
> > >>>
> > >>>
> > >
> >
>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Richa Khandelwal <ri...@maprtech.com>.
Hi Guys,
I would vote +1 to remove the hadoop jar from the pig.jar. That would
definitely add to the flexibility of using Pig with any version or variant
of hadoop.
Also, there is "hdfs://" hardcoded at a few places in the pig source. I can
provide a patch which removes the hard-coding and makes it possible to use
pig on any distributed filesystem that can be used with hadoop, based on
pig.properties configuration file. Please let me know.
Thanks,
Richa
On Thu, Jun 23, 2011 at 2:56 PM, Olga Natkovich <ol...@yahoo-inc.com> wrote:
> My experience with sample command is that people use it for some quick
> testing and debugging not for production problems. This issue has been there
> since the initial introduction of sample. I still don't see a strong need to
> hold the beta release for it.
>
> Olga
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Thursday, June 23, 2011 10:14 AM
> To: dev@pig.apache.org
> Cc: dev@pig.apache.org
> Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
>
> There is a workaround - turning off optimizations, or at least
> pushUpFilter. I haven't checked 8 with the new logical plan disabled, that
> may work too.
> If this failed execution, I'd let it pass since there is a workaround, but
> as is it silently returns incorrect data, with no way for an analyst to know
> her statistics are now fundamentally wrong. Silent and subtle data
> corruption is just about the wrist big we can have.
> I can't really block the release, you guys can outvote me. But I'd rather
> you didn't; we can patch this problem today.
>
> On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>
> > Are you referring to PIG-2137? I have a few of questions on that before
> I vote for this release candidate or to reroll.
> >
> > Is this a new issue introduced in 0.9?
> >
> > Is there a workaround for this?
> >
> > We have already discussed that 0.9.0 will be beta quality, and a follow
> up release will be needed as users find bugs. As sample is not a heavily
> used feature I am inclined to view this bug as ok. You feel this is serious
> enough to block a beta release?
> >
> > Alan.
> >
> > On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
> >
> >> -1
> >>
> >> I discovered a critical bug in how SAMPLE is treated; I don't think we
> >> should release until it's fixed (Thejas is testing the fix).
> >>
> >> D
> >>
> >> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
> wrote:
> >>
> >>> I have created a candidate build for Pig 0.9.0 release. This release
> >>> introduces control structures, changes query parser, and performs
> semantic
> >>> cleanup.
> >>>
> >>> The rat report showed no issues in Java files outside of build
> directory.
> >>>
> >>>
> >>> Keys used to sign the release are available at
> >>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
> >>>
> >>>
> >>>
> >>> Please try it out:
> >>>
> >>>
> >>>
> >>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
> >>>
> >>>
> >>>
> >>> Build is also available in maven:
> >>>
> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
> >>>
> >>>
> >>>
> >>> Should we release this? Vote closes on Monday, June 27.
> >>>
> >>>
> >>>
> >>> Olga
> >>>
> >>>
> >
>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Scott Carey <sc...@richrelevance.com>.
Except for data analysts who use it to actually sample a portion of
partially processed data into something like R. I'll certainly be using a
patched release if this is not in. Is there plans to do a bugfix release
for 0.9? Or is the general feeling that a short cycle 0.10 is likely?
On 6/23/11 2:56 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
>My experience with sample command is that people use it for some quick
>testing and debugging not for production problems. This issue has been
>there since the initial introduction of sample. I still don't see a
>strong need to hold the beta release for it.
>
>Olga
>
>-----Original Message-----
>From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>Sent: Thursday, June 23, 2011 10:14 AM
>To: dev@pig.apache.org
>Cc: dev@pig.apache.org
>Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
>
>There is a workaround - turning off optimizations, or at least
>pushUpFilter. I haven't checked 8 with the new logical plan disabled,
>that may work too.
>If this failed execution, I'd let it pass since there is a workaround,
>but as is it silently returns incorrect data, with no way for an analyst
>to know her statistics are now fundamentally wrong. Silent and subtle
>data corruption is just about the wrist big we can have.
>I can't really block the release, you guys can outvote me. But I'd rather
>you didn't; we can patch this problem today.
>
>On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>
>> Are you referring to PIG-2137? I have a few of questions on that
>>before I vote for this release candidate or to reroll.
>>
>> Is this a new issue introduced in 0.9?
>>
>> Is there a workaround for this?
>>
>> We have already discussed that 0.9.0 will be beta quality, and a follow
>>up release will be needed as users find bugs. As sample is not a
>>heavily used feature I am inclined to view this bug as ok. You feel
>>this is serious enough to block a beta release?
>>
>> Alan.
>>
>> On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
>>
>>> -1
>>>
>>> I discovered a critical bug in how SAMPLE is treated; I don't think we
>>> should release until it's fixed (Thejas is testing the fix).
>>>
>>> D
>>>
>>> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com>
>>>wrote:
>>>
>>>> I have created a candidate build for Pig 0.9.0 release. This release
>>>> introduces control structures, changes query parser, and performs
>>>>semantic
>>>> cleanup.
>>>>
>>>> The rat report showed no issues in Java files outside of build
>>>>directory.
>>>>
>>>>
>>>> Keys used to sign the release are available at
>>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>>>>
>>>>
>>>>
>>>> Please try it out:
>>>>
>>>>
>>>>
>>>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>>>>
>>>>
>>>>
>>>> Build is also available in maven:
>>>>
>>>>https://repository.apache.org/content/repositories/orgapachepig-042/org
>>>>/apache/pig/pig/0.9.0/
>>>>
>>>>
>>>>
>>>> Should we release this? Vote closes on Monday, June 27.
>>>>
>>>>
>>>>
>>>> Olga
>>>>
>>>>
>>
RE: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
My experience with sample command is that people use it for some quick testing and debugging not for production problems. This issue has been there since the initial introduction of sample. I still don't see a strong need to hold the beta release for it.
Olga
-----Original Message-----
From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
Sent: Thursday, June 23, 2011 10:14 AM
To: dev@pig.apache.org
Cc: dev@pig.apache.org
Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
There is a workaround - turning off optimizations, or at least pushUpFilter. I haven't checked 8 with the new logical plan disabled, that may work too.
If this failed execution, I'd let it pass since there is a workaround, but as is it silently returns incorrect data, with no way for an analyst to know her statistics are now fundamentally wrong. Silent and subtle data corruption is just about the wrist big we can have.
I can't really block the release, you guys can outvote me. But I'd rather you didn't; we can patch this problem today.
On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> Are you referring to PIG-2137? I have a few of questions on that before I vote for this release candidate or to reroll.
>
> Is this a new issue introduced in 0.9?
>
> Is there a workaround for this?
>
> We have already discussed that 0.9.0 will be beta quality, and a follow up release will be needed as users find bugs. As sample is not a heavily used feature I am inclined to view this bug as ok. You feel this is serious enough to block a beta release?
>
> Alan.
>
> On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
>
>> -1
>>
>> I discovered a critical bug in how SAMPLE is treated; I don't think we
>> should release until it's fixed (Thejas is testing the fix).
>>
>> D
>>
>> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com> wrote:
>>
>>> I have created a candidate build for Pig 0.9.0 release. This release
>>> introduces control structures, changes query parser, and performs semantic
>>> cleanup.
>>>
>>> The rat report showed no issues in Java files outside of build directory.
>>>
>>>
>>> Keys used to sign the release are available at
>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>>>
>>>
>>>
>>> Please try it out:
>>>
>>>
>>>
>>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>>>
>>>
>>>
>>> Build is also available in maven:
>>> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>>>
>>>
>>>
>>> Should we release this? Vote closes on Monday, June 27.
>>>
>>>
>>>
>>> Olga
>>>
>>>
>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
There is a workaround - turning off optimizations, or at least pushUpFilter. I haven't checked 8 with the new logical plan disabled, that may work too.
If this failed execution, I'd let it pass since there is a workaround, but as is it silently returns incorrect data, with no way for an analyst to know her statistics are now fundamentally wrong. Silent and subtle data corruption is just about the wrist big we can have.
I can't really block the release, you guys can outvote me. But I'd rather you didn't; we can patch this problem today.
On Jun 23, 2011, at 7:47 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> Are you referring to PIG-2137? I have a few of questions on that before I vote for this release candidate or to reroll.
>
> Is this a new issue introduced in 0.9?
>
> Is there a workaround for this?
>
> We have already discussed that 0.9.0 will be beta quality, and a follow up release will be needed as users find bugs. As sample is not a heavily used feature I am inclined to view this bug as ok. You feel this is serious enough to block a beta release?
>
> Alan.
>
> On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
>
>> -1
>>
>> I discovered a critical bug in how SAMPLE is treated; I don't think we
>> should release until it's fixed (Thejas is testing the fix).
>>
>> D
>>
>> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com> wrote:
>>
>>> I have created a candidate build for Pig 0.9.0 release. This release
>>> introduces control structures, changes query parser, and performs semantic
>>> cleanup.
>>>
>>> The rat report showed no issues in Java files outside of build directory.
>>>
>>>
>>> Keys used to sign the release are available at
>>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>>>
>>>
>>>
>>> Please try it out:
>>>
>>>
>>>
>>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>>>
>>>
>>>
>>> Build is also available in maven:
>>> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>>>
>>>
>>>
>>> Should we release this? Vote closes on Monday, June 27.
>>>
>>>
>>>
>>> Olga
>>>
>>>
>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Alan Gates <ga...@yahoo-inc.com>.
Are you referring to PIG-2137? I have a few of questions on that
before I vote for this release candidate or to reroll.
Is this a new issue introduced in 0.9?
Is there a workaround for this?
We have already discussed that 0.9.0 will be beta quality, and a
follow up release will be needed as users find bugs. As sample is not
a heavily used feature I am inclined to view this bug as ok. You feel
this is serious enough to block a beta release?
Alan.
On Jun 22, 2011, at 9:11 PM, Dmitriy Ryaboy wrote:
> -1
>
> I discovered a critical bug in how SAMPLE is treated; I don't think we
> should release until it's fixed (Thejas is testing the fix).
>
> D
>
> On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <olgan@yahoo-
> inc.com> wrote:
>
>> I have created a candidate build for Pig 0.9.0 release. This release
>> introduces control structures, changes query parser, and performs
>> semantic
>> cleanup.
>>
>> The rat report showed no issues in Java files outside of build
>> directory.
>>
>>
>> Keys used to sign the release are available at
>> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>>
>>
>>
>> Please try it out:
>>
>>
>>
>> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>>
>>
>>
>> Build is also available in maven:
>> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>>
>>
>>
>> Should we release this? Vote closes on Monday, June 27.
>>
>>
>>
>> Olga
>>
>>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
-1
I discovered a critical bug in how SAMPLE is treated; I don't think we
should release until it's fixed (Thejas is testing the fix).
D
On Wed, Jun 22, 2011 at 5:56 PM, Olga Natkovich <ol...@yahoo-inc.com> wrote:
> I have created a candidate build for Pig 0.9.0 release. This release
> introduces control structures, changes query parser, and performs semantic
> cleanup.
>
> The rat report showed no issues in Java files outside of build directory.
>
>
> Keys used to sign the release are available at
> http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>
>
>
> Please try it out:
>
>
>
> http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>
>
>
> Build is also available in maven:
> https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
>
>
>
> Should we release this? Vote closes on Monday, June 27.
>
>
>
> Olga
>
>
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Scott Carey <sc...@richrelevance.com>.
I have never once been able to use Pig in production out of the box
because of how it is packaged by default -- including too many of its
dependencies in the jar. I've complained about it in a couple Jira
tickets related to the output maven artifacts -- it is not a standard
artifact if it bundles any dependencies at all...
In every project I use pig as a Maven artifact I have to use the
withouthadoop variant at minimum, but in most I have to manually make my
own jar rather than simply configure my project to use an official Pig
artifact and compatible downstream dependencies.
I'm happy to keep rebuilding Pig for now -- this is an enhancement request
and not something technically wrong that can block a release.
in On 6/23/11 2:54 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
>I don't believe we have seen any requests for this in the apache mailing
>lists. Moreover they can build one from the source code enclosed. Our
>package is already 40MB and I had to split it 5 ways to upload to the
>people.apache.org. I don't think we want to make it any bigger than
>necessary.
>
>Olga
>
>From: Thejas M Nair
>Sent: Thursday, June 23, 2011 2:35 PM
>To: dev@pig.apache.org; Olga Natkovich
>Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
>
>I have seen many users have problems when they use the pig.jar which is
>bundled with hadoop, because their cluster is using a different version
>of hadoop.
>I think it will be convenient for such users, if pig ships with a
>jar-without hadoop.
>
>-Thejas
>
>
>
>On 6/22/11 5:56 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
>I have created a candidate build for Pig 0.9.0 release. This release
>introduces control structures, changes query parser, and performs
>semantic cleanup.
>
>The rat report showed no issues in Java files outside of build directory.
>
>
>Keys used to sign the release are available at
>http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
>
>
>
>Please try it out:
>
>
>
>http://people.apache.org/~olga/pig-0.9.0-candidate-0/
>
>
>
>Build is also available in maven:
>https://repository.apache.org/content/repositories/orgapachepig-042/org/ap
>ache/pig/pig/0.9.0/
>
>
>
>Should we release this? Vote closes on Monday, June 27.
>
>
>
>Olga
>
>
>
>--
RE: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
I don't believe we have seen any requests for this in the apache mailing lists. Moreover they can build one from the source code enclosed. Our package is already 40MB and I had to split it 5 ways to upload to the people.apache.org. I don't think we want to make it any bigger than necessary.
Olga
From: Thejas M Nair
Sent: Thursday, June 23, 2011 2:35 PM
To: dev@pig.apache.org; Olga Natkovich
Subject: Re: [VOTE] Release Pig 0.9.0 (candidate 0)
I have seen many users have problems when they use the pig.jar which is bundled with hadoop, because their cluster is using a different version of hadoop.
I think it will be convenient for such users, if pig ships with a jar-without hadoop.
-Thejas
On 6/22/11 5:56 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
I have created a candidate build for Pig 0.9.0 release. This release introduces control structures, changes query parser, and performs semantic cleanup.
The rat report showed no issues in Java files outside of build directory.
Keys used to sign the release are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
Please try it out:
http://people.apache.org/~olga/pig-0.9.0-candidate-0/
Build is also available in maven: https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
Should we release this? Vote closes on Monday, June 27.
Olga
--
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Thejas M Nair <te...@yahoo-inc.com>.
-1
I think PIG-2144 is a critical bug that needs to be fixed before the release.
I am working on a fix for it.
-Thejas
On 6/22/11 5:56 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
I have created a candidate build for Pig 0.9.0 release. This release introduces control structures, changes query parser, and performs semantic cleanup.
The rat report showed no issues in Java files outside of build directory.
Keys used to sign the release are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
Please try it out:
http://people.apache.org/~olga/pig-0.9.0-candidate-0/
Build is also available in maven: https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
Should we release this? Vote closes on Monday, June 27.
Olga
--
Re: [VOTE] Release Pig 0.9.0 (candidate 0)
Posted by Thejas M Nair <te...@yahoo-inc.com>.
I have seen many users have problems when they use the pig.jar which is bundled with hadoop, because their cluster is using a different version of hadoop.
I think it will be convenient for such users, if pig ships with a jar-without hadoop.
-Thejas
On 6/22/11 5:56 PM, "Olga Natkovich" <ol...@yahoo-inc.com> wrote:
I have created a candidate build for Pig 0.9.0 release. This release introduces control structures, changes query parser, and performs semantic cleanup.
The rat report showed no issues in Java files outside of build directory.
Keys used to sign the release are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup.
Please try it out:
http://people.apache.org/~olga/pig-0.9.0-candidate-0/
Build is also available in maven: https://repository.apache.org/content/repositories/orgapachepig-042/org/apache/pig/pig/0.9.0/
Should we release this? Vote closes on Monday, June 27.
Olga
--