You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Jonathan Coveney <jc...@gmail.com> on 2011/11/11 21:22:16 UTC
Should you be able to nest operators in a foreach like you normally can?
This works:
a = load 'thing';
b = foreach (group a all) {
sorted = order a by $0;
top = limit sorted 100;
generate flatten(top);
}
This does not (neither in pig8 nor in pig9, to my knowledge):
a = load 'thing';
b = foreach (group a all) {
top = limit (order a by $0) 100;
generate flatten(top);
}
Just curious if this is a design choice, design limitation, annoying to
implement, etc. Thought I'd bring it up.
Jon, your man for "nested foreach" posts to the dev listserv
Re: Should you be able to nest operators in a foreach like you
normally can?
Posted by Jonathan Coveney <jc...@gmail.com>.
Here we go:
https://issues.apache.org/jira/browse/PIG-2360
I need to work on getting up to speed on how ANTLR works...
2011/11/13 Gianmarco De Francisci Morales <gd...@apache.org>
> I think there is no good reason to leave it like this.
> I was already thinking about making a major update to Pig grammar in order
> to simplify and rationalize it.
> See also PIG-2138 for example.
> If I manage to bring myself to work on it, this improvement should
> definitely go in as well.
>
> Could you open a Jira to keep track of the idea?
>
> Cheers,
> --
> Gianmarco
>
>
>
> On Sun, Nov 13, 2011 at 02:25, Jonathan Coveney <jc...@gmail.com>
> wrote:
>
> > It's not the most important issue by far (I think the scoped names issue
> is
> > more frustrating, as far as usability), it's just a bit counterintuitive
> > that the syntax would not be consistent. What reason is there (besides
> > implementation) for a user to assume that syntax inside of the foreach
> and
> > outside of the foreach would be different (except for the nested foreach
> > which is a documented technical limitation that someone is working on)?
> >
> > 2011/11/12 Gianmarco De Francisci Morales <
> gianmarco.dfmorales@imtlucca.it
> > >
> >
> > > I think the answer is:
> > > annoying to implement and with not so compelling use cases.
> > >
> > > The nested expressions take a different path in the parser in order to
> > > distinguish them from the non-nested ones. So you would probably need
> to
> > > replicate some code here and there to make this work. It can be done if
> > you
> > > think it is worth doing.
> > >
> > > Cheers,
> > > --
> > > Gianmarco
> > >
> > >
> > >
> > > On Fri, Nov 11, 2011 at 21:22, Jonathan Coveney <jc...@gmail.com>
> > > wrote:
> > >
> > > > This works:
> > > >
> > > > a = load 'thing';
> > > > b = foreach (group a all) {
> > > > sorted = order a by $0;
> > > > top = limit sorted 100;
> > > > generate flatten(top);
> > > > }
> > > >
> > > > This does not (neither in pig8 nor in pig9, to my knowledge):
> > > >
> > > > a = load 'thing';
> > > > b = foreach (group a all) {
> > > > top = limit (order a by $0) 100;
> > > > generate flatten(top);
> > > > }
> > > >
> > > > Just curious if this is a design choice, design limitation, annoying
> to
> > > > implement, etc. Thought I'd bring it up.
> > > >
> > > > Jon, your man for "nested foreach" posts to the dev listserv
> > > >
> > >
> >
>
Re: Should you be able to nest operators in a foreach like you
normally can?
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
I think there is no good reason to leave it like this.
I was already thinking about making a major update to Pig grammar in order
to simplify and rationalize it.
See also PIG-2138 for example.
If I manage to bring myself to work on it, this improvement should
definitely go in as well.
Could you open a Jira to keep track of the idea?
Cheers,
--
Gianmarco
On Sun, Nov 13, 2011 at 02:25, Jonathan Coveney <jc...@gmail.com> wrote:
> It's not the most important issue by far (I think the scoped names issue is
> more frustrating, as far as usability), it's just a bit counterintuitive
> that the syntax would not be consistent. What reason is there (besides
> implementation) for a user to assume that syntax inside of the foreach and
> outside of the foreach would be different (except for the nested foreach
> which is a documented technical limitation that someone is working on)?
>
> 2011/11/12 Gianmarco De Francisci Morales <gianmarco.dfmorales@imtlucca.it
> >
>
> > I think the answer is:
> > annoying to implement and with not so compelling use cases.
> >
> > The nested expressions take a different path in the parser in order to
> > distinguish them from the non-nested ones. So you would probably need to
> > replicate some code here and there to make this work. It can be done if
> you
> > think it is worth doing.
> >
> > Cheers,
> > --
> > Gianmarco
> >
> >
> >
> > On Fri, Nov 11, 2011 at 21:22, Jonathan Coveney <jc...@gmail.com>
> > wrote:
> >
> > > This works:
> > >
> > > a = load 'thing';
> > > b = foreach (group a all) {
> > > sorted = order a by $0;
> > > top = limit sorted 100;
> > > generate flatten(top);
> > > }
> > >
> > > This does not (neither in pig8 nor in pig9, to my knowledge):
> > >
> > > a = load 'thing';
> > > b = foreach (group a all) {
> > > top = limit (order a by $0) 100;
> > > generate flatten(top);
> > > }
> > >
> > > Just curious if this is a design choice, design limitation, annoying to
> > > implement, etc. Thought I'd bring it up.
> > >
> > > Jon, your man for "nested foreach" posts to the dev listserv
> > >
> >
>
Re: Should you be able to nest operators in a foreach like you
normally can?
Posted by Jonathan Coveney <jc...@gmail.com>.
It's not the most important issue by far (I think the scoped names issue is
more frustrating, as far as usability), it's just a bit counterintuitive
that the syntax would not be consistent. What reason is there (besides
implementation) for a user to assume that syntax inside of the foreach and
outside of the foreach would be different (except for the nested foreach
which is a documented technical limitation that someone is working on)?
2011/11/12 Gianmarco De Francisci Morales <gi...@imtlucca.it>
> I think the answer is:
> annoying to implement and with not so compelling use cases.
>
> The nested expressions take a different path in the parser in order to
> distinguish them from the non-nested ones. So you would probably need to
> replicate some code here and there to make this work. It can be done if you
> think it is worth doing.
>
> Cheers,
> --
> Gianmarco
>
>
>
> On Fri, Nov 11, 2011 at 21:22, Jonathan Coveney <jc...@gmail.com>
> wrote:
>
> > This works:
> >
> > a = load 'thing';
> > b = foreach (group a all) {
> > sorted = order a by $0;
> > top = limit sorted 100;
> > generate flatten(top);
> > }
> >
> > This does not (neither in pig8 nor in pig9, to my knowledge):
> >
> > a = load 'thing';
> > b = foreach (group a all) {
> > top = limit (order a by $0) 100;
> > generate flatten(top);
> > }
> >
> > Just curious if this is a design choice, design limitation, annoying to
> > implement, etc. Thought I'd bring it up.
> >
> > Jon, your man for "nested foreach" posts to the dev listserv
> >
>
Re: Should you be able to nest operators in a foreach like you
normally can?
Posted by Gianmarco De Francisci Morales <gi...@imtlucca.it>.
I think the answer is:
annoying to implement and with not so compelling use cases.
The nested expressions take a different path in the parser in order to
distinguish them from the non-nested ones. So you would probably need to
replicate some code here and there to make this work. It can be done if you
think it is worth doing.
Cheers,
--
Gianmarco
On Fri, Nov 11, 2011 at 21:22, Jonathan Coveney <jc...@gmail.com> wrote:
> This works:
>
> a = load 'thing';
> b = foreach (group a all) {
> sorted = order a by $0;
> top = limit sorted 100;
> generate flatten(top);
> }
>
> This does not (neither in pig8 nor in pig9, to my knowledge):
>
> a = load 'thing';
> b = foreach (group a all) {
> top = limit (order a by $0) 100;
> generate flatten(top);
> }
>
> Just curious if this is a design choice, design limitation, annoying to
> implement, etc. Thought I'd bring it up.
>
> Jon, your man for "nested foreach" posts to the dev listserv
>