You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Olga Natkovich <ol...@yahoo-inc.com> on 2008/05/30 22:25:22 UTC
Becoming hadoop subproject
Hi,
What do people think about becoming a Hadoop subproject?
A couple of advantages of this approach:
- Having direct access to hadoop user community would be good for Pig
adoption
- Being part of a more mature open source project would help in
attracting more developers and committers. Also, it will allow us to
avoid the cost of administrative overhead.
Now this, of course, would not mean architecturally depending on Hadoop.
I believe that maintaining backend independence is an important goal for
Pig.
Comments?
Olga
Re: Becoming hadoop subproject
Posted by Doug Cutting <cu...@apache.org>.
Torsten Curdt wrote:
> 1) Provide the current hadoop distributions with PIG integrated. (Should
> be communicated with the hadoop project of course)
For Pig to ship as a part of Hadoop Core releases, Pig would probably
need to become a part of Hadoop Core, e.g., as a contrib module. It
makes more sense for Hadoop to ship as a part of Pig releases.
Doug
Re: Becoming hadoop subproject
Posted by Doug Cutting <cu...@apache.org>.
Olga Natkovich wrote:
> So seems like the current thinking is that it is a good idea for Pig to
> join Hadoop at a later point. For now Pig needs to focus on making
> progress on its own, perhaps trying for an incubator release? Is this
> correct? Any other opinions?
A release from the incubator would be great progress towards graduation.
Doug
RE: Becoming hadoop subproject
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
So seems like the current thinking is that it is a good idea for Pig to
join Hadoop at a later point. For now Pig needs to focus on making
progress on its own, perhaps trying for an incubator release? Is this
correct? Any other opinions?
Olga
> -----Original Message-----
> From: Torsten Curdt [mailto:tcurdt@apache.org]
> Sent: Monday, June 02, 2008 6:04 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Becoming hadoop subproject
>
>
> On Jun 2, 2008, at 14:27, pi song wrote:
>
> > 1) Whether we're becoming a subproject or not, this idea is
> a kind of
> > cool.
> > We can just ship out as a utility in Hadoop to harvest more users.
> > More Pig
> > users = more Hadoop users anyway so it doesn't seem to
> conflict with
> > anything.
>
> Yepp
>
> > 2) Let's do like the way they promoted Ruby on Rails. Wow!! you can
> > make a website in 5 minutes!!
>
> Yepp ...it certainly worked for them :)
>
> cheers
> --
> Torsten
>
Re: Becoming hadoop subproject
Posted by Torsten Curdt <tc...@apache.org>.
On Jun 2, 2008, at 14:27, pi song wrote:
> 1) Whether we're becoming a subproject or not, this idea is a kind
> of cool.
> We can just ship out as a utility in Hadoop to harvest more users.
> More Pig
> users = more Hadoop users anyway so it doesn't seem to conflict with
> anything.
Yepp
> 2) Let's do like the way they promoted Ruby on Rails. Wow!! you can
> make a
> website in 5 minutes!!
Yepp ...it certainly worked for them :)
cheers
--
Torsten
Re: Becoming hadoop subproject
Posted by Chris Olston <ol...@yahoo-inc.com>.
We are working on a Pig Tutorial, scheduled to be placed on the
apache site in June.
-Chris
On Jun 2, 2008, at 5:27 AM, pi song wrote:
> 1) Whether we're becoming a subproject or not, this idea is a kind
> of cool.
> We can just ship out as a utility in Hadoop to harvest more users.
> More Pig
> users = more Hadoop users anyway so it doesn't seem to conflict with
> anything.
>
> 2) Let's do like the way they promoted Ruby on Rails. Wow!! you can
> make a
> website in 5 minutes!!
>
> Pi
>
> On Mon, Jun 2, 2008 at 10:02 PM, Torsten Curdt <tc...@apache.org>
> wrote:
>
>> Currently, there are really 1-2 people that pop out in design
>> discussions.
>>> I
>>> would agree on anything that will make us more like a real
>>> community!
>>>
>>> However, the main idea will never change. The more usable
>>> application =
>>> more
>>> users = more developers.
>>>
>>
>> The chicken and the egg ...I know
>>
>> But I guess there are a couple of things pig could do.
>>
>> 1) Provide the current hadoop distributions with PIG integrated.
>> (Should be
>> communicated with the hadoop project of course)
>>
>> 2) A tutorial / screencast what you can already do with it. Maybe run
>> through the usual examples.
>>
>> 3) Make a list of companies/projects that already use PIG in
>> production
>>
>> 4) Get some people to blog about this once things are in place
>>
>> I know this isn't all nice and sexy development stuff. But I am
>> sure it
>> will help to find a few more hands.
>>
>> My 2 cents
>>
>> cheers
>> --
>> Torsten
>>
--
Christopher Olston, Ph.D.
Sr. Research Scientist
Yahoo! Research
Re: Becoming hadoop subproject
Posted by pi song <pi...@gmail.com>.
1) Whether we're becoming a subproject or not, this idea is a kind of cool.
We can just ship out as a utility in Hadoop to harvest more users. More Pig
users = more Hadoop users anyway so it doesn't seem to conflict with
anything.
2) Let's do like the way they promoted Ruby on Rails. Wow!! you can make a
website in 5 minutes!!
Pi
On Mon, Jun 2, 2008 at 10:02 PM, Torsten Curdt <tc...@apache.org> wrote:
> Currently, there are really 1-2 people that pop out in design discussions.
>> I
>> would agree on anything that will make us more like a real community!
>>
>> However, the main idea will never change. The more usable application =
>> more
>> users = more developers.
>>
>
> The chicken and the egg ...I know
>
> But I guess there are a couple of things pig could do.
>
> 1) Provide the current hadoop distributions with PIG integrated. (Should be
> communicated with the hadoop project of course)
>
> 2) A tutorial / screencast what you can already do with it. Maybe run
> through the usual examples.
>
> 3) Make a list of companies/projects that already use PIG in production
>
> 4) Get some people to blog about this once things are in place
>
> I know this isn't all nice and sexy development stuff. But I am sure it
> will help to find a few more hands.
>
> My 2 cents
>
> cheers
> --
> Torsten
>
Re: Becoming hadoop subproject
Posted by Torsten Curdt <tc...@apache.org>.
> Currently, there are really 1-2 people that pop out in design
> discussions. I
> would agree on anything that will make us more like a real community!
>
> However, the main idea will never change. The more usable
> application = more
> users = more developers.
The chicken and the egg ...I know
But I guess there are a couple of things pig could do.
1) Provide the current hadoop distributions with PIG integrated.
(Should be communicated with the hadoop project of course)
2) A tutorial / screencast what you can already do with it. Maybe run
through the usual examples.
3) Make a list of companies/projects that already use PIG in production
4) Get some people to blog about this once things are in place
I know this isn't all nice and sexy development stuff. But I am sure
it will help to find a few more hands.
My 2 cents
cheers
--
Torsten
Re: Becoming hadoop subproject
Posted by pi song <pi...@gmail.com>.
Currently, there are really 1-2 people that pop out in design discussions. I
would agree on anything that will make us more like a real community!
However, the main idea will never change. The more usable application = more
users = more developers.
Pi
On Sat, May 31, 2008 at 7:05 AM, Torsten Curdt <tc...@apache.org> wrote:
> Don't get me wrong ...but personally I would like to see a little more
> activity on dev list for a graduation.
> That said - I also think it would be beneficial. For both projects!
>
> cheers
> --
> Torsten
>
>
> On May 30, 2008, at 22:25, Olga Natkovich wrote:
>
> Hi,
>>
>> What do people think about becoming a Hadoop subproject?
>>
>> A couple of advantages of this approach:
>>
>> - Having direct access to hadoop user community would be good for Pig
>> adoption
>> - Being part of a more mature open source project would help in
>> attracting more developers and committers. Also, it will allow us to
>> avoid the cost of administrative overhead.
>>
>> Now this, of course, would not mean architecturally depending on Hadoop.
>> I believe that maintaining backend independence is an important goal for
>> Pig.
>>
>> Comments?
>>
>> Olga
>>
>
>
Re: Becoming hadoop subproject
Posted by Torsten Curdt <tc...@apache.org>.
Don't get me wrong ...but personally I would like to see a little more
activity on dev list for a graduation.
That said - I also think it would be beneficial. For both projects!
cheers
--
Torsten
On May 30, 2008, at 22:25, Olga Natkovich wrote:
> Hi,
>
> What do people think about becoming a Hadoop subproject?
>
> A couple of advantages of this approach:
>
> - Having direct access to hadoop user community would be good for Pig
> adoption
> - Being part of a more mature open source project would help in
> attracting more developers and committers. Also, it will allow us to
> avoid the cost of administrative overhead.
>
> Now this, of course, would not mean architecturally depending on
> Hadoop.
> I believe that maintaining backend independence is an important goal
> for
> Pig.
>
> Comments?
>
> Olga