You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Chris K Wensel <ch...@wensel.net> on 2008/01/22 04:46:29 UTC
Cascading
Hey all,
Some months ago I decided to poke around deeper with Hadoop, after
having been a Nutch user for some time.
After way to much of the aforementioned poking, I decided to give my
work a name, a home, and set it free.
You can find the results here:
http://www.cascading.org/
It seems to be useful, but that may not hold true for anyone other
than me.
But do please give it a look and let me know what you think, if you
are so inclined to think about it. If you use it, I would love to hear
about that as well.
If it works for whatever you need it for, great news. If not, feel
free to complain or contribute. It's hosted on google code and
subsequently has a mail list and issue tracker.
The lack of comprehensive documentation is slowly being remedied.
regards,
ckw
Chris K Wensel
chris@wensel.net
http://chris.wensel.net/
Re: Cascading
Posted by Chris K Wensel <ch...@wensel.net>.
Hey thanks!
I wasn't aware of Dryad until after I got started. But it is much
lower level than Cascading.
That is, I believe you can implement the MapReduce model using Dryad,
Cascading (currently) relies on the MapReduce model.
Pig and Jaql also showed up on my radar after I got started. but I see
them as complimentary, they can participate in a Flow, or be a kind of
Flow. that's currently undecided.
But it makes perfect sense to use Cascading to accumulate and cleanup
data, and then hand it to Pig/Jaql for the really hard stuff, then let
Cascading put on the finishing touches.
cheers
ckw
On Jan 23, 2008, at 12:27 PM, Steve Severance wrote:
> Hi Chris,
> Really interesting stuff. Definitely very useful for building more
> complex apps. I am wondering if you were inspired at all by
> Microsofts Dryad
> project which implements a vertex passed distributed processing
> platform.
> Once again good job.
>
> Steve
>
>> -----Original Message-----
>> From: Chris K Wensel [mailto:chris@wensel.net]
>> Sent: Wednesday, January 23, 2008 1:34 PM
>> To: core-user@hadoop.apache.org
>> Subject: Re: Cascading
>>
>> Thought I would briefly mention a new overview page that attempts to
>> explain what Cascading does in a little more detail.
>>
>> It should answer some of the questions I've received.
>>
>> http://www.cascading.org/overview.html
>>
>> I'll leave you alone now.. <grin>
>>
>> cheers
>> ckw
>>
>> On Jan 21, 2008, at 7:46 PM, Chris K Wensel wrote:
>>
>>> Hey all,
>>>
>>> Some months ago I decided to poke around deeper with Hadoop, after
>>> having been a Nutch user for some time.
>>>
>>> After way to much of the aforementioned poking, I decided to give my
>>> work a name, a home, and set it free.
>>>
>>> You can find the results here:
>>> http://www.cascading.org/
>>>
>>> It seems to be useful, but that may not hold true for anyone other
>>> than me.
>>>
>>> But do please give it a look and let me know what you think, if you
>>> are so inclined to think about it. If you use it, I would love to
>>> hear about that as well.
>>>
>>> If it works for whatever you need it for, great news. If not, feel
>>> free to complain or contribute. It's hosted on google code and
>>> subsequently has a mail list and issue tracker.
>>>
>>> The lack of comprehensive documentation is slowly being remedied.
>>>
>>> regards,
>>> ckw
>>>
>>> Chris K Wensel
>>> chris@wensel.net
>>> http://chris.wensel.net/
>>>
>>>
>>>
>>
>> Chris K Wensel
>> chris@wensel.net
>> http://chris.wensel.net/
>>
>
>
>
Chris K Wensel
chris@wensel.net
http://chris.wensel.net/
RE: Cascading
Posted by Steve Severance <li...@medwizard.net>.
Hi Chris,
Really interesting stuff. Definitely very useful for building more
complex apps. I am wondering if you were inspired at all by Microsofts Dryad
project which implements a vertex passed distributed processing platform.
Once again good job.
Steve
> -----Original Message-----
> From: Chris K Wensel [mailto:chris@wensel.net]
> Sent: Wednesday, January 23, 2008 1:34 PM
> To: core-user@hadoop.apache.org
> Subject: Re: Cascading
>
> Thought I would briefly mention a new overview page that attempts to
> explain what Cascading does in a little more detail.
>
> It should answer some of the questions I've received.
>
> http://www.cascading.org/overview.html
>
> I'll leave you alone now.. <grin>
>
> cheers
> ckw
>
> On Jan 21, 2008, at 7:46 PM, Chris K Wensel wrote:
>
> > Hey all,
> >
> > Some months ago I decided to poke around deeper with Hadoop, after
> > having been a Nutch user for some time.
> >
> > After way to much of the aforementioned poking, I decided to give my
> > work a name, a home, and set it free.
> >
> > You can find the results here:
> > http://www.cascading.org/
> >
> > It seems to be useful, but that may not hold true for anyone other
> > than me.
> >
> > But do please give it a look and let me know what you think, if you
> > are so inclined to think about it. If you use it, I would love to
> > hear about that as well.
> >
> > If it works for whatever you need it for, great news. If not, feel
> > free to complain or contribute. It's hosted on google code and
> > subsequently has a mail list and issue tracker.
> >
> > The lack of comprehensive documentation is slowly being remedied.
> >
> > regards,
> > ckw
> >
> > Chris K Wensel
> > chris@wensel.net
> > http://chris.wensel.net/
> >
> >
> >
>
> Chris K Wensel
> chris@wensel.net
> http://chris.wensel.net/
>
Re: Cascading
Posted by Chris K Wensel <ch...@wensel.net>.
Thought I would briefly mention a new overview page that attempts to
explain what Cascading does in a little more detail.
It should answer some of the questions I've received.
http://www.cascading.org/overview.html
I'll leave you alone now.. <grin>
cheers
ckw
On Jan 21, 2008, at 7:46 PM, Chris K Wensel wrote:
> Hey all,
>
> Some months ago I decided to poke around deeper with Hadoop, after
> having been a Nutch user for some time.
>
> After way to much of the aforementioned poking, I decided to give my
> work a name, a home, and set it free.
>
> You can find the results here:
> http://www.cascading.org/
>
> It seems to be useful, but that may not hold true for anyone other
> than me.
>
> But do please give it a look and let me know what you think, if you
> are so inclined to think about it. If you use it, I would love to
> hear about that as well.
>
> If it works for whatever you need it for, great news. If not, feel
> free to complain or contribute. It's hosted on google code and
> subsequently has a mail list and issue tracker.
>
> The lack of comprehensive documentation is slowly being remedied.
>
> regards,
> ckw
>
> Chris K Wensel
> chris@wensel.net
> http://chris.wensel.net/
>
>
>
Chris K Wensel
chris@wensel.net
http://chris.wensel.net/