You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Mayur Rustagi <ma...@gmail.com> on 2014/08/28 07:03:25 UTC
Update on Pig on Spark initiative
Hi,
We have migrated Pig functionality on top of Spark passing 100% e2e for
success cases in pig test suite. That means UDF, Joins & other
functionality is working quite nicely. We are in the process of merging
with Apache Pig trunk(something that should happen over the next 2 weeks).
Meanwhile if you are interested in giving it a go, you can try it at
https://github.com/sigmoidanalytics/spork
This contains all the major changes but may not have all the patches
required for 100% e2e, if you are trying it out let me know any issues you
face
Whole bunch of folks contributed on this
Julien Le Dem (Twitter), Praveen R (Sigmoid Analytics), Akhil Das (Sigmoid
Analytics), Bill Graham (Twitter), Dmitriy Ryaboy (Twitter), Kamal Banga
(Sigmoid Analytics), Anish Haldiya (Sigmoid Analytics), Aniket Mokashi
(Google), Greg Owen (DataBricks), Amit Kumar Behera (Sigmoid Analytics),
Mahesh Kalakoti (Sigmoid Analytics)
Not to mention Spark & Pig communities.
Regards
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>
Re: Update on Pig on Spark initiative
Posted by Russell Jurney <ru...@gmail.com>.
This is really exciting! Thanks so much for this work, I think you've
guaranteed Pig's continued vitality.
On Wednesday, August 27, 2014, Matei Zaharia <ma...@gmail.com>
wrote:
> Awesome to hear this, Mayur! Thanks for putting this together.
>
> Matei
>
> On August 27, 2014 at 10:04:12 PM, Mayur Rustagi (mayur.rustagi@gmail.com
> <javascript:_e(%7B%7D,'cvml','mayur.rustagi@gmail.com');>) wrote:
>
> Hi,
> We have migrated Pig functionality on top of Spark passing 100% e2e for
> success cases in pig test suite. That means UDF, Joins & other
> functionality is working quite nicely. We are in the process of merging
> with Apache Pig trunk(something that should happen over the next 2 weeks).
> Meanwhile if you are interested in giving it a go, you can try it at
> https://github.com/sigmoidanalytics/spork
> This contains all the major changes but may not have all the patches
> required for 100% e2e, if you are trying it out let me know any issues you
> face
>
> Whole bunch of folks contributed on this
>
> Julien Le Dem (Twitter), Praveen R (Sigmoid Analytics), Akhil Das
> (Sigmoid Analytics), Bill Graham (Twitter), Dmitriy Ryaboy (Twitter), Kamal
> Banga (Sigmoid Analytics), Anish Haldiya (Sigmoid Analytics), Aniket
> Mokashi (Google), Greg Owen (DataBricks), Amit Kumar Behera (Sigmoid
> Analytics), Mahesh Kalakoti (Sigmoid Analytics)
>
> Not to mention Spark & Pig communities.
>
> Regards
> Mayur Rustagi
> Ph: +1 (760) 203 3257
> http://www.sigmoidanalytics.com
> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>
>
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com
Re: Update on Pig on Spark initiative
Posted by Matei Zaharia <ma...@gmail.com>.
Awesome to hear this, Mayur! Thanks for putting this together.
Matei
On August 27, 2014 at 10:04:12 PM, Mayur Rustagi (mayur.rustagi@gmail.com) wrote:
Hi,
We have migrated Pig functionality on top of Spark passing 100% e2e for success cases in pig test suite. That means UDF, Joins & other functionality is working quite nicely. We are in the process of merging with Apache Pig trunk(something that should happen over the next 2 weeks).
Meanwhile if you are interested in giving it a go, you can try it at https://github.com/sigmoidanalytics/spork
This contains all the major changes but may not have all the patches required for 100% e2e, if you are trying it out let me know any issues you face
Whole bunch of folks contributed on this
Julien Le Dem (Twitter), Praveen R (Sigmoid Analytics), Akhil Das (Sigmoid Analytics), Bill Graham (Twitter), Dmitriy Ryaboy (Twitter), Kamal Banga (Sigmoid Analytics), Anish Haldiya (Sigmoid Analytics), Aniket Mokashi (Google), Greg Owen (DataBricks), Amit Kumar Behera (Sigmoid Analytics), Mahesh Kalakoti (Sigmoid Analytics)
Not to mention Spark & Pig communities.
Regards
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi
Re: Update on Pig on Spark initiative
Posted by Matei Zaharia <ma...@gmail.com>.
Awesome to hear this, Mayur! Thanks for putting this together.
Matei
On August 27, 2014 at 10:04:12 PM, Mayur Rustagi (mayur.rustagi@gmail.com) wrote:
Hi,
We have migrated Pig functionality on top of Spark passing 100% e2e for success cases in pig test suite. That means UDF, Joins & other functionality is working quite nicely. We are in the process of merging with Apache Pig trunk(something that should happen over the next 2 weeks).
Meanwhile if you are interested in giving it a go, you can try it at https://github.com/sigmoidanalytics/spork
This contains all the major changes but may not have all the patches required for 100% e2e, if you are trying it out let me know any issues you face
Whole bunch of folks contributed on this
Julien Le Dem (Twitter), Praveen R (Sigmoid Analytics), Akhil Das (Sigmoid Analytics), Bill Graham (Twitter), Dmitriy Ryaboy (Twitter), Kamal Banga (Sigmoid Analytics), Anish Haldiya (Sigmoid Analytics), Aniket Mokashi (Google), Greg Owen (DataBricks), Amit Kumar Behera (Sigmoid Analytics), Mahesh Kalakoti (Sigmoid Analytics)
Not to mention Spark & Pig communities.
Regards
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi