You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Alex McLintock <al...@gmail.com> on 2012/09/25 10:23:34 UTC
IDE choice?
Forgive me for asking a FAQ but what is the current IDE of choice for Pig?
I used to use a text editor and command line.
I understand that PigPen (eclipse plugin) is no longer supported and
does not work with current Hadoop. (But I haven't seen the
documentation or wiki updated.)
I have heard of PigEditor and will try it out.
We normally use IntelliJ so would love to hear anything that helps
integrate that.
I am also a bit confused as to testing Pig. I am sitting in front of a
Windows box and would love to be able to debug my scripts locally, but
I don't see that is possible.
If we have a bunch of developers I wonder if they should be all
testing on the same cluster, or have individual clusters, and how they
should be spun up and down....
Re: IDE choice?
Posted by Russell Jurney <ru...@gmail.com>.
I use pig local mode extensively, which cuts development time
dramatically. If you can do this in Cygwin, it would help you be much
more productive.
pig -l /tmp -x local -v -w
This is covered in chapters 3 and 4 of my book, in which we go from
localhost to a cluster on EMR.
http://ofps.oreilly.com/titles/9781449326265/chapter_3.html
For UDFs I use IntelliJ. Would setup instructions be helpful? I could blog them.
Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com
On Sep 25, 2012, at 10:46 AM, Siddharth Tiwari <si...@tcs.com> wrote:
> Hi Alex,
>
> You can always use cygwin for development purposes only. It will work like charm, with no fuss.
>
> Regards
> Siddharth
>
>
> -----Alex McLintock <al...@gmail.com> wrote: -----
> To: user@pig.apache.org
> From: Alex McLintock <al...@gmail.com>
> Date: 09/25/2012 01:54PM
> Subject: IDE choice?
>
> Forgive me for asking a FAQ but what is the current IDE of choice for Pig?
>
> I used to use a text editor and command line.
> I understand that PigPen (eclipse plugin) is no longer supported and
> does not work with current Hadoop. (But I haven't seen the
> documentation or wiki updated.)
> I have heard of PigEditor and will try it out.
>
> We normally use IntelliJ so would love to hear anything that helps
> integrate that.
>
> I am also a bit confused as to testing Pig. I am sitting in front of a
> Windows box and would love to be able to debug my scripts locally, but
> I don't see that is possible.
>
> If we have a bunch of developers I wonder if they should be all
> testing on the same cluster, or have individual clusters, and how they
> should be spun up and down....
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
Re: IDE choice?
Posted by Siddharth Tiwari <si...@tcs.com>.
Hi Alex,
You can always use cygwin for development purposes only. It will work like charm, with no fuss.
Regards
Siddharth
-----Alex McLintock <al...@gmail.com> wrote: -----
To: user@pig.apache.org
From: Alex McLintock <al...@gmail.com>
Date: 09/25/2012 01:54PM
Subject: IDE choice?
Forgive me for asking a FAQ but what is the current IDE of choice for Pig?
I used to use a text editor and command line.
I understand that PigPen (eclipse plugin) is no longer supported and
does not work with current Hadoop. (But I haven't seen the
documentation or wiki updated.)
I have heard of PigEditor and will try it out.
We normally use IntelliJ so would love to hear anything that helps
integrate that.
I am also a bit confused as to testing Pig. I am sitting in front of a
Windows box and would love to be able to debug my scripts locally, but
I don't see that is possible.
If we have a bunch of developers I wonder if they should be all
testing on the same cluster, or have individual clusters, and how they
should be spun up and down....
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
RE: IDE choice?
Posted by Anurag Gulati <An...@aexp.com>.
I use the cloudera virtual machine (on my windows and mac systems) for all my testing and script writing before we deploy on the cluster.
You can download it free from their webpage. I like that their distro will auto update all hadoop ecosystem programs (Hive, Pig, Mahout, etc).
Regards,
Anurag Gulati
-----Original Message-----
From: pablomar [mailto:pablo.daniel.martinez@gmail.com]
Sent: Tuesday, September 25, 2012 8:33 AM
To: user@pig.apache.org
Subject: Re: IDE choice?
I have virtualbox running on my windows laptop with a linux VM and a single-node installation of hadoop + pig for my needs, it is good enough to do pig scripts and UDFs and test them
On Tue, Sep 25, 2012 at 4:23 AM, Alex McLintock <al...@gmail.com>wrote:
> Forgive me for asking a FAQ but what is the current IDE of choice for Pig?
>
> I used to use a text editor and command line.
> I understand that PigPen (eclipse plugin) is no longer supported and
> does not work with current Hadoop. (But I haven't seen the
> documentation or wiki updated.) I have heard of PigEditor and will try
> it out.
>
> We normally use IntelliJ so would love to hear anything that helps
> integrate that.
>
> I am also a bit confused as to testing Pig. I am sitting in front of a
> Windows box and would love to be able to debug my scripts locally, but
> I don't see that is possible.
>
> If we have a bunch of developers I wonder if they should be all
> testing on the same cluster, or have individual clusters, and how they
> should be spun up and down....
>
American Express made the following annotations on Tue Sep 25 2012 08:44:37
******************************************************************************
"This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you."
American Express a ajout� le commentaire suivant le Tue Sep 25 2012 08:44:37
Ce courrier et toute pi�ce jointe qu'il contient sont r�serv�s au seul destinataire indiqu� et peuvent renfermer des renseignements confidentiels et privil�gi�s. Si vous n'�tes pas le destinataire pr�vu, toute divulgation, duplication, utilisation ou distribution du courrier ou de toute pi�ce jointe est interdite. Si vous avez re�u cette communication par erreur, veuillez nous en aviser par courrier et d�truire imm�diatement le courrier et les pi�ces jointes. Merci.
******************************************************************************
-------------------------------------------------------------------------------
Re: IDE choice?
Posted by pablomar <pa...@gmail.com>.
I have virtualbox running on my windows laptop with a linux VM and a
single-node installation of hadoop + pig
for my needs, it is good enough to do pig scripts and UDFs and test them
On Tue, Sep 25, 2012 at 4:23 AM, Alex McLintock <al...@gmail.com>wrote:
> Forgive me for asking a FAQ but what is the current IDE of choice for Pig?
>
> I used to use a text editor and command line.
> I understand that PigPen (eclipse plugin) is no longer supported and
> does not work with current Hadoop. (But I haven't seen the
> documentation or wiki updated.)
> I have heard of PigEditor and will try it out.
>
> We normally use IntelliJ so would love to hear anything that helps
> integrate that.
>
> I am also a bit confused as to testing Pig. I am sitting in front of a
> Windows box and would love to be able to debug my scripts locally, but
> I don't see that is possible.
>
> If we have a bunch of developers I wonder if they should be all
> testing on the same cluster, or have individual clusters, and how they
> should be spun up and down....
>