You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Avram Aelony <Av...@eharmony.com> on 2009/03/02 19:39:48 UTC

RE: pigpen installation?

Hi Shubham,

Here is my pigpen feedback and continuing install notes so far... almost there (I hope)!

1. The PigPen wiki (http://wiki.apache.org/pig/PigPen/) might want probably to mention that setting configuration in Eclipse 3.4.1 involves Window -> Preferences -> Pig (rather than Eclipse preferences -> PIG).   Being new to Eclipse, I had to go through each menu item in search of a Preferences menu and then set the ConfigurationPath.  

ConfigurationPath.  The ConfigurationPath wiki link referred to on the PigPen page does not yet exist.  
The screenshot on how to set it is helpful but needs more detail.  It remains to be seen if what I did works correctly: I copied files "hadoop-site.xml" and "pig.properties" (that work correctly) from their respective, correct linux paths to a local windows directory I created (c:\hadoop_config\conf ) so Eclipse could find it when I specify the ConfigurationPath variable. Is there another (better) way to point to the config files the distant hadoop cluster uses?

2. Perspectives.  I believe Window -> Open Perspective -> Other is needed in the docs.   Pig did not show up for me as a perspective although other items were visible in a tree to choose from.  Once I did an explicit search for pig in the search-box, Eclipse found pig and the pig perspective.  As a result, I now see "Pig Pen" in the "Open Perspective" section of eclipse.  I believe this will likely enable the Operator graph and Example Generator to work, but I am still trying to get Pig Pen to talk to hadoop (see #3).

3. Connection issues.  I am running Eclipse from a windows PC which does not have a "whoami" command.  This is causing an error after the PigPen plugin starts talking to the hadoop cluster.  Is there any way around this?  Perhaps a setting in hadoop-site.xml or in pig.properties?

Here is what the PigPen Plugin Console reports:

Launching the job!
Using the configuration from C:\hadoop_config\conf
2009-03-02 09:29:00,236 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop.XX.XXXXX.com:9000
2009-03-02 09:29:00,268 [main] WARN  org.apache.hadoop.fs.FileSystem - uri=hdfs://hadoop.XX.XXXXX.com:9000 javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
	at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
	at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
	at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:257)
	at org.apache.hadoop.security.UserGroupInformation.login(UserGroupInformation.java:67)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1410)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1348)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:213)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:118)
	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:71)
	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:57)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:184)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:132)
	at org.apache.pig.impl.PigContext.connect(PigContext.java:172)
	at org.apache.pig.PigServer.<init>(PigServer.java:137)
	at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:43)
	at org.apache.pig.Main.main(Main.java:241)
java.lang.RuntimeException: Failed to create DataStorage
	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:74)
	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:57)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:184)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:132)
	at org.apache.pig.impl.PigContext.connect(PigContext.java:172)
	at org.apache.pig.PigServer.<init>(PigServer.java:137)
	at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:43)
	at org.apache.pig.Main.main(Main.java:241)
Caused by: java.io.IOException
	at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:171)
	at org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem.java:67)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1339)
	at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:56)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1351)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:213)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:118)
	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:71)
	... 7 more
Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
	at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
	at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
	at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:169)
	... 14 more


Thanks for any thoughts and help.

Avram



-----Original Message-----
From: Shubham Chopra [mailto:shubhamc@yahoo-inc.com] 
Sent: Friday, February 27, 2009 7:42 AM
To: pig-user@hadoop.apache.org; ielghandour@acm.org
Cc: Avram Aelony
Subject: Re: pigpen installation?

Hi Iman,

Thanks for mailing the procedure. That's the right way to install a 
plugin from a local directory.

Can you pls. give me more details (if possible the stack-trace) of the 
conflict you observed between PigPen and JavaCC. I do have both working 
in my eclipse installation without any issues.

Also, pls. feel free to mail me any feedback on the plugin.

Thanks,
Shubham.

Iman Elghandour wrote:
> Hi Avram,
> These are the steps that I have followed (this works only with eclipse Ganymede):
> 1- I first downloaded https://issues.apache.org/jira/secure/attachment/12400858/PigPen.tgz and untarred it, say to c:\PigPen
> 2- In Help>Software Updates and Add-ons, select the "Available Software" tab
> 3-select to "Add Site" and then "local", a browser will appear to choose the location of PigPen directory.
> 
> After many trials, I realized that it conflicts with other plug-ins. for example when i installed javacc, the icons of the pig disappeared,. So if the above did not work,  you might want to try to check other plugins that you have installed.
> 
> by the way, I did the above on a linux machine.
> Iman
> 
> --- On Thu, 2/26/09, Avram Aelony <Av...@eharmony.com> wrote:
> From: Avram Aelony <Av...@eharmony.com>
> Subject: RE: pigpen installation?
> To: "pig-user@hadoop.apache.org" <pi...@hadoop.apache.org>, "ielghandour@acm.org" <ie...@acm.org>
> Received: Thursday, February 26, 2009, 1:12 PM
> 
> Hi Iman,
> 
> I do see the PigPen jar file under Help -> Software Updates... ->
> Available Software. Clicking on the checkbox tells me "No repository found
> at file:/C:Program Files/eclipse/PigPen-0.0.4".
> 
> Does it show up for you under Window -> Preferences ?  It does not for me.  
> Can you describe what you did to get it working?
> 
> Avram
> 
> 
> -----Original Message-----
> From: Iman Elghandour [mailto:ielghand@yahoo.com] 
> Sent: Thursday, February 26, 2009 7:28 AM
> To: pig-user@hadoop.apache.org
> Subject: Re: pigpen installation?
> 
> Hi Avram,
> I followed the steps described by Shubham in this thread
> http://www.mail-archive.com/pig-user@hadoop.apache.org/msg00264.html and it
> worked for me.
> 
> Iman
> 
> --- On Wed, 2/25/09, Avram Aelony <Av...@eharmony.com> wrote:
> From: Avram Aelony <Av...@eharmony.com>
> Subject: pigpen installation?
> To: "pig-user@hadoop.apache.org" <pi...@hadoop.apache.org>
> Received: Wednesday, February 25, 2009, 7:03 PM
> 
> What is the best way to install the PigPen Eclipse plugin ?
> 
> I have found the following jira
> (https://issues.apache.org/jira/browse/PIG-366;jsessionid=8DA2C99251763B79998C0E813D071F92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ) but haven't determined how to "direct the eclipse software update to
> use that for installing PigPen" as stated in the jira.
> 
> Putting the org.apache.pig.pigpen_0.0.4.jar file into my eclipse/plugins
> directory doesn't do anything noticeable. Eclipse seems to ignore it
> despite
> using the -clean flag. I am using Eclipse 3.4.1.
> 
> Any suggestions?
> 
> Thank-you,
> Avram
> 
> 
> 
>       __________________________________________________________________
> Instant Messaging, free SMS, sharing photos and more... Try the new Yahoo!
> Canada Messenger at http://ca.beta.messenger.yahoo.com/
> 
> 
> 
>       __________________________________________________________________
> Ask a question on any topic and get answers from real people. Go to Yahoo! Answers and share what you know at http://ca.answers.yahoo.com