Problem running Pig 0.2 programmatically in local mode

[Gregory Harman <gr...@harmantechnologies.com>]

Today, I grabbed the Pig 0.2 release and started working on porting my  
application to it. In a nutshell, I need to run Pig programmatically  
(I'm using PigServer as the entry point) and development versions run  
in local mode. I downloaded the Pig 0.2 release, built the project  
with the default ant task (to get the jar that includes dependencies),  
and then with that jar in my classpath did a new Pig("local");

That gives me this stack trace:

java.lang.RuntimeException: Failed to create DataStorage
	at  
org 
.apache 
.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
	at  
org 
.apache 
.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java: 
53)
	at org.apache.pig.impl.PigContext.connect(PigContext.java:163)
	at org.apache.pig.PigServer.<init>(PigServer.java:144)
	at org.apache.pig.PigServer.<init>(PigServer.java:128)
	at org.apache.pig.PigServer.<init>(PigServer.java:124)
	at org.apache.pig.PigServer.<init>(PigServer.java:120)
Caused by: java.io.IOException: No FileSystem for scheme: file
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java: 
1336)
	at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:56)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1351)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:213)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:118)
	at  
org 
.apache 
.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
	... 17 more

I've also gotten the same result by taking the pre-built pig.jar in  
the distribution and including that and /lib/hadoop18.jar in my  
classpath instead.

A little history to this: I had a similar problem with Pig 0.1 that I  
was able to fix by patching to PigContext to call LocalDataStorage  
instead of HDataStorage, but LocalDataStorage seems to have  
disappeared in v0.2 so that workaround isn't going to work this time.  
Details in my earlier thread: http://www.mail-archive.com/pig-user@hadoop.apache.org/msg00301.html

I'm assuming that others have successfully run v0.2 in local mode - am  
I missing something?

thanks,
Greg

Re: Problem running Pig 0.2 programmatically in local mode

[Gregory Harman <gr...@harmantechnologies.com>]

I think I got it working (and without my 0.1 patch) - it seems to have  
been a classloader issue specific to my environment and shouldn't  
affect other users.

Sorry for the noise!

On May 13, 2009, at 4:14 PM, Gregory Harman wrote:

> Sorry, typo: new PigServer("local");
>
> On May 13, 2009, at 4:04 PM, Gregory Harman wrote:
>
>> Today, I grabbed the Pig 0.2 release and started working on porting  
>> my application to it. In a nutshell, I need to run Pig  
>> programmatically (I'm using PigServer as the entry point) and  
>> development versions run in local mode. I downloaded the Pig 0.2  
>> release, built the project with the default ant task (to get the  
>> jar that includes dependencies), and then with that jar in my  
>> classpath did a new Pig("local");
>

Re: Problem running Pig 0.2 programmatically in local mode

[Gregory Harman <gr...@harmantechnologies.com>]

Sorry, typo: new PigServer("local");

On May 13, 2009, at 4:04 PM, Gregory Harman wrote:

> Today, I grabbed the Pig 0.2 release and started working on porting  
> my application to it. In a nutshell, I need to run Pig  
> programmatically (I'm using PigServer as the entry point) and  
> development versions run in local mode. I downloaded the Pig 0.2  
> release, built the project with the default ant task (to get the jar  
> that includes dependencies), and then with that jar in my classpath  
> did a new Pig("local");