You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Renato Marroquín Mogrovejo <re...@gmail.com> on 2011/02/03 18:58:23 UTC

Pig 0.8 and PigMix

Hey all,

I wanted to know if the patch from
https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and how to
apply it is the same way as shown in the JIRA.
Thanks.


Renato M.

Re: Pig 0.8 and PigMix

Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.

Thanks Daniel!
Now I am having trouble generating the data for the queries. I am getting a
NoClassDefFoundError: org/apache/hadoop/util/Tool for the main method of the
DataGenerator.class
This is how I have been trying to run it:

java -cp
/home/workspace/Pig0.8/pig.jar:/home/workspace/Pig0.8/lib/sdsuLibJKD12.jar:/home/workspace/Pig0.8/pigperf.jar
org.apache.pig.test.utils.datagen.DataGenerator -r 100
s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20

I took the column specification from the PigMix wiki site. It is the user
table.
Any kind of suggestion/advice is highly appreciated. Thanks in advance.

Renato M.

2011/2/3 Daniel Dai <ji...@yahoo-inc.com>

> Yes, we do use it for 0.8.
>
> Daniel
>
>
> Renato Marroquín Mogrovejo wrote:
>
>> Hey all,
>>
>> I wanted to know if the patch from
>> https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and how
>> to
>> apply it is the same way as shown in the JIRA.
>> Thanks.
>>
>>
>> Renato M.
>>
>>
>
>

Re: Pig 0.8 and PigMix

Posted by Daniel Dai <ji...@yahoo-inc.com>.

Hi, Renato,
I always use generate_data.sh to drive. Modify the specs in 
generate_data.sh, and run it. It will first generate page table, and 
then generate other dependent tables.

Daniel

Renato Marroquín Mogrovejo wrote:
> I managed to run it locally putting all my hadoop libraries into the
> classpath too.
> But I have another question, do I have to create one output file for each
> column? or how do I specify several columns?
> Thanks again.
>
> Renato M.
>
>
> 2011/2/4 Renato Marroquín Mogrovejo <re...@gmail.com>
>
>   
>> Thanks Daniel!
>> I have been trying to generate data for the user table but I haven't been
>> able to.
>> I tried locally:
>>
>> java -cp
>> /home/workspace/Pig0.8/pig.jar:/home/workspace/Pig0.8/lib/sdsuLibJKD12.jar:/home/workspace/Pig0.8/pigperf.jar
>> org.apache.pig.test.utils.datagen.DataGenerator -r 100
>> s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20
>>
>> And I get an exception of the following type
>> java.lang.NoClassDefFoundError: org/apache/hadoop/util/Tool
>> I tried in pseudo-distributed way; having $CONFIG_FILE pointing to my
>> core-site.xml:
>>
>> bin/hadoop jar $DATAGENJAR org.apache.pig.test.utils.datagen.DataGenerator
>> -libjars $ZIPFJAR -conf $CONFIG_FILE -r 100
>> s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20
>>
>> But I am getting an Exception in thread "main"
>> java.lang.NoClassDefFoundError: org/apache/pig/tools/cmdline/CmdLineParser
>> Any suggestion or advice is highly appreciated!
>> Thanks in advance.
>>
>> Renato M.
>>
>> 2011/2/3 Daniel Dai <ji...@yahoo-inc.com>
>>
>> Yes, we do use it for 0.8.
>>     
>>> Daniel
>>>
>>> Renato Marroquín Mogrovejo wrote:
>>>
>>>       
>>>> Hey all,
>>>>
>>>> I wanted to know if the patch from
>>>> https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and
>>>> how to
>>>> apply it is the same way as shown in the JIRA.
>>>> Thanks.
>>>>
>>>>
>>>> Renato M.
>>>>
>>>>
>>>>         
>>>

Re: Pig 0.8 and PigMix

Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.

I managed to run it locally putting all my hadoop libraries into the
classpath too.
But I have another question, do I have to create one output file for each
column? or how do I specify several columns?
Thanks again.

Renato M.


2011/2/4 Renato Marroquín Mogrovejo <re...@gmail.com>

> Thanks Daniel!
> I have been trying to generate data for the user table but I haven't been
> able to.
> I tried locally:
>
> java -cp
> /home/workspace/Pig0.8/pig.jar:/home/workspace/Pig0.8/lib/sdsuLibJKD12.jar:/home/workspace/Pig0.8/pigperf.jar
> org.apache.pig.test.utils.datagen.DataGenerator -r 100
> s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20
>
> And I get an exception of the following type
> java.lang.NoClassDefFoundError: org/apache/hadoop/util/Tool
> I tried in pseudo-distributed way; having $CONFIG_FILE pointing to my
> core-site.xml:
>
> bin/hadoop jar $DATAGENJAR org.apache.pig.test.utils.datagen.DataGenerator
> -libjars $ZIPFJAR -conf $CONFIG_FILE -r 100
> s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20
>
> But I am getting an Exception in thread "main"
> java.lang.NoClassDefFoundError: org/apache/pig/tools/cmdline/CmdLineParser
> Any suggestion or advice is highly appreciated!
> Thanks in advance.
>
> Renato M.
>
> 2011/2/3 Daniel Dai <ji...@yahoo-inc.com>
>
> Yes, we do use it for 0.8.
>>
>> Daniel
>>
>> Renato Marroquín Mogrovejo wrote:
>>
>>> Hey all,
>>>
>>> I wanted to know if the patch from
>>> https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and
>>> how to
>>> apply it is the same way as shown in the JIRA.
>>> Thanks.
>>>
>>>
>>> Renato M.
>>>
>>>
>>
>>
>

Re: Pig 0.8 and PigMix

Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.

Thanks Daniel!
I have been trying to generate data for the user table but I haven't been
able to.
I tried locally:

java -cp
/home/workspace/Pig0.8/pig.jar:/home/workspace/Pig0.8/lib/sdsuLibJKD12.jar:/home/workspace/Pig0.8/pigperf.jar
org.apache.pig.test.utils.datagen.DataGenerator -r 100
s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20

And I get an exception of the following type java.lang.NoClassDefFoundError:
org/apache/hadoop/util/Tool
I tried in pseudo-distributed way; having $CONFIG_FILE pointing to my
core-site.xml:

bin/hadoop jar $DATAGENJAR org.apache.pig.test.utils.datagen.DataGenerator
-libjars $ZIPFJAR -conf $CONFIG_FILE -r 100
s:20:16000000:z:7,i:1:2:u:0,i:1:20:z:0,s:10:18000000:z:20,l:1:1000000:z:0,l:1:86400:u:0,d:1:100000:z:5,m:15:1:z:0,bx:50:1:z:20

But I am getting an Exception in thread "main"
java.lang.NoClassDefFoundError: org/apache/pig/tools/cmdline/CmdLineParser
Any suggestion or advice is highly appreciated!
Thanks in advance.

Renato M.

2011/2/3 Daniel Dai <ji...@yahoo-inc.com>

> Yes, we do use it for 0.8.
>
> Daniel
>
> Renato Marroquín Mogrovejo wrote:
>
>> Hey all,
>>
>> I wanted to know if the patch from
>> https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and how
>> to
>> apply it is the same way as shown in the JIRA.
>> Thanks.
>>
>>
>> Renato M.
>>
>>
>
>

Re: Pig 0.8 and PigMix

Posted by Daniel Dai <ji...@yahoo-inc.com>.

Yes, we do use it for 0.8.

Daniel

Renato Marroquín Mogrovejo wrote:
> Hey all,
>
> I wanted to know if the patch from
> https://issues.apache.org/jira/browse/PIG-200 is safe for Pig0.8, and how to
> apply it is the same way as shown in the JIRA.
> Thanks.
>
>
> Renato M.
>