Timed out waiting for rpc response Error

[George Pang <p0...@gmail.com>]

Dear users,

Today when I run pig shell, one error message appears:

"
ERROR 2999: Unexpected internal error. Failed to create DataStorage
...........

Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
response
.......


The shell ran ok yesterday, maybe something wrong I did between now and
then.  Do you have an idea?  Thanks a lot.

George

Pig with unordered attributes

["Naber, Chad" <CN...@edmunds.com>]

Hi folks,

Do any of you guys have any advice on parsing unordered attributes out of a file using Pig?  

I have a weblog that I am trying to get a distinct count of one attribute, and the only way I can think of doing it is by creating 100 variables, and UNION each variable set.  That seems like overkill for me. 

Is there any way to have PigStorage store into variable-length array?  Am I thinking about this incorrectly?

Here is the code I am playing around with, but I'm going to have to create variables up to around f100. :(

A = LOAD '/user/cnaber/input/edw.log' USING PigStorage('&') as (f1:chararray, f2:chararray, f3:chararray);
sess1 = FILTER f1 BY key matches '.*edwck.*';
sess2 = FILTER f2 BY key matches '.*edwck.*';
sess3 = FILTER f3 BY key matches '.*edwck.*'; sess = UNION sess1,sess2,sess3; sessdistinct = DISTINCT sess; grouper = GROUP sessdistinct ALL; sesscount = FOREACH grouper GENERATE COUNT(sessdistinct); STORE sesscount INTO 'results';


Here is what I really want to do:

A = LOAD '/user/cnaber/input/edw.log' USING PigStorage('&') as (f1:array); B = foreach A generate flatten(TOKENIZE(f1)) AS key; sess = FILTER B BY key matches '.*edwck.*'; sessdistinct = DISTINCT sess; grouper = GROUP sessdistinct ALL; sesscount = FOREACH grouper GENERATE COUNT(sessdistinct); STORE sesscount INTO 'test';

Thanks for the help in advance,
Chad

Re: Timed out waiting for rpc response Error

[George Pang <p0...@gmail.com>]

It works now. I don't know why. What I did before the error, is to set the
PATH and JAVA_HOME and other environment variables in /etc/profile. There
seems to be no .bash_profile in Ubuntu, so that's how I set the PATH.

After doing that, problem appeared as in last post. I cleared the
environment variables in /etc/profile, then pig -x mapreduce works again.
George

2009/5/21 George Pang <p0...@gmail.com>

> Dear users,
>
> Today when I run pig shell, one error message appears:
>
> "
> ERROR 2999: Unexpected internal error. Failed to create DataStorage
> ...........
>
> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
> response
> .......
>
>
> The shell ran ok yesterday, maybe something wrong I did between now and
> then.  Do you have an idea?  Thanks a lot.
>
> George
>