You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Yonggang Qiao <yo...@gmail.com> on 2010/05/17 22:57:49 UTC
Exception: Unable to find clone for op Project 4-16 Projections
Hi,
anyone has seen this error before? normally our script runs fine, but
sometime recently it began to throw this exception. also usually it
will go away if I rerun it.
Caused by: java.lang.RuntimeException: Unable to find clone for op
Project 4-16 Projections: [9] Overloaded: false
at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
at ...
Thanks,
Yonggang
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Yonggang Qiao <yo...@gmail.com>.
sure. removed the irrelevant stuff.
PigServer pigServer = new PigServer("mapreduce");
pigServer.registerQuery(...);
...
ExecJob job = pigServer.store("RET", outputPath);
then periodically check the job.getStatus().
On Mon, May 17, 2010 at 2:43 PM, Ashutosh Chauhan
<as...@gmail.com> wrote:
> if you are creating new instance for each query.. you should be fine..
> which Pig version are you using ? can you paste the snippet of java
> code where you are creating new pig server instance and then using it
> for a new query ?
>
> Ashutosh
> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>> sorry, actually yes. no, we use new instance for each script.
>>
>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>> yes. yes.
>>>
>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>> <as...@gmail.com> wrote:
>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>> you trying to run multiple queries in different threads against same
>>>> Pig server instance?
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>> sometime recently it began to throw this exception. also usually it
>>>>> will go away if I rerun it.
>>>>>
>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>> at ...
>>>>>
>>>>> Thanks,
>>>>> Yonggang
>>>>>
>>>>
>>>
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Yonggang Qiao <yo...@gmail.com>.
will definitely give it a try. Thanks!
On Tue, May 18, 2010 at 5:04 PM, Jeff Zhang <zj...@gmail.com> wrote:
> And you can refer here http://issues.apache.org/jira/browse/PIG-240
> People has done some work for this issue although it is still not
> resolved completely.
>
>
>
> On Tue, May 18, 2010 at 12:34 PM, Ashutosh Chauhan
> <as...@gmail.com> wrote:
>>> we do have multiple threads,
>> This is what I suspected from beginning :)
>>
>> PigServer is not thread-safe, yet.
>>
>> Ashutosh
>>
>> On Tue, May 18, 2010 at 12:04, Yonggang Qiao <yo...@gmail.com> wrote:
>>> seems it is a known issue. we do have multiple threads, each thread
>>> will take one script and new a PigServer to run it...
>>>
>>> Thanks,
>>> Yonggang
>>>
>>> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
>>> <as...@gmail.com> wrote:
>>>> From Yonggang description and code snippet .. it seems to me he is not
>>>> having multithreaded environment.. there is only one thread and he is
>>>> creating new PigServer instance in it repeatedly for each query... and
>>>> since static variables are reset everytime.. this should work...
>>>> PigServer still doesnt work in multi-threaded environment...
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <zj...@gmail.com> wrote:
>>>>> Ashutosh,
>>>>>
>>>>> Are you sure pig now can work on multi-thread environment ? As I know
>>>>> pig 0.5 can not work on multi-thread environment even you are create
>>>>> new PigServer for each pig script.
>>>>>
>>>>>
>>>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>>>> <as...@gmail.com> wrote:
>>>>>> if you are creating new instance for each query.. you should be fine..
>>>>>> which Pig version are you using ? can you paste the snippet of java
>>>>>> code where you are creating new pig server instance and then using it
>>>>>> for a new query ?
>>>>>>
>>>>>> Ashutosh
>>>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>>> yes. yes.
>>>>>>>>
>>>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>>>> <as...@gmail.com> wrote:
>>>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>>>> Pig server instance?
>>>>>>>>>
>>>>>>>>> Ashutosh
>>>>>>>>>
>>>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>>>> will go away if I rerun it.
>>>>>>>>>>
>>>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>>>> at ...
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Yonggang
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>>
>>>>> Jeff Zhang
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Jeff Zhang <zj...@gmail.com>.
And you can refer here http://issues.apache.org/jira/browse/PIG-240
People has done some work for this issue although it is still not
resolved completely.
On Tue, May 18, 2010 at 12:34 PM, Ashutosh Chauhan
<as...@gmail.com> wrote:
>> we do have multiple threads,
> This is what I suspected from beginning :)
>
> PigServer is not thread-safe, yet.
>
> Ashutosh
>
> On Tue, May 18, 2010 at 12:04, Yonggang Qiao <yo...@gmail.com> wrote:
>> seems it is a known issue. we do have multiple threads, each thread
>> will take one script and new a PigServer to run it...
>>
>> Thanks,
>> Yonggang
>>
>> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
>> <as...@gmail.com> wrote:
>>> From Yonggang description and code snippet .. it seems to me he is not
>>> having multithreaded environment.. there is only one thread and he is
>>> creating new PigServer instance in it repeatedly for each query... and
>>> since static variables are reset everytime.. this should work...
>>> PigServer still doesnt work in multi-threaded environment...
>>>
>>> Ashutosh
>>>
>>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <zj...@gmail.com> wrote:
>>>> Ashutosh,
>>>>
>>>> Are you sure pig now can work on multi-thread environment ? As I know
>>>> pig 0.5 can not work on multi-thread environment even you are create
>>>> new PigServer for each pig script.
>>>>
>>>>
>>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>>> <as...@gmail.com> wrote:
>>>>> if you are creating new instance for each query.. you should be fine..
>>>>> which Pig version are you using ? can you paste the snippet of java
>>>>> code where you are creating new pig server instance and then using it
>>>>> for a new query ?
>>>>>
>>>>> Ashutosh
>>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>>
>>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>> yes. yes.
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>>> <as...@gmail.com> wrote:
>>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>>> Pig server instance?
>>>>>>>>
>>>>>>>> Ashutosh
>>>>>>>>
>>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>>> will go away if I rerun it.
>>>>>>>>>
>>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>>> at ...
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Yonggang
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>
>
--
Best Regards
Jeff Zhang
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Ashutosh Chauhan <as...@gmail.com>.
> we do have multiple threads,
This is what I suspected from beginning :)
PigServer is not thread-safe, yet.
Ashutosh
On Tue, May 18, 2010 at 12:04, Yonggang Qiao <yo...@gmail.com> wrote:
> seems it is a known issue. we do have multiple threads, each thread
> will take one script and new a PigServer to run it...
>
> Thanks,
> Yonggang
>
> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
> <as...@gmail.com> wrote:
>> From Yonggang description and code snippet .. it seems to me he is not
>> having multithreaded environment.. there is only one thread and he is
>> creating new PigServer instance in it repeatedly for each query... and
>> since static variables are reset everytime.. this should work...
>> PigServer still doesnt work in multi-threaded environment...
>>
>> Ashutosh
>>
>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <zj...@gmail.com> wrote:
>>> Ashutosh,
>>>
>>> Are you sure pig now can work on multi-thread environment ? As I know
>>> pig 0.5 can not work on multi-thread environment even you are create
>>> new PigServer for each pig script.
>>>
>>>
>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>> <as...@gmail.com> wrote:
>>>> if you are creating new instance for each query.. you should be fine..
>>>> which Pig version are you using ? can you paste the snippet of java
>>>> code where you are creating new pig server instance and then using it
>>>> for a new query ?
>>>>
>>>> Ashutosh
>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>
>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>> yes. yes.
>>>>>>
>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>> <as...@gmail.com> wrote:
>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>> Pig server instance?
>>>>>>>
>>>>>>> Ashutosh
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>> will go away if I rerun it.
>>>>>>>>
>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>> at ...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yonggang
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Yonggang Qiao <yo...@gmail.com>.
seems it is a known issue. we do have multiple threads, each thread
will take one script and new a PigServer to run it...
Thanks,
Yonggang
On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
<as...@gmail.com> wrote:
> From Yonggang description and code snippet .. it seems to me he is not
> having multithreaded environment.. there is only one thread and he is
> creating new PigServer instance in it repeatedly for each query... and
> since static variables are reset everytime.. this should work...
> PigServer still doesnt work in multi-threaded environment...
>
> Ashutosh
>
> On Mon, May 17, 2010 at 18:00, Jeff Zhang <zj...@gmail.com> wrote:
>> Ashutosh,
>>
>> Are you sure pig now can work on multi-thread environment ? As I know
>> pig 0.5 can not work on multi-thread environment even you are create
>> new PigServer for each pig script.
>>
>>
>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>> <as...@gmail.com> wrote:
>>> if you are creating new instance for each query.. you should be fine..
>>> which Pig version are you using ? can you paste the snippet of java
>>> code where you are creating new pig server instance and then using it
>>> for a new query ?
>>>
>>> Ashutosh
>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>>>> sorry, actually yes. no, we use new instance for each script.
>>>>
>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>> yes. yes.
>>>>>
>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>> <as...@gmail.com> wrote:
>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>> you trying to run multiple queries in different threads against same
>>>>>> Pig server instance?
>>>>>>
>>>>>> Ashutosh
>>>>>>
>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>> will go away if I rerun it.
>>>>>>>
>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>> at ...
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Yonggang
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Ashutosh Chauhan <as...@gmail.com>.
>From Yonggang description and code snippet .. it seems to me he is not
having multithreaded environment.. there is only one thread and he is
creating new PigServer instance in it repeatedly for each query... and
since static variables are reset everytime.. this should work...
PigServer still doesnt work in multi-threaded environment...
Ashutosh
On Mon, May 17, 2010 at 18:00, Jeff Zhang <zj...@gmail.com> wrote:
> Ashutosh,
>
> Are you sure pig now can work on multi-thread environment ? As I know
> pig 0.5 can not work on multi-thread environment even you are create
> new PigServer for each pig script.
>
>
> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
> <as...@gmail.com> wrote:
>> if you are creating new instance for each query.. you should be fine..
>> which Pig version are you using ? can you paste the snippet of java
>> code where you are creating new pig server instance and then using it
>> for a new query ?
>>
>> Ashutosh
>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>>> sorry, actually yes. no, we use new instance for each script.
>>>
>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>>> yes. yes.
>>>>
>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>> <as...@gmail.com> wrote:
>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>> you trying to run multiple queries in different threads against same
>>>>> Pig server instance?
>>>>>
>>>>> Ashutosh
>>>>>
>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>> will go away if I rerun it.
>>>>>>
>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>> at ...
>>>>>>
>>>>>> Thanks,
>>>>>> Yonggang
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Jeff Zhang <zj...@gmail.com>.
Ashutosh,
Are you sure pig now can work on multi-thread environment ? As I know
pig 0.5 can not work on multi-thread environment even you are create
new PigServer for each pig script.
On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
<as...@gmail.com> wrote:
> if you are creating new instance for each query.. you should be fine..
> which Pig version are you using ? can you paste the snippet of java
> code where you are creating new pig server instance and then using it
> for a new query ?
>
> Ashutosh
> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
>> sorry, actually yes. no, we use new instance for each script.
>>
>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>>> yes. yes.
>>>
>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>> <as...@gmail.com> wrote:
>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>> you trying to run multiple queries in different threads against same
>>>> Pig server instance?
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>> sometime recently it began to throw this exception. also usually it
>>>>> will go away if I rerun it.
>>>>>
>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>> at ...
>>>>>
>>>>> Thanks,
>>>>> Yonggang
>>>>>
>>>>
>>>
>>
>
--
Best Regards
Jeff Zhang
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Ashutosh Chauhan <as...@gmail.com>.
if you are creating new instance for each query.. you should be fine..
which Pig version are you using ? can you paste the snippet of java
code where you are creating new pig server instance and then using it
for a new query ?
Ashutosh
On Mon, May 17, 2010 at 14:17, Yonggang Qiao <yo...@gmail.com> wrote:
> sorry, actually yes. no, we use new instance for each script.
>
> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
>> yes. yes.
>>
>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>> <as...@gmail.com> wrote:
>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>> you trying to run multiple queries in different threads against same
>>> Pig server instance?
>>>
>>> Ashutosh
>>>
>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> anyone has seen this error before? normally our script runs fine, but
>>>> sometime recently it began to throw this exception. also usually it
>>>> will go away if I rerun it.
>>>>
>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>> Project 4-16 Projections: [9] Overloaded: false
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>> at ...
>>>>
>>>> Thanks,
>>>> Yonggang
>>>>
>>>
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Yonggang Qiao <yo...@gmail.com>.
sorry, actually yes. no, we use new instance for each script.
On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <yo...@gmail.com> wrote:
> yes. yes.
>
> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
> <as...@gmail.com> wrote:
>> Are you using PigServer java api to run your Pig queries ? If so, are
>> you trying to run multiple queries in different threads against same
>> Pig server instance?
>>
>> Ashutosh
>>
>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>>> Hi,
>>>
>>> anyone has seen this error before? normally our script runs fine, but
>>> sometime recently it began to throw this exception. also usually it
>>> will go away if I rerun it.
>>>
>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>> Project 4-16 Projections: [9] Overloaded: false
>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>> at ...
>>>
>>> Thanks,
>>> Yonggang
>>>
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Yonggang Qiao <yo...@gmail.com>.
yes. yes.
On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
<as...@gmail.com> wrote:
> Are you using PigServer java api to run your Pig queries ? If so, are
> you trying to run multiple queries in different threads against same
> Pig server instance?
>
> Ashutosh
>
> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
>> Hi,
>>
>> anyone has seen this error before? normally our script runs fine, but
>> sometime recently it began to throw this exception. also usually it
>> will go away if I rerun it.
>>
>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>> Project 4-16 Projections: [9] Overloaded: false
>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>> at ...
>>
>> Thanks,
>> Yonggang
>>
>
Re: Exception: Unable to find clone for op Project 4-16 Projections
Posted by Ashutosh Chauhan <as...@gmail.com>.
Are you using PigServer java api to run your Pig queries ? If so, are
you trying to run multiple queries in different threads against same
Pig server instance?
Ashutosh
On Mon, May 17, 2010 at 13:57, Yonggang Qiao <yo...@gmail.com> wrote:
> Hi,
>
> anyone has seen this error before? normally our script runs fine, but
> sometime recently it began to throw this exception. also usually it
> will go away if I rerun it.
>
> Caused by: java.lang.RuntimeException: Unable to find clone for op
> Project 4-16 Projections: [9] Overloaded: false
> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
> at ...
>
> Thanks,
> Yonggang
>