You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Richard Ding (JIRA)" <ji...@apache.org> on 2010/01/27 18:22:34 UTC
[jira] Updated: (PIG-1204) Pig hangs when joining two streaming
relations in local mode
[ https://issues.apache.org/jira/browse/PIG-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1204:
------------------------------
Description:
The following script hangs running in local mode when inpuf files contains many lines (e.g. 10K). The same script works when runing in MR mode.
{code}
A = load 'input1' as (a0, a1, a2);
B = stream A through `head -1` as (a0, a1, a2);
C = load 'input2' as (a0, a1, a2);
D = stream C through `head -1` as (a0, a1, a2);
E = join B by a0, D by a0;
dump E
{code}
Here is one stack trace:
"Thread-13" prio=10 tid=0x09938400 nid=0x1232 in Object.wait() [0x8fffe000..0x8ffff030]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
at java.lang.Object.wait(Object.java:485)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:291)
- locked <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNext(POStream.java:214)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:272)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POUnion.getNext(POUnion.java:162)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
was:
The following script hangs running in local mode when inpuf files contains many lines (e.g. 10K). The same script works when runing in MR mode.
{code}
A = load 'input1' as (a0, a1, a2);
B = stream A through `head -1` as (a0, a1, a2);
C = load 'input2' as (a0, a1, a2);
D = stream C through `head -1` as (a0, a1, a2);
E = join B by a0, D by a0;
dump E
{code}
Here is one stack trace:
"Thread-13" prio=10 tid=0x09938400 nid=0x1232 in Object.wait() [0x8fffe000..0x8ffff030]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
at java.lang.Object.wait(Object.java:485)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:291)
- locked <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNext(POStream.java:214)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:272)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POUnion.getNext(POUnion.java:162)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
Summary: Pig hangs when joining two streaming relations in local mode (was: Join two streaming relations hang in local mode)
> Pig hangs when joining two streaming relations in local mode
> ------------------------------------------------------------
>
> Key: PIG-1204
> URL: https://issues.apache.org/jira/browse/PIG-1204
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1204.patch
>
>
> The following script hangs running in local mode when inpuf files contains many lines (e.g. 10K). The same script works when runing in MR mode.
> {code}
> A = load 'input1' as (a0, a1, a2);
> B = stream A through `head -1` as (a0, a1, a2);
> C = load 'input2' as (a0, a1, a2);
> D = stream C through `head -1` as (a0, a1, a2);
> E = join B by a0, D by a0;
> dump E
> {code}
> Here is one stack trace:
> "Thread-13" prio=10 tid=0x09938400 nid=0x1232 in Object.wait() [0x8fffe000..0x8ffff030]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNextHelper(POStream.java:291)
> - locked <0x9b8e0a40> (a org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream.getNext(POStream.java:214)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:272)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POUnion.getNext(POUnion.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.