You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Andreas Paepcke <pa...@cs.stanford.edu> on 2011/03/25 18:44:10 UTC
ORDER statement leads to 'Cannot convert a Unknown to a String'
Hi,
Has anyone seen the following?
I am getting an error when running ORDER:
ERROR 1071: Cannot convert a Unknown to a String
The error occurs in DataType.java:885. At the end of that switch
statement variable 'type' is -1, and variable 'o' is a string that looks
like a leftover from the prior statements A or B. The value of 'o' is:
%!PS-Adobe-2.0
%%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
%%Title: arXiv:astro-ph/0005123 v3 2 Oct 2000
%%Pages: 7
%%PageOrder: As...
Note that if I skip the ORDER statement, everything works, and looks
correct in the resulting file. Random order, of course.
The error does not occur if I make one simple change:
Store D into a tmp file, then LOAD that file and execute E without
any change to that statement.
Pseudocode below, followed by the stack trace.
A = LOAD 'foo' "
USING aLoader()
AS (url:chararray,
date:chararray,
pageSize:int,
position:int,
docidInCrawl:int,
httpHeader:chararray,
content:chararray);
B = FOREACH A GENERATE
udf();
-- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
... }
D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
tokenPos:int);
E = ORDER D BY token ASC;
STORE E INTO 'bar';
org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
convert a Unknown to a String
at org.apache.pig.data.DataType.toString(DataType.java:885)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Re: ORDER statement leads to 'Cannot convert a Unknown to a String'
Posted by Daniel Dai <ji...@yahoo-inc.com>.
When you say "Store D into a tmp file", which store func are you using?
On 03/25/2011 10:44 AM, Andreas Paepcke wrote:
> Hi,
>
> Has anyone seen the following?
>
> I am getting an error when running ORDER:
> ERROR 1071: Cannot convert a Unknown to a String
>
> The error occurs in DataType.java:885. At the end of that switch
> statement variable 'type' is -1, and variable 'o' is a string that looks
> like a leftover from the prior statements A or B. The value of 'o' is:
>
> %!PS-Adobe-2.0
> %%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
> %%Title: arXiv:astro-ph/0005123 v3 2 Oct 2000
> %%Pages: 7
> %%PageOrder: As...
>
> Note that if I skip the ORDER statement, everything works, and looks
> correct in the resulting file. Random order, of course.
>
> The error does not occur if I make one simple change:
> Store D into a tmp file, then LOAD that file and execute E without
> any change to that statement.
>
> Pseudocode below, followed by the stack trace.
>
> A = LOAD 'foo' "
> USING aLoader()
> AS (url:chararray,
> date:chararray,
> pageSize:int,
> position:int,
> docidInCrawl:int,
> httpHeader:chararray,
> content:chararray);
>
> B = FOREACH A GENERATE
> udf();
>
> -- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
> ... }
>
> D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
> tokenPos:int);
> E = ORDER D BY token ASC;
> STORE E INTO 'bar';
>
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a Unknown to a String
> at org.apache.pig.data.DataType.toString(DataType.java:885)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Re: ORDER statement leads to 'Cannot convert a Unknown to a String'
Posted by Daniel Dai <ji...@yahoo-inc.com>.
Sounds like aLoad() feed a data type Pig cannot understand.
Daniel
On 03/25/2011 10:44 AM, Andreas Paepcke wrote:
> Hi,
>
> Has anyone seen the following?
>
> I am getting an error when running ORDER:
> ERROR 1071: Cannot convert a Unknown to a String
>
> The error occurs in DataType.java:885. At the end of that switch
> statement variable 'type' is -1, and variable 'o' is a string that looks
> like a leftover from the prior statements A or B. The value of 'o' is:
>
> %!PS-Adobe-2.0
> %%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
> %%Title: arXiv:astro-ph/0005123 v3 2 Oct 2000
> %%Pages: 7
> %%PageOrder: As...
>
> Note that if I skip the ORDER statement, everything works, and looks
> correct in the resulting file. Random order, of course.
>
> The error does not occur if I make one simple change:
> Store D into a tmp file, then LOAD that file and execute E without
> any change to that statement.
>
> Pseudocode below, followed by the stack trace.
>
> A = LOAD 'foo' "
> USING aLoader()
> AS (url:chararray,
> date:chararray,
> pageSize:int,
> position:int,
> docidInCrawl:int,
> httpHeader:chararray,
> content:chararray);
>
> B = FOREACH A GENERATE
> udf();
>
> -- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
> ... }
>
> D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
> tokenPos:int);
> E = ORDER D BY token ASC;
> STORE E INTO 'bar';
>
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a Unknown to a String
> at org.apache.pig.data.DataType.toString(DataType.java:885)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)