You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Andreas Paepcke <pa...@cs.stanford.edu> on 2011/03/25 18:44:10 UTC

ORDER statement leads to 'Cannot convert a Unknown to a String'

Hi,

Has anyone seen the following?

I am getting an error when running ORDER:
   ERROR 1071: Cannot convert a Unknown to a String

The error occurs in DataType.java:885. At the end of that switch
statement variable 'type' is -1, and variable 'o' is a string that looks
like a leftover from the prior statements A or B. The value of 'o' is:

%!PS-Adobe-2.0
%%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
%%Title: arXiv:astro-ph/0005123 v3   2 Oct 2000
%%Pages: 7
%%PageOrder: As...

Note that if I skip the ORDER statement, everything works, and looks
correct in the resulting file. Random order, of course.

The error does not occur if I make one simple change:
Store D into a tmp file, then LOAD that file and execute E without
any change to that statement.

Pseudocode below, followed by the stack trace.

A    = LOAD 'foo' "
       USING aLoader()
       AS (url:chararray,
              date:chararray,
              pageSize:int,
              position:int,
              docidInCrawl:int,
              httpHeader:chararray,
              content:chararray);

B    = FOREACH A GENERATE
       udf();

-- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
... }

D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
tokenPos:int);
E = ORDER D BY token ASC;
STORE E INTO 'bar';


org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
convert a Unknown to a String
    at org.apache.pig.data.DataType.toString(DataType.java:885)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
    at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
    at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

Re: ORDER statement leads to 'Cannot convert a Unknown to a String'

Posted by Daniel Dai <ji...@yahoo-inc.com>.
When you say "Store D into a tmp file", which store func are you using?

On 03/25/2011 10:44 AM, Andreas Paepcke wrote:
> Hi,
>
> Has anyone seen the following?
>
> I am getting an error when running ORDER:
>     ERROR 1071: Cannot convert a Unknown to a String
>
> The error occurs in DataType.java:885. At the end of that switch
> statement variable 'type' is -1, and variable 'o' is a string that looks
> like a leftover from the prior statements A or B. The value of 'o' is:
>
> %!PS-Adobe-2.0
> %%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
> %%Title: arXiv:astro-ph/0005123 v3   2 Oct 2000
> %%Pages: 7
> %%PageOrder: As...
>
> Note that if I skip the ORDER statement, everything works, and looks
> correct in the resulting file. Random order, of course.
>
> The error does not occur if I make one simple change:
> Store D into a tmp file, then LOAD that file and execute E without
> any change to that statement.
>
> Pseudocode below, followed by the stack trace.
>
> A    = LOAD 'foo' "
>         USING aLoader()
>         AS (url:chararray,
>                date:chararray,
>                pageSize:int,
>                position:int,
>                docidInCrawl:int,
>                httpHeader:chararray,
>                content:chararray);
>
> B    = FOREACH A GENERATE
>         udf();
>
> -- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
> ... }
>
> D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
> tokenPos:int);
> E = ORDER D BY token ASC;
> STORE E INTO 'bar';
>
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a Unknown to a String
>      at org.apache.pig.data.DataType.toString(DataType.java:885)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>      at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)


Re: ORDER statement leads to 'Cannot convert a Unknown to a String'

Posted by Daniel Dai <ji...@yahoo-inc.com>.
Sounds like aLoad() feed a data type Pig cannot understand.

Daniel

On 03/25/2011 10:44 AM, Andreas Paepcke wrote:
> Hi,
>
> Has anyone seen the following?
>
> I am getting an error when running ORDER:
>     ERROR 1071: Cannot convert a Unknown to a String
>
> The error occurs in DataType.java:885. At the end of that switch
> statement variable 'type' is -1, and variable 'o' is a string that looks
> like a leftover from the prior statements A or B. The value of 'o' is:
>
> %!PS-Adobe-2.0
> %%Creator: dvips(k) 5.86 Copyright 1999 Radical Eye Software
> %%Title: arXiv:astro-ph/0005123 v3   2 Oct 2000
> %%Pages: 7
> %%PageOrder: As...
>
> Note that if I skip the ORDER statement, everything works, and looks
> correct in the resulting file. Random order, of course.
>
> The error does not occur if I make one simple change:
> Store D into a tmp file, then LOAD that file and execute E without
> any change to that statement.
>
> Pseudocode below, followed by the stack trace.
>
> A    = LOAD 'foo' "
>         USING aLoader()
>         AS (url:chararray,
>                date:chararray,
>                pageSize:int,
>                position:int,
>                docidInCrawl:int,
>                httpHeader:chararray,
>                content:chararray);
>
> B    = FOREACH A GENERATE
>         udf();
>
> -- B is of the form {(chararray,chararray,int), (chararray,chararray,int),
> ... }
>
> D = FOREACH B GENERATE flatten($0) AS (token:chararray, docID:chararray,
> tokenPos:int);
> E = ORDER D BY token ASC;
> STORE E INTO 'bar';
>
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a Unknown to a String
>      at org.apache.pig.data.DataType.toString(DataType.java:885)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:642)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:367)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
>      at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:240)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
>      at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>      at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)