You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Niels Basjes <Ni...@basjes.nl> on 2013/07/10 21:37:08 UTC
ERROR 1071: Cannot convert a generic_writablecomparable to a String
Hi,
I'm writing a custom Loader for Pig and I'm stuck on the following problem.
When I use this loader from a pig script and I do a "DESCRIBE Clicks;" I get
*Clicks: {useragent: chararray,status: chararray}*
So apparently the data from my loader is seen correctly.
How ever when I then do:
DUMP Clicks;
I get
*java.lang.Exception:
org.apache.pig.backend.executionengine.ExecException: ERROR
1071: Cannot convert a generic_writablecomparable to a String*
* at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)*
*Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
1071: Cannot convert a generic_writablecomparable to a String*
* at org.apache.pig.data.DataType.toString(DataType.java:1105)*
What am I doing wrong here?
My current Loader in summary looks like this:
*public class Loader extends LoadFunc {
*
* public Loader(String... parameters) { ... }
*
* public InputFormat<?, ?> getInputFormat() throws IOException { ... }*
* public Tuple getNext() throws IOException { *
* ...*
* List<Text> values = new ArrayList<Text>();*
* for (...) {*
* Text value = <get the value from somewhere>*
* values.add((Text)value);
*
* }*
* tuple = tupleFactory.newTuple(values);
*
* return tuple;
*
* }*
*
*
* public void prepareToRead(...) { ... }
*
* public void setLocation(String location, Job job) { ... }
*
*}*
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: ERROR 1071: Cannot convert a generic_writablecomparable to a String
Posted by Niels Basjes <Ni...@basjes.nl>.
Hi,
On Wed, Jul 10, 2013 at 10:01 PM, Pradeep Gollakota <pr...@gmail.com>wrote:
> Instead of doing "values.add((Text) value);" try doing
> "values.add(value.toString());" (And make sure that values is of type
> List<String> instead of List<Text>)
>
> I'm not too sure of the details but the problem seems to be that you're
> trying to cast a GenericWritableComparable (i.e. Text) into String
> (chararray).
>
Yes, that worked. Thanks.
Apparently I've been programming MR jobs in native Java for too long ;)
Niels Basjes
Re: ERROR 1071: Cannot convert a generic_writablecomparable to a String
Posted by Pradeep Gollakota <pr...@gmail.com>.
Instead of doing "values.add((Text) value);" try doing
"values.add(value.toString());" (And make sure that values is of type
List<String> instead of List<Text>)
I'm not too sure of the details but the problem seems to be that you're
trying to cast a GenericWritableComparable (i.e. Text) into String
(chararray).
On Wed, Jul 10, 2013 at 3:51 PM, Niels Basjes <Ni...@basjes.nl> wrote:
> Additional info:
>
> If I do
> Clicks =
> LOAD 'access_log.gz'
> USING <my loader> AS (useragent, status);
>
> I get
> Clicks: {useragent: bytearray,status: bytearray}
> Followed by (on screen) the correct data.
>
> I get the previously mentioned error when i do
> Clicks =
> LOAD 'access_log.gz'
> USING <my loader> AS (useragent:*chararray*, status:*chararray*);
>
> Niels
>
>
>
> On Wed, Jul 10, 2013 at 9:37 PM, Niels Basjes <Ni...@basjes.nl> wrote:
>
> > Hi,
> >
> > I'm writing a custom Loader for Pig and I'm stuck on the following
> problem.
> >
> > When I use this loader from a pig script and I do a "DESCRIBE Clicks;" I
> > get
> > *Clicks: {useragent: chararray,status: chararray}*
> >
> > So apparently the data from my loader is seen correctly.
> >
> > How ever when I then do:
> > DUMP Clicks;
> >
> > I get
> >
> > *java.lang.Exception:
> > org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> > convert a generic_writablecomparable to a String*
> > * at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)*
> > *Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> > 1071: Cannot convert a generic_writablecomparable to a String*
> > * at org.apache.pig.data.DataType.toString(DataType.java:1105)*
> >
> > What am I doing wrong here?
> >
> >
> > My current Loader in summary looks like this:
> >
> > *public class Loader extends LoadFunc {
> > *
> > * public Loader(String... parameters) { ... }
> > *
> > * public InputFormat<?, ?> getInputFormat() throws IOException { ... }*
> > * public Tuple getNext() throws IOException { *
> > * ...*
> > * List<Text> values = new ArrayList<Text>();*
> > * for (...) {*
> > * Text value = <get the value from somewhere>*
> > * values.add((Text)value);
> > *
> > * }*
> > * tuple = tupleFactory.newTuple(values);
> > *
> > * return tuple;
> > *
> > * }*
> > *
> > *
> > * public void prepareToRead(...) { ... }
> > *
> > * public void setLocation(String location, Job job) { ... }
> > *
> > *}*
> >
> >
> > --
> > Best regards / Met vriendelijke groeten,
> >
> > Niels Basjes
> >
>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
Re: ERROR 1071: Cannot convert a generic_writablecomparable to a String
Posted by Niels Basjes <Ni...@basjes.nl>.
Additional info:
If I do
Clicks =
LOAD 'access_log.gz'
USING <my loader> AS (useragent, status);
I get
Clicks: {useragent: bytearray,status: bytearray}
Followed by (on screen) the correct data.
I get the previously mentioned error when i do
Clicks =
LOAD 'access_log.gz'
USING <my loader> AS (useragent:*chararray*, status:*chararray*);
Niels
On Wed, Jul 10, 2013 at 9:37 PM, Niels Basjes <Ni...@basjes.nl> wrote:
> Hi,
>
> I'm writing a custom Loader for Pig and I'm stuck on the following problem.
>
> When I use this loader from a pig script and I do a "DESCRIBE Clicks;" I
> get
> *Clicks: {useragent: chararray,status: chararray}*
>
> So apparently the data from my loader is seen correctly.
>
> How ever when I then do:
> DUMP Clicks;
>
> I get
>
> *java.lang.Exception:
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a generic_writablecomparable to a String*
> * at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)*
> *Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> 1071: Cannot convert a generic_writablecomparable to a String*
> * at org.apache.pig.data.DataType.toString(DataType.java:1105)*
>
> What am I doing wrong here?
>
>
> My current Loader in summary looks like this:
>
> *public class Loader extends LoadFunc {
> *
> * public Loader(String... parameters) { ... }
> *
> * public InputFormat<?, ?> getInputFormat() throws IOException { ... }*
> * public Tuple getNext() throws IOException { *
> * ...*
> * List<Text> values = new ArrayList<Text>();*
> * for (...) {*
> * Text value = <get the value from somewhere>*
> * values.add((Text)value);
> *
> * }*
> * tuple = tupleFactory.newTuple(values);
> *
> * return tuple;
> *
> * }*
> *
> *
> * public void prepareToRead(...) { ... }
> *
> * public void setLocation(String location, Job job) { ... }
> *
> *}*
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes