You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Janne Jalkanen (Created) (JIRA)" <ji...@apache.org> on 2012/02/26 21:52:50 UTC
[jira] [Created] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
CassandraStorage can't cast fields from a CF correctly
------------------------------------------------------
Key: CASSANDRA-3962
URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.0.8
Environment: OSX 10.6.latest, Pig 0.9.2.
Reporter: Janne Jalkanen
Attachments: test.cli
Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
{code}
java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Janne Jalkanen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217161#comment-13217161 ]
Janne Jalkanen commented on CASSANDRA-3962:
-------------------------------------------
Could the fact that these are row keys, not columns, have something to do with the issue? Looking at CassandraStorage.getNext(), there's a line
// set the key
tuple.append(new DataByteArray(ByteBufferUtil.getArray(key)));
So it looks to me like the key is *always* added as a DataByteArray, regardless of it's actual type? getSchema() does seem to read the value from CfDef correctly tho'.
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217271#comment-13217271 ]
Pavel Yaskevich commented on CASSANDRA-3962:
--------------------------------------------
+1
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, 0002-Compose-key-from-marshaller.txt, test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3962:
----------------------------------------
Attachment: 0001-Add-LoadCaster-to-CassandraStorage.txt
Patch to add a LoadCaster. It does get used and converts the byte[] to String, but the join still fails with the same error :(
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Janne Jalkanen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Janne Jalkanen updated CASSANDRA-3962:
--------------------------------------
Attachment: test.pig
The test pig script for both cases, you might want to comment out the "dump a" to let it continue.
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Attachments: test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3962:
----------------------------------------
Reviewer: xedin
Fix Version/s: 1.1.0
1.0.9
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, 0002-Compose-key-from-marshaller.txt, test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Janne Jalkanen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Janne Jalkanen updated CASSANDRA-3962:
--------------------------------------
Attachment: test.cli
Generate the test CFs.
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Attachments: test.cli
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Brandon Williams (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams reassigned CASSANDRA-3962:
-------------------------------------------
Assignee: Brandon Williams
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Attachments: test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Janne Jalkanen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216834#comment-13216834 ]
Janne Jalkanen commented on CASSANDRA-3962:
-------------------------------------------
Relevant IRC log from #cassandra
{code}
[22:26] driftx: hmm, I think this is the udfcontext signature reuse problem
[22:26] driftx: jeromatron: what's the workaround for that again?
[22:26] Ecyrd: Is there an open bug?
[22:28] driftx: #2869, but we fixed it. hmm.
[22:28] CassBotJr: https://issues.apache.org/jira/browse/CASSANDRA-2869 : CassandraStorage does not function properly when used multiple times in a single pig script due to UDFContext sharing issues
[22:34] driftx: case DataType.CHARARRAY:
[22:34] driftx: return new NullableText((String)o);
[22:34] driftx: so it thinks it has a chararray, but it still has a bytearray
[22:42] driftx: I think we have to implement a LoadCaster to get around this
[22:43] Ecyrd: So I'm not insane, this is a real bug
{code}
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Attachments: test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216836#comment-13216836 ]
Brandon Williams commented on CASSANDRA-3962:
---------------------------------------------
I think implementing LoadCaster will fix this, but it's strange to me that pig doesn't allow going to the other way, casting a chararray to a bytearray since that's the only thing guaranteed to work here, in case the Bytes CF has keys that won't map to UTF8.
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Attachments: test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3962) CassandraStorage can't cast
fields from a CF correctly
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3962:
----------------------------------------
Attachment: 0002-Compose-key-from-marshaller.txt
You're totally right. It wasn't a problem casting the key from Bytes, but trying to use the one from U8. Patch to compose the key from the marshaller.
> CassandraStorage can't cast fields from a CF correctly
> ------------------------------------------------------
>
> Key: CASSANDRA-3962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3962
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.0.8
> Environment: OSX 10.6.latest, Pig 0.9.2.
> Reporter: Janne Jalkanen
> Assignee: Brandon Williams
> Labels: hadoop, pig
> Attachments: 0001-Add-LoadCaster-to-CassandraStorage.txt, 0002-Compose-key-from-marshaller.txt, test.cli, test.pig
>
>
> Included scripts demonstrate the problem. Regardless of whether the key is cast as a chararray or not, the Pig scripts fail with
> {code}
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.lang.String
> at org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:72)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira