You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jeremy Hanna (JIRA)" <ji...@apache.org> on 2013/08/09 16:04:47 UTC
[jira] [Created] (CASSANDRA-5867) The Pig
CqlStorage/AbstractCassandraStorage classes don't handle collection types
Jeremy Hanna created CASSANDRA-5867:
---------------------------------------
Summary: The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types
Key: CASSANDRA-5867
URL: https://issues.apache.org/jira/browse/CASSANDRA-5867
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Jeremy Hanna
The CqlStorage class gets the Pig data type for values from the AbstractCassandraStorage class, in the getPigType method. If it isn't a known data type, it makes the value into a ByteArray. Currently there aren't any cases there for lists, maps, and sets.
https://github.com/apache/cassandra/blob/cassandra-1.2.8/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java#L336
See this describe output from the grunt shell:
{code}
grunt> describe listdata ;
listdata: {id: (name: chararray,value: int),alist: (name: chararray,value: bytearray),amap: (name: chararray,value: bytearray),aset: (name: chararray,value: bytearray)}
{code}
where the cql data structures had this schema:
{code}
CREATE TABLE alltypes (
id int PRIMARY KEY,
alist list<text>,
amap map<text, text>,
aset set<text>
{code}
It turns out that if you cast the map in grunt to a pig map, then it sort of works, but I don't think we should probably use a pig map. Lists don't appear to work at all, as there is no Pig analogue. I *think* you could probably just do a UDF to cast these things, but we already have all of the type information, so we just need to change them to tuples or bags or whatever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira