You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andy Tolbert (JIRA)" <ji...@apache.org> on 2016/09/01 23:16:21 UTC
[jira] [Comment Edited] (CASSANDRA-12594) sstabledump fails on
frozen collection cells
[ https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456906#comment-15456906 ]
Andy Tolbert edited comment on CASSANDRA-12594 at 9/1/16 11:15 PM:
-------------------------------------------------------------------
I have a fix for this which may or may not be a good solution. As the individual {{CollectionType}} implementation's {{getString()}} method just returns a raw hexString, we could instead use {{AbstractType#toJSONString}} if the given cell type {{isFrozenCollection}} evaluates to true.
This provides the following output for the example data above:
unfrozen case (existing that works):
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:34.950608Z" },
"cells" : [
{ "name" : "m", "deletion_info" : { "marked_deleted" : "2016-09-01T22:06:34.950607Z", "local_delete_time" : "2016-09-01T22:06:34Z" } },
{ "name" : "m", "path" : [ "a" ], "value" : "b:c" },
{ "name" : "m", "path" : [ "d" ], "value" : "e:f" },
{ "name" : "m", "path" : [ "g" ], "value" : "h:i" }
]
}
]
}
]
{code}
frozen case:
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" : {"a": ["b", "c"], "d": ["e", "f"], "g": ["h", "i"]} }
]
}
]
}
]
{code}
Attaching patch shortly.
was (Author: andrew.tolbert):
I have a fix for this which may or may not be a good solution. As the individual {{CollecitonType}} implementation's {{getString()}} method just returns a raw hexString, we could instead use {{AbstractType#toJSONString}} if the given cell type {{isFrozenCollection}} evaluates to true.
This provides the following output for the example data above:
unfrozen case (existing that works):
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:34.950608Z" },
"cells" : [
{ "name" : "m", "deletion_info" : { "marked_deleted" : "2016-09-01T22:06:34.950607Z", "local_delete_time" : "2016-09-01T22:06:34Z" } },
{ "name" : "m", "path" : [ "a" ], "value" : "b:c" },
{ "name" : "m", "path" : [ "d" ], "value" : "e:f" },
{ "name" : "m", "path" : [ "g" ], "value" : "h:i" }
]
}
]
}
]
{code}
frozen case:
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" : {"a": ["b", "c"], "d": ["e", "f"], "g": ["h", "i"]} }
]
}
]
}
]
{code}
Attaching patch shortly.
> sstabledump fails on frozen collection cells
> --------------------------------------------
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
> Reporter: Andy Tolbert
> Assignee: Andy Tolbert
> Priority: Minor
>
> sstabledump throws an exception when attempting to parse a cell that is a frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen collection, and attempts to get the string representation using the value type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map<text, frozen<tuple<text, text>>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen<map<text, frozen<tuple<text, text>>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'), 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'), 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)