You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Arpan Khandelwal (JIRA)" <ji...@apache.org> on 2017/06/23 14:42:00 UTC
[jira] [Created] (CASSANDRA-13633) Data not loading to keyspace
using sstable create via CQLSSTableWriter
Arpan Khandelwal created CASSANDRA-13633:
--------------------------------------------
Summary: Data not loading to keyspace using sstable create via CQLSSTableWriter
Key: CASSANDRA-13633
URL: https://issues.apache.org/jira/browse/CASSANDRA-13633
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Linux
Reporter: Arpan Khandelwal
Fix For: 3.10
Attachments: dataloading_result.png
Scenario : Need to read CSV, Write SSTable using CQLSSTableWriter and load it to the keyspace. [Explained here|https://stackoverflow.com/questions/44713777/json-cassandra-field-value-parsing-using-antlr4] . That was not working so tried a simple test case available here[https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java#L378]
Which did not worked either. Following is what i tried.
Created cql_keyspace3 with below table and types.
{code:java}
CREATE TYPE cql_keyspace3.tuple2 (a int, b int);
CREATE TYPE cql_keyspace3.tuple3 (a int, b int, c int)
CREATE TABLE cql_keyspace3.table3 ( k int, v1 list<frozen<tuple2>>, v2 frozen<tuple3>, PRIMARY KEY (k));
{code}
ran code
{code:java}
final String KS = "cql_keyspace3";
final String TABLE = "table3";
final String schema = "CREATE TABLE " + KS + "." + TABLE + " (" + " k int," + " v1 list<frozen<tuple2>>,"
+ " v2 frozen<tuple3>," + " PRIMARY KEY (k)" + ")";
File tempdir = Files.createTempDir();
File dataDir = new File(tempdir.getAbsolutePath() + File.separator + KS + File.separator + TABLE);
System.out.println(dataDir);
assert dataDir.mkdirs();
CQLSSTableWriter writer = CQLSSTableWriter.builder().inDirectory(dataDir)
.withType("CREATE TYPE " + KS + ".tuple2 (a int, b int)")
.withType("CREATE TYPE " + KS + ".tuple3 (a int, b int, c int)").forTable(schema)
.using("INSERT INTO " + KS + "." + TABLE + " (k, v1, v2) " + "VALUES (?, ?, ?)").build();
UserType tuple2Type = writer.getUDType("tuple2");
UserType tuple3Type = writer.getUDType("tuple3");
for (int i = 0; i < 100; i++) {
writer.addRow(i,
ImmutableList.builder().add(tuple2Type.newValue().setInt("a", i * 10).setInt("b", i * 20))
.add(tuple2Type.newValue().setInt("a", i * 30).setInt("b", i * 40)).build(),
tuple3Type.newValue().setInt("a", i * 100).setInt("b", i * 200).setInt("c", i * 300));
}
writer.close();
{code}
It generated sstable in "/tmp/1498224996687-0/cql_keyspace3" dir
Loaded data using following command
{code:java}
/tmp/1498224996687-0/cql_keyspace3 $ sstableloader -d localhost table3-e6e0fa61581911e78be6a72ebce4c745/
Established connection to initial hosts
Opening sstables and calculating sections to stream
Streaming relevant part of /tmp/1498224996687-0/cql_keyspace3/table3-e6e0fa61581911e78be6a72ebce4c745/mc-2-big-Data.db to [localhost/127.0.0.1]
progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 1.060KiB/s (avg: 1.060KiB/s)
progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 0.000KiB/s (avg: 0.984KiB/s)
Summary statistics:
Connections per host : 1
Total files transferred : 1
Total bytes transferred : 5.572KiB
Total duration : 5668 ms
Average transfer rate : 0.982KiB/s
Peak transfer rate : 1.060KiB/s
{code}
||k||v1||v2||
|92|[{a:920,b:1840}, {a:2760,b:3680}]|
find full result in attached snapshot.
Please let me know which version of cassandra I should which can allow me to load reseverd types, collections, UDT from sstable create using CQLSSTableWriter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org