You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2013/10/09 15:22:43 UTC
[jira] [Resolved] (CASSANDRA-6171) Bulk loading to another keyspace
with sstableloader throws EOFException
[ https://issues.apache.org/jira/browse/CASSANDRA-6171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-6171.
---------------------------------------
Resolution: Not A Problem
The design is not very friendly, but it is working as designed.
This will be improved by CASSANDRA-5202 moving the KS name out of the filename.
> Bulk loading to another keyspace with sstableloader throws EOFException
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-6171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6171
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Linux Mint x64 (Olive)
> Cassandra 1.2.9
> [cqlsh 3.1.7 | Cassandra 1.2.9-1~precise~ppa1 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
> Reporter: palacsint
> Labels: sstableloader
>
> I'm trying to to load a snapshot of keyspace {{test}} to another keyspace, called {{test2}}.
> I've got the {{java.io.EOFException}}s below.
> *Steps to reproduce*:
> {noformat}
> # cqlsh
> cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
> cqlsh> CREATE TABLE test.test (key varchar PRIMARY KEY, value varchar);
> cqlsh> INSERT INTO test.test (key, value) VALUES ('ryan', 'ryan');
> # nodetool --host localhost snapshot
> Requested creating snapshot for: all keyspaces
> Snapshot directory: 1381309132532
> # mkdir test2
> # mkdir test2/test
> # cp -R /var/lib/cassandra/data/test/test/snapshots/1381309132532/* test2/test/
> # cqlsh
> cqlsh> CREATE KEYSPACE test2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
> cqlsh> CREATE TABLE test2.test (key varchar PRIMARY KEY, value varchar);
> cqlsh>
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Streaming revelant part of test2/test/test-test-ic-1-Data.db to [/127.0.0.1]
> progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1 failed
> ERROR 11:02:43,612 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.io.EOFException
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
> at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ... 3 more
> Exception in thread "Streaming to /127.0.0.1:1" java.lang.RuntimeException: java.io.EOFException
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
> at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ... 3 more
> progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]Streaming to the following hosts failed:
> [/127.0.0.1]
> #
> {noformat}
> The {{system.log}} file contains the following:
> {noformat}
> ERROR [Thread-4] 2013-10-09 11:02:43,608 CassandraDaemon.java (line 192) Exception in thread Thread[Thread-4,5,main]
> java.lang.AssertionError
> at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:168)
> at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:138)
> at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
> at org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
> at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
> {noformat}
> *A workaround is renaming the snapshot files too* (not just the directory):
> {noformat}
> # rename --verbose 's/test-test/test2-test/' test2/test/*
> test2/test/test-test-ic-1-CompressionInfo.db renamed as test2/test/test2-test-ic-1-CompressionInfo.db
> test2/test/test-test-ic-1-Data.db renamed as test2/test/test2-test-ic-1-Data.db
> test2/test/test-test-ic-1-Filter.db renamed as test2/test/test2-test-ic-1-Filter.db
> test2/test/test-test-ic-1-Index.db renamed as test2/test/test2-test-ic-1-Index.db
> test2/test/test-test-ic-1-Statistics.db renamed as test2/test/test2-test-ic-1-Statistics.db
> test2/test/test-test-ic-1-Summary.db renamed as test2/test/test2-test-ic-1-Summary.db
> test2/test/test-test-ic-1-TOC.txt renamed as test2/test/test2-test-ic-1-TOC.txt
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Streaming revelant part of test2/test/test2-test-ic-1-Data.db to [/127.0.0.1]
> progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1 completed (waiting on 0 outstanding sessions)
> progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]
> #
> {noformat}
> {{sstableloader --help}} mentions this:
> {noformat}
> sstableloader --help
> usage: sstableloader [options] <dir_path>
> --
> Bulk load the sstables found in the directory <dir_path> to the configured
> cluster.The parent directory of <dir_path> is used as the keyspace name.
> So for instance, to load an sstable named Standard1-g-1-Data.db into
> keyspace Keyspace1, you will need to have the files Standard1-g-1-Data.db
> and Standard1-g-1-Index.db in a directory Keyspace1/Standard1/ in the
> directory and call: sstableloader Keyspace1/Standard1
> {noformat}
> However, renaming the snapshot files to the suggested format (if I'm right here) still not help:
> {noformat}
> # rename --verbose 's/test2-test-/test-/' test2/test/*
> test2/test/test2-test-ic-1-CompressionInfo.db renamed as test2/test/test-ic-1-CompressionInfo.db
> test2/test/test2-test-ic-1-Data.db renamed as test2/test/test-ic-1-Data.db
> test2/test/test2-test-ic-1-Filter.db renamed as test2/test/test-ic-1-Filter.db
> test2/test/test2-test-ic-1-Index.db renamed as test2/test/test-ic-1-Index.db
> test2/test/test2-test-ic-1-Statistics.db renamed as test2/test/test-ic-1-Statistics.db
> test2/test/test2-test-ic-1-Summary.db renamed as test2/test/test-ic-1-Summary.db
> test2/test/test2-test-ic-1-TOC.txt renamed as test2/test/test-ic-1-TOC.txt
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Skipping file test-ic-1-Data.db: column family test2.ic doesn't exist
> No sstables to stream
> progress: [total: 100 - 0MB/s (avg: 0MB/s)][11:42:53] (0)
> {noformat}
> An upgraded {{sstableloader --help}} message and a helpful error message (instead of the {{EOFException}}) would be much better here.
--
This message was sent by Atlassian JIRA
(v6.1#6144)