[jira] [Commented] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace

["Benyi Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259509#comment-14259509 ] 

Benyi Wang commented on CASSANDRA-7768:
---------------------------------------

I encounter this problem in cassandra-all.2.1.2 if you write two column families of the same keyspace in one JVM. The following code can reproduce the error. 

{code}
package org.apache.cassandra.hadoop.cql3;

import java.lang.reflect.Field;
import java.util.concurrent.ScheduledExecutorService;

import org.apache.cassandra.config.Config;
import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;

public class TWriter2 {

	public static void main(String[] args) {
		String schema = "create table mgindex.test (x int primary key, y int)";
		String insert = "insert into mgindex.test (x,y) values(?,?)";

		String schema1 = "create table mgindex.test1 (x int primary key, y int)";
		String insert1 = "insert into mgindex.test1 (x,y) values(?,?)";

		Config.setClientMode(true);

		try {
			CQLSSTableWriter w = CQLSSTableWriter.builder()
					.inDirectory("/tmp/feat").forTable(schema).using(insert)
					.build();
			w.addRow(1, 1);
			w.close();

			CQLSSTableWriter w1 = CQLSSTableWriter.builder()
					.inDirectory("/tmp/feat").forTable(schema1).using(insert1)
					.build();
			w1.addRow(4, 4);
			w1.close();
		} catch (Exception e) {
			System.out.println(e);
		}

	}
}
{code}

You will see this exception when the second {{forTable}} is called.

{code}
Exception in thread "main" java.lang.ExceptionInInitializerError
	at org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.forTable(CQLSSTableWriter.java:360)
	at org.apache.cassandra.hadoop.cql3.TWriter2.main(TWriter2.java:39)
Caused by: java.lang.NullPointerException
	at org.apache.cassandra.config.DatabaseDescriptor.createAllDirectories(DatabaseDescriptor.java:710)
	at org.apache.cassandra.db.Keyspace.<clinit>(Keyspace.java:72)
	... 2 more
{code}

Here is how the code fails for the second column family:
# For the first column family: ksm is null, and it will work correctly
# For the second column family: ksm is not null, it goes into the second branch to check if the table exists. If not, load the this.schema into Schema.instance. 

{code}
                    // We need to register the keyspace/table metadata through Schema, otherwise we won't be able to properly
                    // build the insert statement in using().
                    KSMetaData ksm = Schema.instance.getKSMetaData(this.schema.ksName);
                    if (ksm == null)
                    {
                        ksm = KSMetaData.newKeyspace(this.schema.ksName,
                                AbstractReplicationStrategy.getClass("org.apache.cassandra.locator.SimpleStrategy"),
                                ImmutableMap.of("replication_factor", "1"),
                                true,
                                Collections.singleton(this.schema));
                        Schema.instance.load(ksm);
                    }
                    else if (Schema.instance.getCFMetaData(this.schema.ksName, this.schema.cfName) == null)
                    {
                        Schema.instance.load(this.schema);
                        ksm = KSMetaData.cloneWith(ksm, Iterables.concat(ksm.cfMetaData().values(), Collections.singleton(this.schema)));
                        Schema.instance.setKeyspaceDefinition(ksm);
                        Keyspace.open(ksm.name).initCf(this.schema.cfId, this.schema.cfName, false);
                    }
                    return this;
{code}

The problem happens when Keyspace class is initialized at {{Keyspace.open...}}, 

{code}
    static
    {
        if (!StorageService.instance.isClientMode())
            DatabaseDescriptor.createAllDirectories();
    }
{code}

At this time, StorageService is not in client mode, and createAllDirectories will fail because conf.data_file_directories is null.

I'm not quite sure why {{Keyspace.open(...}} is needed. When I comment out this line, everything seems working.

Could you reopen this jira and fix it?

> Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
> ---------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7768
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hadoop
>            Reporter: Paul Pak
>            Assignee: Paul Pak
>            Priority: Minor
>              Labels: cql3, hadoop
>         Attachments: trunk-7768-v1.txt
>
>
> The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loads the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)