You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by sl...@apache.org on 2016/07/19 13:49:23 UTC
cassandra git commit: Option to leave omitted columns in INSERT JSON
unset
Repository: cassandra
Updated Branches:
refs/heads/trunk d314b6057 -> 12911352d
Option to leave omitted columns in INSERT JSON unset
patch by Oded Peer; reviewed by Sylvain Lebresne for CASSANDRA-11424
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12911352
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12911352
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12911352
Branch: refs/heads/trunk
Commit: 12911352de32c948282779a70848211008409441
Parents: d314b60
Author: Sylvain Lebresne <sy...@datastax.com>
Authored: Mon Jul 18 16:05:31 2016 +0200
Committer: Sylvain Lebresne <sy...@datastax.com>
Committed: Tue Jul 19 15:49:09 2016 +0200
----------------------------------------------------------------------
CHANGES.txt | 1 +
NEWS.txt | 5 +--
doc/source/cql/changes.rst | 5 +++
doc/source/cql/dml.rst | 2 +-
doc/source/cql/json.rst | 7 ++-
src/antlr/Lexer.g | 2 +
src/antlr/Parser.g | 4 +-
.../org/apache/cassandra/cql3/Constants.java | 26 +++++++++++
src/java/org/apache/cassandra/cql3/Json.java | 34 ++++++++++-----
.../cql3/statements/UpdateStatement.java | 6 ++-
.../cql3/validation/entities/JsonTest.java | 46 ++++++++++++++++++++
11 files changed, 119 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d30aa4..1a32c63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.10
+ * Option to leave omitted columns in INSERT JSON unset (CASSANDRA-11424)
* Support json/yaml output in nodetool tpstats (CASSANDRA-12035)
* Expose metrics for successful/failed authentication attempts (CASSANDRA-10635)
* Prepend snapshot name with "truncated" or "dropped" when a snapshot
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/NEWS.txt
----------------------------------------------------------------------
diff --git a/NEWS.txt b/NEWS.txt
index 99948fe..0492b25 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -23,18 +23,17 @@ New features
the system keyspace. Upon startup, this table is used to preload all
previously prepared statements - i.e. in many cases clients do not need to
re-prepare statements against restarted nodes.
-
- cqlsh can now connect to older Cassandra versions by downgrading the native
protocol version. Please note that this is currently not part of our release
testing and, as a consequence, it is not guaranteed to work in all cases.
See CASSANDRA-12150 for more details.
-
- Snapshots that are automatically taken before a table is dropped or truncated
will have a "dropped" or "truncated" prefix on their snapshot tag name.
-
- Metrics are exposed for successful and failed authentication attempts.
These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
+ - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
+ See CASSANDRA-11424 for details
Upgrading
---------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/changes.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/changes.rst b/doc/source/cql/changes.rst
index d9aea85..d0c51cc 100644
--- a/doc/source/cql/changes.rst
+++ b/doc/source/cql/changes.rst
@@ -21,6 +21,11 @@ Changes
The following describes the changes in each version of CQL.
+3.4.3
+^^^^^
+
+- Adds a ``DEFAULT UNSET`` option for ``INSERT JSON`` to ignore omitted columns (:jira:`11424`).
+
3.4.2
^^^^^
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/dml.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/dml.rst b/doc/source/cql/dml.rst
index b5f9e9f..f1c126b 100644
--- a/doc/source/cql/dml.rst
+++ b/doc/source/cql/dml.rst
@@ -295,7 +295,7 @@ Inserting data for a row is done using an ``INSERT`` statement:
: [ IF NOT EXISTS ]
: [ USING `update_parameter` ( AND `update_parameter` )* ]
names_values: `names` VALUES `tuple_literal`
- json_clause: JSON `string`
+ json_clause: JSON `string` [ DEFAULT ( NULL | UNSET ) ]
names: '(' `column_name` ( ',' `column_name` )* ')'
For instance::
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/json.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/json.rst b/doc/source/cql/json.rst
index f83f16c..539180a 100644
--- a/doc/source/cql/json.rst
+++ b/doc/source/cql/json.rst
@@ -49,8 +49,11 @@ table with two columns named "myKey" and "value", you would do the following::
INSERT INTO mytable JSON '{ "\"myKey\"": 0, "value": 0}'
-Any columns which are omitted from the ``JSON`` map will be defaulted to a ``NULL`` value (which will result in a
-tombstone being created).
+By default (or if ``DEFAULT NULL`` is explicitly used), a column omitted from the ``JSON`` map will be set to ``NULL``,
+meaning that any pre-existing value for that column will be removed (resulting in a tombstone being created).
+Alternatively, if the ``DEFAULT UNSET`` directive is used after the value, omitted column values will be left unset,
+meaning that pre-existing values for those column will be preserved.
+
JSON Encoding of Cassandra Data Types
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/antlr/Lexer.g
----------------------------------------------------------------------
diff --git a/src/antlr/Lexer.g b/src/antlr/Lexer.g
index 16b2ac4..a65b4f5 100644
--- a/src/antlr/Lexer.g
+++ b/src/antlr/Lexer.g
@@ -195,6 +195,8 @@ K_OR: O R;
K_REPLACE: R E P L A C E;
K_JSON: J S O N;
+K_DEFAULT: D E F A U L T;
+K_UNSET: U N S E T;
K_LIKE: L I K E;
// Case-insensitive alpha characters
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/antlr/Parser.g
----------------------------------------------------------------------
diff --git a/src/antlr/Parser.g b/src/antlr/Parser.g
index f61f464..f00f9d0 100644
--- a/src/antlr/Parser.g
+++ b/src/antlr/Parser.g
@@ -359,12 +359,14 @@ jsonInsertStatement [CFName cf] returns [UpdateStatement.ParsedInsertJson expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
boolean ifNotExists = false;
+ boolean defaultUnset = false;
}
: val=jsonValue
+ ( K_DEFAULT ( K_NULL | ( { defaultUnset = true; } K_UNSET) ) )?
( K_IF K_NOT K_EXISTS { ifNotExists = true; } )?
( usingClause[attrs] )?
{
- $expr = new UpdateStatement.ParsedInsertJson(cf, attrs, val, ifNotExists);
+ $expr = new UpdateStatement.ParsedInsertJson(cf, attrs, val, defaultUnset, ifNotExists);
}
;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/Constants.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java b/src/java/org/apache/cassandra/cql3/Constants.java
index 913ea97..f108e8b 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -40,6 +40,32 @@ public abstract class Constants
STRING, INTEGER, UUID, FLOAT, BOOLEAN, HEX;
}
+ private static class UnsetLiteral extends Term.Raw
+ {
+ public Term prepare(String keyspace, ColumnSpecification receiver) throws InvalidRequestException
+ {
+ return UNSET_VALUE;
+ }
+
+ public AssignmentTestable.TestResult testAssignment(String keyspace, ColumnSpecification receiver)
+ {
+ return AssignmentTestable.TestResult.NOT_ASSIGNABLE;
+ }
+
+ public String getText()
+ {
+ return "";
+ }
+
+ public AbstractType<?> getExactTypeIfKnown(String keyspace)
+ {
+ return null;
+ }
+ }
+
+ // We don't have "unset" literal in the syntax, but it's used implicitely for JSON "DEFAULT UNSET" option
+ public static final UnsetLiteral UNSET_LITERAL = new UnsetLiteral();
+
public static final Value UNSET_VALUE = new Value(ByteBufferUtil.UNSET_BYTE_BUFFER);
private static class NullLiteral extends Term.Raw
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/Json.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/Json.java b/src/java/org/apache/cassandra/cql3/Json.java
index c018f24..2e67a1e 100644
--- a/src/java/org/apache/cassandra/cql3/Json.java
+++ b/src/java/org/apache/cassandra/cql3/Json.java
@@ -111,7 +111,7 @@ public class Json
*/
public static abstract class Prepared
{
- public abstract Term.Raw getRawTermForColumn(ColumnDefinition def);
+ public abstract Term.Raw getRawTermForColumn(ColumnDefinition def, boolean defaultUnset);
}
/**
@@ -126,10 +126,12 @@ public class Json
this.columnMap = columnMap;
}
- public Term.Raw getRawTermForColumn(ColumnDefinition def)
+ public Term.Raw getRawTermForColumn(ColumnDefinition def, boolean defaultUnset)
{
Term value = columnMap.get(def.name);
- return value == null ? Constants.NULL_LITERAL : new ColumnValue(value);
+ return value == null
+ ? (defaultUnset ? Constants.UNSET_LITERAL : Constants.NULL_LITERAL)
+ : new ColumnValue(value);
}
}
@@ -147,9 +149,9 @@ public class Json
this.columns = columns;
}
- public RawDelayedColumnValue getRawTermForColumn(ColumnDefinition def)
+ public RawDelayedColumnValue getRawTermForColumn(ColumnDefinition def, boolean defaultUnset)
{
- return new RawDelayedColumnValue(this, def);
+ return new RawDelayedColumnValue(this, def, defaultUnset);
}
}
@@ -198,17 +200,19 @@ public class Json
{
private final PreparedMarker marker;
private final ColumnDefinition column;
+ private final boolean defaultUnset;
- public RawDelayedColumnValue(PreparedMarker prepared, ColumnDefinition column)
+ public RawDelayedColumnValue(PreparedMarker prepared, ColumnDefinition column, boolean defaultUnset)
{
this.marker = prepared;
this.column = column;
+ this.defaultUnset = defaultUnset;
}
@Override
public Term prepare(String keyspace, ColumnSpecification receiver) throws InvalidRequestException
{
- return new DelayedColumnValue(marker, column);
+ return new DelayedColumnValue(marker, column, defaultUnset);
}
@Override
@@ -235,11 +239,13 @@ public class Json
{
private final PreparedMarker marker;
private final ColumnDefinition column;
+ private final boolean defaultUnset;
- public DelayedColumnValue(PreparedMarker prepared, ColumnDefinition column)
+ public DelayedColumnValue(PreparedMarker prepared, ColumnDefinition column, boolean defaultUnset)
{
this.marker = prepared;
this.column = column;
+ this.defaultUnset = defaultUnset;
}
@Override
@@ -258,7 +264,9 @@ public class Json
public Terminal bind(QueryOptions options) throws InvalidRequestException
{
Term term = options.getJsonColumnValue(marker.bindIndex, column.name, marker.columns);
- return term == null ? null : term.bind(options);
+ return term == null
+ ? (defaultUnset ? Constants.UNSET_VALUE : null)
+ : term.bind(options);
}
@Override
@@ -284,10 +292,16 @@ public class Json
Map<ColumnIdentifier, Term> columnMap = new HashMap<>(expectedReceivers.size());
for (ColumnSpecification spec : expectedReceivers)
{
+ // We explicitely test containsKey() because the value itself can be null, and we want to distinguish an
+ // explicit null value from no value
+ if (!valueMap.containsKey(spec.name.toString()))
+ continue;
+
Object parsedJsonObject = valueMap.remove(spec.name.toString());
if (parsedJsonObject == null)
{
- columnMap.put(spec.name, null);
+ // This is an explicit user null
+ columnMap.put(spec.name, Constants.NULL_VALUE);
}
else
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java b/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
index 3657f94..6bcfd9c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
@@ -203,11 +203,13 @@ public class UpdateStatement extends ModificationStatement
public static class ParsedInsertJson extends ModificationStatement.Parsed
{
private final Json.Raw jsonValue;
+ private final boolean defaultUnset;
- public ParsedInsertJson(CFName name, Attributes.Raw attrs, Json.Raw jsonValue, boolean ifNotExists)
+ public ParsedInsertJson(CFName name, Attributes.Raw attrs, Json.Raw jsonValue, boolean defaultUnset, boolean ifNotExists)
{
super(name, StatementType.INSERT, attrs, null, ifNotExists, false);
this.jsonValue = jsonValue;
+ this.defaultUnset = defaultUnset;
}
@Override
@@ -230,7 +232,7 @@ public class UpdateStatement extends ModificationStatement
if (def.isClusteringColumn())
hasClusteringColumnsSet = true;
- Term.Raw raw = prepared.getRawTermForColumn(def);
+ Term.Raw raw = prepared.getRawTermForColumn(def, defaultUnset);
if (def.isPrimaryKeyColumn())
{
whereClause.add(new SingleColumnRelation(ColumnDefinition.Raw.forColumn(def), Operator.EQ, raw));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
----------------------------------------------------------------------
diff --git a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
index 0e255e4..a14e4a5 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
@@ -794,6 +794,52 @@ public class JsonTest extends CQLTester
}
@Test
+ public void testInsertJsonSyntaxDefaultUnset() throws Throwable
+ {
+ createTable("CREATE TABLE %s (k int primary key, v1 int, v2 int)");
+ execute("INSERT INTO %s JSON ?", "{\"k\": 0, \"v1\": 0, \"v2\": 0}");
+
+ // leave v1 unset
+ execute("INSERT INTO %s JSON ? DEFAULT UNSET", "{\"k\": 0, \"v2\": 2}");
+ assertRows(execute("SELECT * FROM %s"),
+ row(0, 0, 2)
+ );
+
+ // explicit specification DEFAULT NULL
+ execute("INSERT INTO %s JSON ? DEFAULT NULL", "{\"k\": 0, \"v2\": 2}");
+ assertRows(execute("SELECT * FROM %s"),
+ row(0, null, 2)
+ );
+
+ // implicitly setting v2 to null
+ execute("INSERT INTO %s JSON ? DEFAULT NULL", "{\"k\": 0}");
+ assertRows(execute("SELECT * FROM %s"),
+ row(0, null, null)
+ );
+
+ // mix setting null explicitly with default unset:
+ // set values for all fields
+ execute("INSERT INTO %s JSON ?", "{\"k\": 1, \"v1\": 1, \"v2\": 1}");
+ // explicitly set v1 to null while leaving v2 unset which retains its value
+ execute("INSERT INTO %s JSON ? DEFAULT UNSET", "{\"k\": 1, \"v1\": null}");
+ assertRows(execute("SELECT * FROM %s WHERE k=1"),
+ row(1, null, 1)
+ );
+
+ // test string literal instead of bind marker
+ execute("INSERT INTO %s JSON '{\"k\": 2, \"v1\": 2, \"v2\": 2}'");
+ // explicitly set v1 to null while leaving v2 unset which retains its value
+ execute("INSERT INTO %s JSON '{\"k\": 2, \"v1\": null}' DEFAULT UNSET");
+ assertRows(execute("SELECT * FROM %s WHERE k=2"),
+ row(2, null, 2)
+ );
+ execute("INSERT INTO %s JSON '{\"k\": 2}' DEFAULT NULL");
+ assertRows(execute("SELECT * FROM %s WHERE k=2"),
+ row(2, null, null)
+ );
+ }
+
+ @Test
public void testCaseSensitivity() throws Throwable
{
createTable("CREATE TABLE %s (k int primary key, \"Foo\" int)");