You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by st...@apache.org on 2017/02/22 10:02:26 UTC
[1/6] cassandra git commit: Fix cqlsh COPY for dates before 1900
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 0eae9084e -> f0502aa79
refs/heads/cassandra-3.11 c77d63ef5 -> 10524bda9
refs/heads/trunk 62c948704 -> 95bcaef73
Fix cqlsh COPY for dates before 1900
patch by Stefania Alborghetti; reviewed by Tyler Hobbs for CASSANDRA-13185
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0502aa7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0502aa7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0502aa7
Branch: refs/heads/cassandra-3.0
Commit: f0502aa791897a30aea371a3032ea5ef679d25cc
Parents: 0eae908
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Tue Feb 21 12:23:17 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 09:59:08 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 2 ++
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 11 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index f2419d6..91a6b31 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
3.0.12
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+
Merged from 2.2
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
* Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/pylib/cqlshlib/formatting.py
----------------------------------------------------------------------
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index dcd08da..097b1a7 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -246,7 +246,15 @@ def strftime(time_format, seconds, timezone=None):
ret_dt = datetime_from_timestamp(seconds).replace(tzinfo=UTC())
if timezone:
ret_dt = ret_dt.astimezone(timezone)
- return ret_dt.strftime(time_format)
+ try:
+ return ret_dt.strftime(time_format)
+ except ValueError:
+ # CASSANDRA-13185: if the date cannot be formatted as a string, return a string with the milliseconds
+ # since the epoch. cqlsh does the exact same thing for values below datetime.MINYEAR (1) or above
+ # datetime.MAXYEAR (9999). Some versions of strftime() also have problems for dates between MIN_YEAR and 1900.
+ # cqlsh COPY assumes milliseconds from the epoch if it fails to parse a datetime string, and so it is
+ # able to correctly import timestamps exported as milliseconds since the epoch.
+ return '%d' % (seconds * 1000.0)
@formatter_for('Date')
[2/6] cassandra git commit: Fix cqlsh COPY for dates before 1900
Posted by st...@apache.org.
Fix cqlsh COPY for dates before 1900
patch by Stefania Alborghetti; reviewed by Tyler Hobbs for CASSANDRA-13185
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0502aa7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0502aa7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0502aa7
Branch: refs/heads/cassandra-3.11
Commit: f0502aa791897a30aea371a3032ea5ef679d25cc
Parents: 0eae908
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Tue Feb 21 12:23:17 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 09:59:08 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 2 ++
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 11 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index f2419d6..91a6b31 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
3.0.12
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+
Merged from 2.2
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
* Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/pylib/cqlshlib/formatting.py
----------------------------------------------------------------------
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index dcd08da..097b1a7 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -246,7 +246,15 @@ def strftime(time_format, seconds, timezone=None):
ret_dt = datetime_from_timestamp(seconds).replace(tzinfo=UTC())
if timezone:
ret_dt = ret_dt.astimezone(timezone)
- return ret_dt.strftime(time_format)
+ try:
+ return ret_dt.strftime(time_format)
+ except ValueError:
+ # CASSANDRA-13185: if the date cannot be formatted as a string, return a string with the milliseconds
+ # since the epoch. cqlsh does the exact same thing for values below datetime.MINYEAR (1) or above
+ # datetime.MAXYEAR (9999). Some versions of strftime() also have problems for dates between MIN_YEAR and 1900.
+ # cqlsh COPY assumes milliseconds from the epoch if it fails to parse a datetime string, and so it is
+ # able to correctly import timestamps exported as milliseconds since the epoch.
+ return '%d' % (seconds * 1000.0)
@formatter_for('Date')
[3/6] cassandra git commit: Fix cqlsh COPY for dates before 1900
Posted by st...@apache.org.
Fix cqlsh COPY for dates before 1900
patch by Stefania Alborghetti; reviewed by Tyler Hobbs for CASSANDRA-13185
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f0502aa7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f0502aa7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f0502aa7
Branch: refs/heads/trunk
Commit: f0502aa791897a30aea371a3032ea5ef679d25cc
Parents: 0eae908
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Tue Feb 21 12:23:17 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 09:59:08 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 2 ++
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 11 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index f2419d6..91a6b31 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
3.0.12
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+
Merged from 2.2
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
* Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f0502aa7/pylib/cqlshlib/formatting.py
----------------------------------------------------------------------
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index dcd08da..097b1a7 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -246,7 +246,15 @@ def strftime(time_format, seconds, timezone=None):
ret_dt = datetime_from_timestamp(seconds).replace(tzinfo=UTC())
if timezone:
ret_dt = ret_dt.astimezone(timezone)
- return ret_dt.strftime(time_format)
+ try:
+ return ret_dt.strftime(time_format)
+ except ValueError:
+ # CASSANDRA-13185: if the date cannot be formatted as a string, return a string with the milliseconds
+ # since the epoch. cqlsh does the exact same thing for values below datetime.MINYEAR (1) or above
+ # datetime.MAXYEAR (9999). Some versions of strftime() also have problems for dates between MIN_YEAR and 1900.
+ # cqlsh COPY assumes milliseconds from the epoch if it fails to parse a datetime string, and so it is
+ # able to correctly import timestamps exported as milliseconds since the epoch.
+ return '%d' % (seconds * 1000.0)
@formatter_for('Date')
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into
cassandra-3.11
Posted by st...@apache.org.
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10524bda
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10524bda
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10524bda
Branch: refs/heads/cassandra-3.11
Commit: 10524bda9e8a197c260daa6c439fd5fa0610b9d4
Parents: c77d63e f0502aa
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Wed Feb 22 10:01:09 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 10:01:09 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 1 +
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 10 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/10524bda/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 846fc20,91a6b31..fd758dc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,14 +1,16 @@@
-3.0.12
+3.11.0
+ * Obfuscate password in stress-graphs (CASSANDRA-12233)
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
+ * nodetool stopdaemon errors out (CASSANDRA-13030)
+ * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
+ * Fix primary index calculation for SASI (CASSANDRA-12910)
+ * More fixes to the TokenAllocator (CASSANDRA-12990)
+ * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983)
+Merged from 3.0:
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
-
-Merged from 2.2
- * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
- * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018)
-
-3.0.11
* Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639)
* Add vm.max_map_count StartupCheck (CASSANDRA-13008)
- * Hint related logging should include the IP address of the destination in addition to
+ * Hint related logging should include the IP address of the destination in addition to
host ID (CASSANDRA-13205)
* Reloading logback.xml does not work (CASSANDRA-13173)
* Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 (CASSANDRA-13109)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/10524bda/pylib/cqlshlib/formatting.py
----------------------------------------------------------------------
diff --cc pylib/cqlshlib/formatting.py
index daa6529,097b1a7..08665fd
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@@ -348,33 -241,21 +348,41 @@@ def format_value_timestamp(val, colorma
bval = "'%s'" % bval
return colorme(bval, colormap, 'timestamp')
+formatter_for('timestamp')(format_value_timestamp)
-def strftime(time_format, seconds, timezone=None):
- ret_dt = datetime_from_timestamp(seconds).replace(tzinfo=UTC())
+
+def strftime(time_format, seconds, microseconds=0, timezone=None):
+ ret_dt = datetime_from_timestamp(seconds) + datetime.timedelta(microseconds=microseconds)
+ ret_dt = ret_dt.replace(tzinfo=UTC())
if timezone:
ret_dt = ret_dt.astimezone(timezone)
- return ret_dt.strftime(time_format)
+ try:
+ return ret_dt.strftime(time_format)
+ except ValueError:
+ # CASSANDRA-13185: if the date cannot be formatted as a string, return a string with the milliseconds
+ # since the epoch. cqlsh does the exact same thing for values below datetime.MINYEAR (1) or above
+ # datetime.MAXYEAR (9999). Some versions of strftime() also have problems for dates between MIN_YEAR and 1900.
+ # cqlsh COPY assumes milliseconds from the epoch if it fails to parse a datetime string, and so it is
+ # able to correctly import timestamps exported as milliseconds since the epoch.
+ return '%d' % (seconds * 1000.0)
+microseconds_regex = re.compile("(.*)(?:\.(\d{1,6}))(.*)")
+
+
+def round_microseconds(val):
+ """
+ For COPY TO, we need to round microsecond to milliseconds because server side
+ TimestampSerializer.dateStringPatterns only parses milliseconds. If we keep microseconds,
+ users may try to import with COPY FROM a file generated with COPY TO and have problems if
+ prepared statements are disabled, see CASSANDRA-11631.
+ """
+ m = microseconds_regex.match(val)
+ if not m:
+ return val
+
+ milliseconds = int(m.group(2)) * pow(10, 3 - len(m.group(2)))
+ return '%s.%03d%s' % (m.group(1), milliseconds, '' if not m.group(3) else m.group(3))
+
@formatter_for('Date')
def format_value_date(val, colormap, **_):
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into
cassandra-3.11
Posted by st...@apache.org.
Merge branch 'cassandra-3.0' into cassandra-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10524bda
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10524bda
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10524bda
Branch: refs/heads/trunk
Commit: 10524bda9e8a197c260daa6c439fd5fa0610b9d4
Parents: c77d63e f0502aa
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Wed Feb 22 10:01:09 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 10:01:09 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 1 +
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 10 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/10524bda/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 846fc20,91a6b31..fd758dc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,14 +1,16 @@@
-3.0.12
+3.11.0
+ * Obfuscate password in stress-graphs (CASSANDRA-12233)
+ * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
+ * nodetool stopdaemon errors out (CASSANDRA-13030)
+ * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
+ * Fix primary index calculation for SASI (CASSANDRA-12910)
+ * More fixes to the TokenAllocator (CASSANDRA-12990)
+ * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983)
+Merged from 3.0:
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
-
-Merged from 2.2
- * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
- * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018)
-
-3.0.11
* Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639)
* Add vm.max_map_count StartupCheck (CASSANDRA-13008)
- * Hint related logging should include the IP address of the destination in addition to
+ * Hint related logging should include the IP address of the destination in addition to
host ID (CASSANDRA-13205)
* Reloading logback.xml does not work (CASSANDRA-13173)
* Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 (CASSANDRA-13109)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/10524bda/pylib/cqlshlib/formatting.py
----------------------------------------------------------------------
diff --cc pylib/cqlshlib/formatting.py
index daa6529,097b1a7..08665fd
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@@ -348,33 -241,21 +348,41 @@@ def format_value_timestamp(val, colorma
bval = "'%s'" % bval
return colorme(bval, colormap, 'timestamp')
+formatter_for('timestamp')(format_value_timestamp)
-def strftime(time_format, seconds, timezone=None):
- ret_dt = datetime_from_timestamp(seconds).replace(tzinfo=UTC())
+
+def strftime(time_format, seconds, microseconds=0, timezone=None):
+ ret_dt = datetime_from_timestamp(seconds) + datetime.timedelta(microseconds=microseconds)
+ ret_dt = ret_dt.replace(tzinfo=UTC())
if timezone:
ret_dt = ret_dt.astimezone(timezone)
- return ret_dt.strftime(time_format)
+ try:
+ return ret_dt.strftime(time_format)
+ except ValueError:
+ # CASSANDRA-13185: if the date cannot be formatted as a string, return a string with the milliseconds
+ # since the epoch. cqlsh does the exact same thing for values below datetime.MINYEAR (1) or above
+ # datetime.MAXYEAR (9999). Some versions of strftime() also have problems for dates between MIN_YEAR and 1900.
+ # cqlsh COPY assumes milliseconds from the epoch if it fails to parse a datetime string, and so it is
+ # able to correctly import timestamps exported as milliseconds since the epoch.
+ return '%d' % (seconds * 1000.0)
+microseconds_regex = re.compile("(.*)(?:\.(\d{1,6}))(.*)")
+
+
+def round_microseconds(val):
+ """
+ For COPY TO, we need to round microsecond to milliseconds because server side
+ TimestampSerializer.dateStringPatterns only parses milliseconds. If we keep microseconds,
+ users may try to import with COPY FROM a file generated with COPY TO and have problems if
+ prepared statements are disabled, see CASSANDRA-11631.
+ """
+ m = microseconds_regex.match(val)
+ if not m:
+ return val
+
+ milliseconds = int(m.group(2)) * pow(10, 3 - len(m.group(2)))
+ return '%s.%03d%s' % (m.group(1), milliseconds, '' if not m.group(3) else m.group(3))
+
@formatter_for('Date')
def format_value_date(val, colormap, **_):
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Posted by st...@apache.org.
Merge branch 'cassandra-3.11' into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95bcaef7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95bcaef7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95bcaef7
Branch: refs/heads/trunk
Commit: 95bcaef735a4034f3fb8586a5210c748d43792c9
Parents: 62c9487 10524bd
Author: Stefania Alborghetti <st...@datastax.com>
Authored: Wed Feb 22 10:01:27 2017 +0000
Committer: Stefania Alborghetti <st...@datastax.com>
Committed: Wed Feb 22 10:01:27 2017 +0000
----------------------------------------------------------------------
CHANGES.txt | 1 +
pylib/cqlshlib/formatting.py | 10 +++++++++-
2 files changed, 10 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95bcaef7/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 751e91d,fd758dc..b4eaf63
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -43,9 -7,9 +43,10 @@@
* More fixes to the TokenAllocator (CASSANDRA-12990)
* NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983)
Merged from 3.0:
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
* Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639)
* Add vm.max_map_count StartupCheck (CASSANDRA-13008)
+ * Obfuscate password in stress-graphs (CASSANDRA-12233)
* Hint related logging should include the IP address of the destination in addition to
host ID (CASSANDRA-13205)
* Reloading logback.xml does not work (CASSANDRA-13173)