You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brad Schoening (Jira)" <ji...@apache.org> on 2023/04/30 02:18:00 UTC

[jira] [Comment Edited] (CASSANDRA-16787) Copy from csv file with duration type fields fails to import

    [ https://issues.apache.org/jira/browse/CASSANDRA-16787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718006#comment-17718006 ] 

Brad Schoening edited comment on CASSANDRA-16787 at 4/30/23 2:17 AM:
---------------------------------------------------------------------

[~brigesh.dungarakoti] [~blerer] [~smiklosovic]  As noted in [CASSANDRA-11873,|http://example.com/] The DataStax Python driver does not appear to support parsing ISO 8601 formatted strings, which defines several formats for duration strings.  There is a [cassandra.util|https://github.com/datastax/python-driver/blob/master/cassandra/util.py] *Duration* class in the driver, but it works with a custom syntax of weeks, days, and nanoseconds. The library [isodate|https://pypi.org/project/isodate/] is commonly used to support parsing ISO strings and could be used here to properly support ISO duration parsing and formating.

It seems to me that deciding upon and placing the library dependency of isodate (or an alternative) and implementing the behavior in the driver would offer the most effective solution for parsing the ISO 8601 string formats instead of bolting it on here. This was mentioned by [~thobbs] in the comments of CASSANDRA-11873: 

 
{noformat}
Last, it looks like COPY FROM doesn't handle duration values when prepared statements are used, but I suppose we need duration support in the python driver for that to be supported.{noformat}
 


was (Author: bschoeni):
[~brigesh.dungarakoti] [~blerer] [~smiklosovic]  As noted in [CASSANDRA-11873|http://example.com]https://issues.apache.org/jira/browse/CASSANDRA-11873] The DataStax Python driver does not appear to support parsing ISO 8601 formatted strings, which defines several formats for duration strings.  There is a [cassandra.util|https://github.com/datastax/python-driver/blob/master/cassandra/util.py] *Duration* class in the driver, but it works with a custom syntax of weeks, days, and nanoseconds. The library [isodate|https://pypi.org/project/isodate/] is commonly used to support parsing ISO strings and could be used here to properly support ISO duration parsing and formmating.

It seems to me that deciding upon and placing the library dependency of isodate (or an alternative) and implementing the behavior in the driver would offer the most effective solution for parsing the ISO 8601 string formats instead of bolting it on here. This was mentioned by [~thobbs] in the comments of CASSANDRA-11873: 

 
{noformat}
Last, it looks like COPY FROM doesn't handle duration values when prepared statements are used, but I suppose we need duration support in the python driver for that to be supported.{noformat}
 

> Copy from csv file with duration type fields fails to import
> ------------------------------------------------------------
>
>                 Key: CASSANDRA-16787
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-16787
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tool/cqlsh
>            Reporter: Brijesh Dungarakoti
>            Assignee: Brad Schoening
>            Priority: Normal
>             Fix For: 4.0.x
>
>         Attachments: error_cassandra_copy_from_1.JPG, error_cassandra_copy_from_2.JPG
>
>
> Getting error:
> {code:java}
> cqlsh> COPY users.user_credentials_by_email FROM '/home/ubuntu/users.csv' WITH HEADER = FALSE AND NULL='null';
> Using 3 child processes
> Starting copy of users.user_credentials_by_email with columns [email, la_duration].
> Failed to make batch statement: Received an argument of invalid type for column "la_duration". Expected: <class 'cassandra.cqltypes.DurationType'>, Got: <type 'str'>; (DurationType arguments must be a Duration.)_
> Failed to import 1 rows: TypeError - Received an argument of invalid type for column "la_duration". Expected: <class 'cassandra.cqltypes.DurationType'>, Got: <type 'str'>; (DurationType arguments must be a Duration.), given up without retries
> Failed to process 1 rows; failed rows written to import_users_user_credentials_by_email.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
> 0 rows imported from 1 files in 0.431 seconds (0 skipped).
> {code}
> *To Reproduce:*
> {code:java}
> CREATE KEYSPACE users WITH replication = {'class': 'SimpleStrategy', 'replication_factor' : 1 } AND durable_writes = true;
> CREATE TABLE users.user_credentials_by_email (
> email text,
> la_duration duration,
> PRIMARY KEY(email)
> );
> {code}
> create users.csv file with:
> {code:java}
> LA@LA.com,8m26s482ms
> {code}
> Run:
> {code:java}
> COPY users.user_credentials_by_email FROM '/home/automaton/users.csv' WITH HEADER = FALSE AND NULL='null';
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org