You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Alejandro Marco Ramos (Jira)" <ji...@apache.org> on 2022/09/20 03:39:00 UTC
[jira] [Updated] (ARROW-17774) Python: write csv cast error
[ https://issues.apache.org/jira/browse/ARROW-17774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Marco Ramos updated ARROW-17774:
------------------------------------------
Description:
Hi, when try to write table with any field in `Decimal128` type, arrow raises with this message:
{code:java}
File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\dataset.py:930, in write_dataset(data, base_dir, basename_template, format, partitioning, partitioning_flavor, schema, filesystem, file_options, use_threads, max_partitions, max_open_files, max_rows_per_file, min_rows_per_group, max_rows_per_group, file_visitor, existing_data_behavior, create_dir) 927 raise ValueError("Cannot specify a schema when writing a Scanner") 928 scanner = data --> 930 _filesystemdataset_write( 931 scanner, base_dir, basename_template, filesystem, partitioning, 932 file_options, max_partitions, file_visitor, existing_data_behavior, 933 max_open_files, max_rows_per_file, 934 min_rows_per_group, max_rows_per_group, create_dir 935 )File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\_dataset.pyx:2737, in pyarrow._dataset._filesystemdataset_write()File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\error.pxi:121, in pyarrow.lib.check_status() ArrowNotImplementedError: Unsupported cast from decimal128(21, 15) to utf8 using function cast_string {code}
my data is:
{noformat}
In [137]: table
Out[137]:
pyarrow.Table
col1: int64
col2: double
col3: decimal128(21, 15)
col4: string
----
col1: [[1,2,3,0]]
col2: [[2.7,0,3.24,3]]
col3: [[-304236.460000000000000,0.E-15,0.E-15,0.E-15]]
col4: [["primera","segunda","tercera","cuarta"]]{noformat}
Thanks in advance.
was:
Hi, when try to write table with any field in `Decimal128` type, arrow raises with this message:
{noformat}
File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\dataset.py:930, in write_dataset(data, base_dir, basename_template, format, partitioning, partitioning_flavor, schema, filesystem, file_options, use_threads, max_partitions, max_open_files, max_rows_per_file, min_rows_per_group, max_rows_per_group, file_visitor, existing_data_behavior, create_dir)
927 raise ValueError("Cannot specify a schema when writing a Scanner")
928 scanner = data
--> 930 _filesystemdataset_write(
931 scanner, base_dir, basename_template, filesystem, partitioning,
932 file_options, max_partitions, file_visitor, existing_data_behavior,
933 max_open_files, max_rows_per_file,
934 min_rows_per_group, max_rows_per_group, create_dir
935 )File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\_dataset.pyx:2737, in pyarrow._dataset._filesystemdataset_write()File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\error.pxi:121, in pyarrow.lib.check_status()
ArrowNotImplementedError: Unsupported cast from decimal128(21, 15) to utf8 using function cast_string{noformat}
my data is:
{noformat}
In [137]: table
Out[137]:
pyarrow.Table
col1: int64
col2: double
col3: decimal128(21, 15)
col4: string
----
col1: [[1,2,3,0]]
col2: [[2.7,0,3.24,3]]
col3: [[-304236.460000000000000,0.E-15,0.E-15,0.E-15]]
col4: [["primera","segunda","tercera","cuarta"]]{noformat}
Thanks in advance.
> Python: write csv cast error
> ----------------------------
>
> Key: ARROW-17774
> URL: https://issues.apache.org/jira/browse/ARROW-17774
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Affects Versions: 8.0.0
> Reporter: Alejandro Marco Ramos
> Priority: Major
>
> Hi, when try to write table with any field in `Decimal128` type, arrow raises with this message:
> {code:java}
> File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\dataset.py:930, in write_dataset(data, base_dir, basename_template, format, partitioning, partitioning_flavor, schema, filesystem, file_options, use_threads, max_partitions, max_open_files, max_rows_per_file, min_rows_per_group, max_rows_per_group, file_visitor, existing_data_behavior, create_dir) 927 raise ValueError("Cannot specify a schema when writing a Scanner") 928 scanner = data --> 930 _filesystemdataset_write( 931 scanner, base_dir, basename_template, filesystem, partitioning, 932 file_options, max_partitions, file_visitor, existing_data_behavior, 933 max_open_files, max_rows_per_file, 934 min_rows_per_group, max_rows_per_group, create_dir 935 )File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\_dataset.pyx:2737, in pyarrow._dataset._filesystemdataset_write()File c:\users\documents\projects\.venv\lib\site-packages\pyarrow\error.pxi:121, in pyarrow.lib.check_status() ArrowNotImplementedError: Unsupported cast from decimal128(21, 15) to utf8 using function cast_string {code}
> my data is:
> {noformat}
> In [137]: table
> Out[137]:
> pyarrow.Table
> col1: int64
> col2: double
> col3: decimal128(21, 15)
> col4: string
> ----
> col1: [[1,2,3,0]]
> col2: [[2.7,0,3.24,3]]
> col3: [[-304236.460000000000000,0.E-15,0.E-15,0.E-15]]
> col4: [["primera","segunda","tercera","cuarta"]]{noformat}
>
> Thanks in advance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)