You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "jelech (via GitHub)" <gi...@apache.org> on 2023/06/12 12:05:23 UTC

[GitHub] [arrow] jelech opened a new issue, #36037: [Python] pyarrow read tsv file failed

jelech opened a new issue, #36037:
URL: https://github.com/apache/arrow/issues/36037

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   pyarrow version: 12.0.0
   
   ## tsv file
   tsv file from [data](https://github.com/google-research-datasets/wit/blob/main/DATA.md) part of [sample file](https://storage.googleapis.com/gresearch/wit/wit_v1.train.all-1percent_sample.tsv.gz)
   
   ## code
   ```python
   import pyarrow.csv as csv_pq
   with open("f.tsv", "rb") as file:
           tsv_file = csv_pq.read_csv(file, parse_options=csv_pq.ParseOptions(delimiter="\t"))
   ```
   
   ## error:
   ```shell
     tsv_file = csv_pq.read_csv(file, parse_options=csv_pq.ParseOptions(delimiter="\t"))
     File "pyarrow/_csv.pyx", line 1235, in pyarrow._csv.read_csv
     File "pyarrow/_csv.pyx", line 1244, in pyarrow._csv.read_csv
     File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
     File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
   pyarrow.lib.ArrowInvalid: CSV parse error: Expected 17 columns, got 1: В декабре 2009 года, Ford объявили, что кастомная раскрас ...
   ```
   
   and the error will change, when you delete line2~line2208. It looks like the error is related to the read buffer pool?
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] westonpace commented on issue #36037: [Python] pyarrow read tsv file failed

Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on issue #36037:
URL: https://github.com/apache/arrow/issues/36037#issuecomment-1591991181

   Those files have newlines in some of the values.  You need `ParseOptions(delimiter="\t", newlines_in_values=True)`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jelech closed issue #36037: [Python] pyarrow read tsv file failed

Posted by "jelech (via GitHub)" <gi...@apache.org>.
jelech closed issue #36037: [Python] pyarrow read tsv file failed
URL: https://github.com/apache/arrow/issues/36037


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jelech commented on issue #36037: [Python] pyarrow read tsv file failed

Posted by "jelech (via GitHub)" <gi...@apache.org>.
jelech commented on issue #36037:
URL: https://github.com/apache/arrow/issues/36037#issuecomment-1596415078

   thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org