You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/10/18 08:22:26 UTC
[GitHub] [arrow] EdisonZiXing edited a comment on issue #11427: ArrowInvalid: CSV parse error: Expected 18 columns, got 13
EdisonZiXing edited a comment on issue #11427:
URL: https://github.com/apache/arrow/issues/11427#issuecomment-945495667
@amol- Thx for replying ,master!
The way 'not working' is that the same error comes when I try the settings above.
Seems like the last row of my file cause the errors. The original file is 1.5GB
Then I trying to make a minimal CSV file with the last row:
Step1 When I use pandas.read_csv() to read the file as pandas.DataFrame ,it worked.
Step 2 Then I take out the last 10 row of the DataFrame and use pandas.to_csv() got another file test_minimal.csv
Step 3 Uesd pyarrow.csv.read_csv() to read the test_minimal,csv again ,then succeed!
So I wonder if the pyarrow.csv.read_csv() NaN detecting problem?
Looking forwards to your replies!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org