[GitHub] [arrow] westonpace commented on issue #10892: Null values in CSV file if values are quoted

[GitBox <gi...@apache.org>]

westonpace commented on issue #10892:
URL: https://github.com/apache/arrow/issues/10892#issuecomment-894508340


   It appears quoted nulls are only allowed for string columns at the moment.  I agree this is not the behavior we want.  I have opened ARROW-13580 .
   
   In the meantime a workaround may be to read the columns as string and then cast to int8.
   
   ```
   table = csv.read_csv(
           fp,
           convert_options=csv.ConvertOptions(
   	    strings_can_be_null=True,
               null_values=[""],
   	    quoted_strings_can_be_null=True
           ))
   new_columns = []
   for column in table.columns:
       if column.type == pa.utf8():
           new_columns.append(pc.cast(column, pa.int8()))
       else:
           new_columns.append(column)
   new_table = pa.Table.from_arrays(new_columns, names=table.column_names)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org