[jira] [Updated] (ARROW-8142) [Python/C++] Casting empty table from after parquet roundtrip causes critical failure

["Florian Jetter (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Florian Jetter updated ARROW-8142:
----------------------------------
    Description: 
When casting a schema of an empty table from dict encoded to non-dict encoded type a critical error is raised and not handled causing the interpreter to shut down.

This only happens after a parquet roundtrip

 
{code:python}
import pyarrow as paimport pandas as pdimport pyarrow.parquet as pq

df = pd.DataFrame({"col": ["a"]}).astype({"col": "category"}).iloc[:0]
table = pa.Table.from_pandas(df)field = table.schema[0]
new_field = pa.field(field.name, field.type.value_type, field.nullable, field.metadata)

buf = pa.BufferOutputStream()
pq.write_table(table, buf)
reader = pa.BufferReader(buf.getvalue().to_pybytes())
table = pq.read_table(reader)

schema = table.schema.remove(0).insert(0, new_field)
new_table = table.cast(schema)
assert new_table.schema == schema
 {code}
 

Output
{code:java}
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0318 09:55:14.266649 299722176 table.cc:47] Check failed: (chunks.size()) > (0) cannot construct ChunkedArray from empty vector and omitted type {code}
 

Tested on pyarrow==0.16.0 

  was:
When casting a schema of an empty table from dict encoded to non-dict encoded type a critical error is raised and not handled causing the interpreter to shut down.

This only happens after a parquet roundtrip

 
{code:python}
import pyarrow as paimport pandas as pdimport pyarrow.parquet as pq

df = pd.DataFrame({"col": ["a"]}).astype({"col": "category"}).iloc[:0]
table = pa.Table.from_pandas(df)field = table.schema[0]
new_field = pa.field(field.name, field.type.value_type, field.nullable, field.metadata)

buf = pa.BufferOutputStream()
pq.write_table(table, buf)
reader = pa.BufferReader(buf.getvalue().to_pybytes())
table = pq.read_table(reader)

schema = table.schema.remove(0).insert(0, new_field)
new_table = table.cast(schema)
assert new_table.schema == schema
 {code}
 

Output
{code:java}
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0318 09:55:14.266649 299722176 table.cc:47] Check failed: (chunks.size()) > (0) cannot construct ChunkedArray from empty vector and omitted type {code}
 


> [Python/C++] Casting empty table from after parquet roundtrip causes critical failure
> -------------------------------------------------------------------------------------
>
>                 Key: ARROW-8142
>                 URL: https://issues.apache.org/jira/browse/ARROW-8142
>             Project: Apache Arrow
>          Issue Type: Bug
>            Reporter: Florian Jetter
>            Priority: Major
>
> When casting a schema of an empty table from dict encoded to non-dict encoded type a critical error is raised and not handled causing the interpreter to shut down.
> This only happens after a parquet roundtrip
>  
> {code:python}
> import pyarrow as paimport pandas as pdimport pyarrow.parquet as pq
> df = pd.DataFrame({"col": ["a"]}).astype({"col": "category"}).iloc[:0]
> table = pa.Table.from_pandas(df)field = table.schema[0]
> new_field = pa.field(field.name, field.type.value_type, field.nullable, field.metadata)
> buf = pa.BufferOutputStream()
> pq.write_table(table, buf)
> reader = pa.BufferReader(buf.getvalue().to_pybytes())
> table = pq.read_table(reader)
> schema = table.schema.remove(0).insert(0, new_field)
> new_table = table.cast(schema)
> assert new_table.schema == schema
>  {code}
>  
> Output
> {code:java}
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> F0318 09:55:14.266649 299722176 table.cc:47] Check failed: (chunks.size()) > (0) cannot construct ChunkedArray from empty vector and omitted type {code}
>  
> Tested on pyarrow==0.16.0 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)