You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "yiming.xu (JIRA)" <ji...@apache.org> on 2018/11/09 03:30:00 UTC

[jira] [Updated] (PARQUET-1456) Use page index, ParquetFileReader throw ArrayIndexOutOfBoundsException

     [ https://issues.apache.org/jira/browse/PARQUET-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

yiming.xu updated PARQUET-1456:
-------------------------------
    Description: 
hi, We use page index to adaptive spark with master branch find a concurrent problem, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,

    PrimitiveTypeName typeName = type.getPrimitiveTypeName();
    ColumnIndexBuilder builder = BUILDERS.get(typeName);
    if (builder == null) {
      builder = createNewBuilder(type, Integer.MAX_VALUE);
      BUILDERS.put(typeName, builder);
    }

  was:
hi, We use page index to adaptive spark with master branch find a concurrent proble, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,

    PrimitiveTypeName typeName = type.getPrimitiveTypeName();
    ColumnIndexBuilder builder = BUILDERS.get(typeName);
    if (builder == null) {
      builder = createNewBuilder(type, Integer.MAX_VALUE);
      BUILDERS.put(typeName, builder);
    }


> Use page index, ParquetFileReader throw ArrayIndexOutOfBoundsException
> ----------------------------------------------------------------------
>
>                 Key: PARQUET-1456
>                 URL: https://issues.apache.org/jira/browse/PARQUET-1456
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>            Reporter: yiming.xu
>            Priority: Blocker
>
> hi, We use page index to adaptive spark with master branch find a concurrent problem, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,
>     PrimitiveTypeName typeName = type.getPrimitiveTypeName();
>     ColumnIndexBuilder builder = BUILDERS.get(typeName);
>     if (builder == null) {
>       builder = createNewBuilder(type, Integer.MAX_VALUE);
>       BUILDERS.put(typeName, builder);
>     }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)