You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "yiming.xu (JIRA)" <ji...@apache.org> on 2018/11/09 03:30:00 UTC
[jira] [Updated] (PARQUET-1456) Use page index, ParquetFileReader
throw ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/PARQUET-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
yiming.xu updated PARQUET-1456:
-------------------------------
Description:
hi, We use page index to adaptive spark with master branch find a concurrent problem, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,
PrimitiveTypeName typeName = type.getPrimitiveTypeName();
ColumnIndexBuilder builder = BUILDERS.get(typeName);
if (builder == null) {
builder = createNewBuilder(type, Integer.MAX_VALUE);
BUILDERS.put(typeName, builder);
}
was:
hi, We use page index to adaptive spark with master branch find a concurrent proble, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,
PrimitiveTypeName typeName = type.getPrimitiveTypeName();
ColumnIndexBuilder builder = BUILDERS.get(typeName);
if (builder == null) {
builder = createNewBuilder(type, Integer.MAX_VALUE);
BUILDERS.put(typeName, builder);
}
> Use page index, ParquetFileReader throw ArrayIndexOutOfBoundsException
> ----------------------------------------------------------------------
>
> Key: PARQUET-1456
> URL: https://issues.apache.org/jira/browse/PARQUET-1456
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Reporter: yiming.xu
> Priority: Blocker
>
> hi, We use page index to adaptive spark with master branch find a concurrent problem, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,
> PrimitiveTypeName typeName = type.getPrimitiveTypeName();
> ColumnIndexBuilder builder = BUILDERS.get(typeName);
> if (builder == null) {
> builder = createNewBuilder(type, Integer.MAX_VALUE);
> BUILDERS.put(typeName, builder);
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)