[impala] branch master updated: IMPALA-10808: Fix crash on illegal decimal precision in parquet file schema

[wz...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

wzhou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new b0d364c  IMPALA-10808: Fix crash on illegal decimal precision in parquet file schema
b0d364c is described below

commit b0d364cc5a62fcfb7ad26d52d3f492239151a3c1
Author: stiga-huang <hu...@gmail.com>
AuthorDate: Tue Jul 20 15:59:28 2021 +0800

    IMPALA-10808: Fix crash on illegal decimal precision in parquet file schema
    
    In IMPALA-7087, we allow parquet file schema to have decimal precision
    lower than the one in table schema. However, we should check illegal
    precisions like 0 or negative values. They will hit a DCHECK in
    ColumnType::GetDecimalByteSize(int).
    
    Change-Id: If84fe14740aea97461ebccedffa4c2ca393333d8
    Reviewed-on: http://gerrit.cloudera.org:8080/17701
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 be/src/exec/parquet/parquet-metadata-utils.cc | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/be/src/exec/parquet/parquet-metadata-utils.cc b/be/src/exec/parquet/parquet-metadata-utils.cc
index dfd6ce3..e7ee0e0 100644
--- a/be/src/exec/parquet/parquet-metadata-utils.cc
+++ b/be/src/exec/parquet/parquet-metadata-utils.cc
@@ -330,12 +330,14 @@ Status ParquetMetadataUtils::ValidateColumn(const char* filename,
           filename, schema_element.name));
     }
 
-    // The other decimal metadata should be there but we don't need it.
+    // We require that the precision be a positive value, and not larger than the
+    // precision in table schema.
     if (!schema_element.__isset.precision) {
       ErrorMsg msg(TErrorCode::PARQUET_MISSING_PRECISION, filename, schema_element.name);
       RETURN_IF_ERROR(state->LogOrReturnError(msg));
     } else {
-      if (schema_element.precision > slot_desc->type().precision) {
+      if (schema_element.precision > slot_desc->type().precision
+          || schema_element.precision <= 0) {
         ErrorMsg msg(TErrorCode::PARQUET_WRONG_PRECISION, filename, schema_element.name,
             schema_element.precision, slot_desc->type().precision);
         RETURN_IF_ERROR(state->LogOrReturnError(msg));