[jira] [Updated] (IMPALA-2729) Support default values for DECIMAL in Avro.

["Tim Armstrong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong updated IMPALA-2729:
----------------------------------
    Issue Type: New Feature  (was: Bug)

> Support default values for DECIMAL in Avro.
> -------------------------------------------
>
>                 Key: IMPALA-2729
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2729
>             Project: IMPALA
>          Issue Type: New Feature
>          Components: Backend
>    Affects Versions: Impala 2.0, Impala 2.1, Impala 2.2, Impala 2.3.0
>            Reporter: Alexander Behm
>            Priority: Minor
>              Labels: ramp-up, usability
>
> Impala may fail to scan Avro files of a table that has default values specified on DECIMAL columns.
> The following criteria must be met for hitting this bug:
> 1. The Impala table was created from an Avro schema that has a default value on a DECIMAL field
> 2. There is an Avro file storing data of that table that does not have the DECIMAL field declared as part of the table schema from 1
> When querying such a table Impala will return the following error
> {code}
> Field 'decimalFieldName' is missing from file and default values of type DECIMAL are not yet supported.
> {code}
> The relevant code can be found in hdfs-avro-scanner.cc:
> {code}
> Status HdfsAvroScanner::WriteDefaultValue(
>     SlotDescriptor* slot_desc, avro_datum_t default_value, const char* field_name) {
>   if (avro_header_->template_tuple == NULL) {
>     avro_header_->template_tuple = template_tuple_ != NULL ?
>         template_tuple_ : scan_node_->InitEmptyTemplateTuple(*scan_node_->tuple_desc());
>   }
>   switch (default_value->type) {
>     case AVRO_BOOLEAN: {
>       // We don't call VerifyTypesMatch() above the switch statement so we don't want to
>       // call it in the default case (since we VerifyTypesMatch() can't handle every type
>       // either, and we want to return the correct error message).
>       RETURN_IF_ERROR(VerifyTypesMatch(slot_desc, default_value));
>       int8_t v;
>       if (avro_boolean_get(default_value, &v)) DCHECK(false);
>       RawValue::Write(&v, avro_header_->template_tuple, slot_desc, NULL);
>       break;
>     }
>     case AVRO_INT32: {
>       RETURN_IF_ERROR(VerifyTypesMatch(slot_desc, default_value));
>       int32_t v;
>       if (avro_int32_get(default_value, &v)) DCHECK(false);
>       RawValue::Write(&v, avro_header_->template_tuple, slot_desc, NULL);
>       break;
>     }
> ...
>     <--- case for AVRO_DECIMAL not handled
>     default:
>       return Status(TErrorCode::AVRO_UNSUPPORTED_DEFAULT_VALUE, field_name,
>           avro_type_name(default_value->type));
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org