You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/07/10 18:06:00 UTC
[jira] [Commented] (DRILL-6588) System table columns incorrectly
marked as non-nullable
[ https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539040#comment-16539040 ]
ASF GitHub Bot commented on DRILL-6588:
---------------------------------------
kkhatua opened a new pull request #1371: DRILL-6588: Make Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371
This addresses the issue of columns in the System tables being marked as non-nullable. While these tables are immutable, they can carry nullable values as well. (e.g. `num_val` in `sys.options`)
This commit introduces an annotation for the `PojoReader` that applies the nullable property if explicitly defined for a column (i.e. explicitly defined `isNullable` for a member of the POJO instance).
```
apache drill 1.14.0-SNAPSHOT
"just drill it"
0: jdbc:drill:schema=sys> select count(*) from sys.options where num_val is null;
+---------+
| EXPR$0 |
+---------+
| 108 |
+---------+
1 row selected (2.703 seconds)
0: jdbc:drill:schema=sys> select count(*) from sys.options where isnull(num_val);
+---------+
| EXPR$0 |
+---------+
| 108 |
+---------+
1 row selected (0.3 seconds)
0: jdbc:drill:schema=sys> select distinct is_nullable, count(*) from INFORMATION_SCHEMA.`COLUMNS` where table_schema = 'sys' group by is_nullable;
+--------------+---------+
| is_nullable | EXPR$1 |
+--------------+---------+
| NO | 36 |
| YES | 50 |
+--------------+---------+
2 rows selected (0.69 seconds)
0: jdbc:drill:schema=sys> describe options;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| optionScope | CHARACTER VARYING | NO |
| status | CHARACTER VARYING | NO |
| num_val | BIGINT | YES |
| string_val | CHARACTER VARYING | YES |
| bool_val | BOOLEAN | YES |
| float_val | DOUBLE | YES |
+-------------------+--------------------+--------------+
9 rows selected (0.221 seconds)
0: jdbc:drill:schema=sys> describe internal_options;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| optionScope | CHARACTER VARYING | NO |
| status | CHARACTER VARYING | NO |
| num_val | BIGINT | YES |
| string_val | CHARACTER VARYING | YES |
| bool_val | BOOLEAN | YES |
| float_val | DOUBLE | YES |
+-------------------+--------------------+--------------+
9 rows selected (0.185 seconds)
0: jdbc:drill:schema=sys> describe options_val;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| val | CHARACTER VARYING | YES |
| optionScope | CHARACTER VARYING | NO |
+-------------------+--------------------+--------------+
5 rows selected (0.183 seconds)
0: jdbc:drill:schema=sys> describe profiles_json;
+--------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--------------+--------------------+--------------+
| queryId | CHARACTER VARYING | NO |
| json | CHARACTER VARYING | YES |
+--------------+--------------------+--------------+
2 rows selected (0.727 seconds)
0: jdbc:drill:schema=sys> describe profiles;
+--------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--------------+--------------------+--------------+
| queryId | CHARACTER VARYING | NO |
| startTime | TIMESTAMP | YES |
| foreman | CHARACTER VARYING | NO |
| fragments | BIGINT | YES |
| user | CHARACTER VARYING | YES |
| queue | CHARACTER VARYING | YES |
| planTime | BIGINT | YES |
| queueTime | BIGINT | YES |
| executeTime | BIGINT | YES |
| totalTime | BIGINT | YES |
| state | CHARACTER VARYING | YES |
| query | CHARACTER VARYING | YES |
+--------------+--------------------+--------------+
12 rows selected (0.309 seconds)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> System table columns incorrectly marked as non-nullable
> --------------------------------------------------------
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
> Affects Versions: 1.13.0
> Reporter: Aman Sinha
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +-------------------+--------------------+--------------+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +-------------------+--------------------+--------------+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +-------------------+--------------------+--------------+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---------------------------------------------------+----------+------------------+-------------+--------+---------+------------+----------+-----------+
> | name | kind | accessibleScopes | optionScope | status | num_val | string_val | bool_val | float_val |
> +---------------------------------------------------+----------+------------------+-------------+--------+---------+------------+----------+-----------+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT | BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as `WHERE <column> IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)