You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2018/02/14 03:53:15 UTC
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9306
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_kudu.py
18 files changed, 689 insertions(+), 598 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/2
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/9306 )
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
Patch Set 2:
(5 comments)
> (1 comment)
>
> I ran the code formatter and updated the patch with those changes.
>
> I think the runtime filter functionality should be added, but I
> should likely do that in a separate patch to keep this patch
> smaller and focused. I can create a jira to track that if you
> agree. I will look how difficult adding decimal min-max filters is
> tomorrow.
Sounds good.
I think the biggest thing now is just making sure we've tested everything well enough. For example, you could add tests for insert/update/delete with decimal values that are outside the range for the precision/scale of the column, creating a table with a decimal primary key, etc.
http://gerrit.cloudera.org:8080/#/c/9306/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/9306/2//COMMIT_MSG@10
PS2, Line 10:
We generally include a "Testing" section here, eg. what tests were added, what did you run, etc.
You should also take a look at tests/query_tests/test_decimal_queries.py. You should be able to add Kudu to TestDecimalQueries with something like:
"v.get_value('table_format').file_format in ['parquet', 'kudu']"
You'll also need to take a look at testdata/datasets/functional/schema_constraints.csv to add the decimal related tables to data load for Kudu.
http://gerrit.cloudera.org:8080/#/c/9306/1/be/src/exec/kudu-util.h
File be/src/exec/kudu-util.h:
http://gerrit.cloudera.org:8080/#/c/9306/1/be/src/exec/kudu-util.h@a69
PS1, Line 69:
> Since it's not used I don't see a reason to keep it around.
Well its useful for debugging
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc
File be/src/exec/kudu-util.cc:
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc@214
PS2, Line 214: return ColumnType(PrimitiveType::TYPE_DECIMAL);
I imagine the formatter told you to do this. We don't follow it 100%, so you can just follow the example of the above lines.
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java:
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@625
PS2, Line 625: // TODO: Support Kudu Decimal Min/Max Filters
I filed IMPALA-6533. Can you reference that here?
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/util/KuduUtil.java
File fe/src/main/java/org/apache/impala/util/KuduUtil.java:
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/util/KuduUtil.java@400
PS2, Line 400: return org.apache.kudu.Type.DECIMAL;
Another place you can ignore the formatter.
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 16 Feb 2018 04:02:49 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#7).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 784 insertions(+), 626 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/7
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#5).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 780 insertions(+), 624 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/5
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#6).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 784 insertions(+), 626 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/6
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has abandoned this change. ( http://gerrit.cloudera.org:8080/9306 )
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
Abandoned
Abandoning in favor of https://gerrit.cloudera.org/#/c/9368/. Unfortunately I needed to iterate with the pre-review test due to my own environment issues and that muddies the patch set history and had an issue with the jenkins job. I will reference this review set there.
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#8).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 794 insertions(+), 632 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/8
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 8
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#9).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 795 insertions(+), 633 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/9
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#10).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
21 files changed, 803 insertions(+), 641 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/10
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#3).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
20 files changed, 691 insertions(+), 603 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/3
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/9306 )
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
Patch Set 3:
(4 comments)
> For example, you could add tests for insert/update/delete with decimal values that are outside the range for the precision/scale of the column, creating a table with a decimal primary key, etc.
Do these tests exist for the decimal type in general? For other formats like text, parquet, and Avro? I looked in the decimal.test and decimal_exprs.test workloads but didn't see anything. I suspect that sort of "validation" occurs before calling down to the specific format. If not, reusing the existing tests would be great.
http://gerrit.cloudera.org:8080/#/c/9306/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/9306/2//COMMIT_MSG@10
PS2, Line 10:
> We generally include a "Testing" section here, eg. what tests were added, w
Done
http://gerrit.cloudera.org:8080/#/c/9306/1/be/src/exec/kudu-util.h
File be/src/exec/kudu-util.h:
http://gerrit.cloudera.org:8080/#/c/9306/1/be/src/exec/kudu-util.h@a69
PS1, Line 69:
> Well its useful for debugging
I filed KUDU-2303 to add a ToString method to KuduSchema itself.
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc
File be/src/exec/kudu-util.cc:
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc@214
PS2, Line 214:
> I imagine the formatter told you to do this. We don't follow it 100%, so yo
Yeah, I just took what the formatter suggested. Will update.
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/util/KuduUtil.java
File fe/src/main/java/org/apache/impala/util/KuduUtil.java:
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/util/KuduUtil.java@400
PS2, Line 400: Fall through below */
> Another place you can ignore the formatter.
Done
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 16 Feb 2018 16:21:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/9306 )
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc
File be/src/exec/kudu-util.cc:
http://gerrit.cloudera.org:8080/#/c/9306/2/be/src/exec/kudu-util.cc@214
PS2, Line 214:
> Yeah, I just took what the formatter suggested. Will update.
Done
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java:
http://gerrit.cloudera.org:8080/#/c/9306/2/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@625
PS2, Line 625: for (TupleId tupleId: targetSlotsByTid.keySet()) {
> I filed IMPALA-6533. Can you reference that here?
Done
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 16 Feb 2018 16:22:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Taras Bobrovytsky,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9306
to look at the new patch set (#4).
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
IMPALA-5752: Add support for DECIMAL on Kudu tables
Adds support for the Kudu DECIMAL type introduced in Kudu 1.7.0.
Note: Adding support for Kudu decimal min/max filters is
tracked in IMPALA-6533.
Tests:
* Added Kudu create with decimal test to AnalyzeDDLTest.java
* Added Kudu table_format to test_decimal_queries.py
** Both decimal.test and decimal-exprs.test workloads
* Added decimal queries to the following Kudu workloads:
** kudu_create.test
** kudu_delete.test
** kudu_insert.test
** kudu_update.test
** kudu_upsert.test
Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-util.cc
M be/src/exec/kudu-util.h
M be/src/exprs/kudu-partition-expr.cc
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_kudu.py
20 files changed, 716 insertions(+), 603 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/9306/4
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-5752: Add support for DECIMAL on Kudu tables
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/9306 )
Change subject: IMPALA-5752: Add support for DECIMAL on Kudu tables
......................................................................
Patch Set 3:
FYI. I am adding some primary key tests and working on running/validating the tests.
--
To view, visit http://gerrit.cloudera.org:8080/9306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f927fce25dc4fa9529a4e0b688825699f5c0ea6
Gerrit-Change-Number: 9306
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Grant Henke <gr...@gmail.com>
Gerrit-Reviewer: Taras Bobrovytsky <tb...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 16 Feb 2018 18:55:33 +0000
Gerrit-HasComments: No