You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2019/11/15 20:22:00 UTC
[jira] [Commented] (IMPALA-9081)
testMtDopValidationWithHDFSNumRowsEstDisabled toggles isTestEnv() changing
table loading behaviour
[ https://issues.apache.org/jira/browse/IMPALA-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975381#comment-16975381 ]
ASF subversion and git services commented on IMPALA-9081:
---------------------------------------------------------
Commit 4cce896b8fd8cd45f4b69e4ebcd540899b072f3b in impala's branch refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4cce896 ]
IMPALA-9081: fix mt_dop validation tests
This undoes the hack of pretending that it's not a test environment for
that single test. That had side effects, e.g. for the metadata loading
path.
Instead we have a special flag to enable the validation code in
frontend tests.
Note that the plans change to include join build sinks as an
expected result of undoing the hack.
Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f
Reviewed-on: http://gerrit.cloudera.org:8080/14707
Reviewed-by: Anurag Mantripragada <an...@cloudera.com>
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> testMtDopValidationWithHDFSNumRowsEstDisabled toggles isTestEnv() changing table loading behaviour
> --------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9081
> URL: https://issues.apache.org/jira/browse/IMPALA-9081
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Thomas Tauber-Marshall
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: broken-build, flaky
> Fix For: Impala 3.4.0
>
>
> {noformat}
> Section PLAN of query:
> insert into functional_parquet.alltypes partition(year,month)
> select * from functional_parquet.alltypessmall
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=1.01GB mem-reservation=12.09MB thread-reservation=1
> WRITE TO HDFS [functional_parquet.alltypes, OVERWRITE=false, PARTITION-KEYS=(year,month)]
> | partitions=unavailable
> ^^^^^^^^^^^^^^^^^^^^^^^^^
> | output exprs: id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col, year, month
> | mem-estimate=1.00GB mem-reservation=0B thread-reservation=0
> |
> 01:SORT
> | order by: year ASC NULLS LAST, month ASC NULLS LAST
> | mem-estimate=12.00MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
> | tuple-ids=2 row-size=80B cardinality=unavailable
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [functional_parquet.alltypessmall]
> HDFS partitions=4/4 files=4 size=14.76KB
> stored statistics:
> table: rows=unavailable size=unavailable
> partitions: 0/4 rows=unavailable
> columns: unavailable
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=16.00MB mem-reservation=88.00KB thread-reservation=0
> tuple-ids=0 row-size=80B cardinality=unavailable
> in pipelines: 00(GETNEXT)
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=1.01GB mem-reservation=12.09MB thread-reservation=1
> WRITE TO HDFS [functional_parquet.alltypes, OVERWRITE=false, PARTITION-KEYS=(year,month)]
> | partitions=4
> | output exprs: id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col, year, month
> | mem-estimate=1.00GB mem-reservation=0B thread-reservation=0
> |
> 01:SORT
> | order by: year ASC NULLS LAST, month ASC NULLS LAST
> | mem-estimate=12.00MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
> | tuple-ids=2 row-size=80B cardinality=unavailable
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [functional_parquet.alltypessmall]
> HDFS partitions=4/4 files=4 size=14.51KB
> stored statistics:
> table: rows=unavailable size=unavailable
> partitions: 0/4 rows=unavailable
> columns missing stats: id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=16.00MB mem-reservation=88.00KB thread-reservation=0
> tuple-ids=0 row-size=80B cardinality=unavailable
> in pipelines: 00(GETNEXT)
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=1.01GB mem-reservation=12.09MB thread-reservation=1
> WRITE TO HDFS [functional_parquet.alltypes, OVERWRITE=false, PARTITION-KEYS=(year,month)]
> | partitions=unavailable
> | output exprs: id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col, year, month
> | mem-estimate=1.00GB mem-reservation=0B thread-reservation=0
> |
> 01:SORT
> | order by: year ASC NULLS LAST, month ASC NULLS LAST
> | mem-estimate=12.00MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
> | tuple-ids=2 row-size=80B cardinality=unavailable
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [functional_parquet.alltypessmall]
> HDFS partitions=4/4 files=4 size=14.76KB
> stored statistics:
> table: rows=unavailable size=unavailable
> partitions: 0/4 rows=unavailable
> columns: unavailable
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=16.00MB mem-reservation=88.00KB thread-reservation=0
> tuple-ids=0 row-size=80B cardinality=unavailable
> in pipelines: 00(GETNEXT)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org