You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/05/20 06:40:32 UTC
[GitHub] [iceberg] maxingui opened a new issue #2619: trino iceberg planning time so long
maxingui opened a new issue #2619:
URL: https://github.com/apache/iceberg/issues/2619
we use trino iceberg with huawei OBS,the planning time is more than 20 s，while the total SCHEDULE and RUNNING time is only 2 s ,I do not what worry with it;
the debug log is like that:
2021-05-20T06:06:01.039Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:icebergsf1_02, tblName:customer, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 8.69ms
2021-05-20T06:06:01.049Z INFO Query-20210520_060537_00323_gbd7b-4734 org.apache.iceberg.BaseTableScan Scanning table "icebergsf1_02".customer snapshot 5772262083695712847 created at 2021-05-18 07:43:19.363 with filter true
2021-05-20T06:06:01.127Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:icebergsf1_02, tblName:customer, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 8.70ms
2021-05-20T06:06:01.136Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:icebergsf1_02, tblName:customer, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 7.93ms
2021-05-20T06:06:01.158Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:icebergsf1_02, tblName:customer, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 8.10ms
2021-05-20T06:06:01.171Z INFO Query-20210520_060537_00323_gbd7b-4734 org.apache.iceberg.BaseTableScan Scanning table "icebergsf1_02".customer snapshot 5772262083695712847 created at 2021-05-18 07:43:19.363 with filter true
2021-05-20T06:06:01.193Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.sql.planner.DistributedExecutionPlanner Dynamic filters: [Descriptor{id=df_2014, input="sr_returned_date_sk_0", operator=EQUAL, nullAllowed=false}]
2021-05-20T06:06:01.203Z DEBUG Query-20210520_060537_00323_gbd7b-4734 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:icebergsf1_02, tblName:store_returns, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 8.39ms
2021-05-20T06:06:01.214Z INFO Query-20210520_060537_00323_gbd7b-4734 org.apache.iceberg.BaseTableScan Scanning table "icebergsf1_02"."store_returns" snapshot 3075814051181790716 created at 2021-05-18 07:44:47.401 with filter true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] dixingxing0 edited a comment on issue #2619: trino iceberg planning time so long
Posted by GitBox <gi...@apache.org>.
dixingxing0 edited a comment on issue #2619:
URL: https://github.com/apache/iceberg/issues/2619#issuecomment-844838873
It seems your log is incomplete, i think it's better to have more log.
Since you created this issue in iceberg project, i assume you are more inclined to think it is related to iceberg, do you have more clues about it?
Did you have tried with other engine like spark sql, if spark sql also planned slow, then maybe it is irrelevant to trino.
BTW, i've tried trino356 (with PR https://github.com/trinodb/trino/pull/7367), the analyze phase was much faster, like from 2s reduced to 1s in my test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] dixingxing0 commented on issue #2619: trino iceberg planning time so long
Posted by GitBox <gi...@apache.org>.
dixingxing0 commented on issue #2619:
URL: https://github.com/apache/iceberg/issues/2619#issuecomment-844838873
It seems that your log is incomplete, i think it's better to have more log.
Since you created this issue in iceberg project, i assume you are more inclined to think it is related to iceberg, do you have more clues about it?
Did you have tried with other engine like spark sql, if spark sql also planned slow, then maybe it is irrelevant to trino.
BTW, i've tried trino356 (with PR https://github.com/trinodb/trino/pull/7367), the analyze phase was much faster, like from 2s reduced to 1s in my test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] maxingui commented on issue #2619: trino iceberg planning time so long
Posted by GitBox <gi...@apache.org>.
maxingui commented on issue #2619:
URL: https://github.com/apache/iceberg/issues/2619#issuecomment-846410643
thinks for you reply，my trino version is 356 too； the issue does not appear in hdfs+hadoop environment,but trino iceberg with huawei OBS accurs
log as below：
[22.log](https://github.com/apache/iceberg/files/6526494/22.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org