You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/17 07:50:04 UTC
[GitHub] [iceberg] vanliu-tx opened a new issue #3907: HiveCatalog listTables takes minutes if there are thousands tables in namespace
vanliu-tx opened a new issue #3907:
URL: https://github.com/apache/iceberg/issues/3907
when we update iceberg 0.9.0 to 0.12.1, we found an issue that HiveCatalog#listTables(Namespace) takes minutes in 0.12.1 in our production env, compared to ~0.1s in 0.9.0.
In our production, this method takes 28s for one namespace contains 517 tables, 75s for one namespace contains 1301 tables, more than half an hour for one namespace with 27000+ tables.
In this issue(apache#1835), client.getTableObjectsByName was added to filter iceberg tables, but this method is really slow if there are thousands tables. In our product env, we separate different table types in different namespace, so there is no need to filter iceberg tables.
```
List<Table> tableObjects = clients.run(client -> client.getTableObjectsByName(database, tableNames));
```
We should add a flag(hive properties) to indicate whether we need to filter iceberg tables in namespace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vanliu-tx closed issue #3907: HiveCatalog listTables takes minutes if there are thousands tables in namespace
Posted by GitBox <gi...@apache.org>.
vanliu-tx closed issue #3907:
URL: https://github.com/apache/iceberg/issues/3907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vanliu-tx closed issue #3907: HiveCatalog listTables takes minutes if there are thousands tables in namespace
Posted by GitBox <gi...@apache.org>.
vanliu-tx closed issue #3907:
URL: https://github.com/apache/iceberg/issues/3907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vanliu-tx commented on issue #3907: HiveCatalog listTables takes minutes if there are thousands tables in namespace
Posted by GitBox <gi...@apache.org>.
vanliu-tx commented on issue #3907:
URL: https://github.com/apache/iceberg/issues/3907#issuecomment-1020721982
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vanliu-tx commented on issue #3907: HiveCatalog listTables takes minutes if there are thousands tables in namespace
Posted by GitBox <gi...@apache.org>.
vanliu-tx commented on issue #3907:
URL: https://github.com/apache/iceberg/issues/3907#issuecomment-1020721982
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org