You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/07/11 23:11:40 UTC
[GitHub] [iceberg] jhsenjaliya opened a new pull request #2804: [Issue-2741] use RetryingMetaStoreClient
jhsenjaliya opened a new pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804
uses `RetryingMetaStoreClient` to handle the connection failures automatically as per the hive internal logic. instead of trying to use `hiveMetastoreClient` directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
szehon-ho commented on pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804#issuecomment-964780514
This has been implemented
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho closed pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
szehon-ho closed pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jhsenjaliya commented on pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
jhsenjaliya commented on pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804#issuecomment-915599021
Hi @szehon-ho , yes, since the clientPool is designed to take care of reties, and RetryingHiveMetastore client does not really allow individual object creation, its hard to replace it without significant code replacement which i feel now unnecessary, I explained the issue in discussion at #2741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on a change in pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
szehon-ho commented on a change in pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804#discussion_r703813734
##########
File path: hive-metastore/src/main/java/org/apache/iceberg/hive/HiveClientPool.java
##########
@@ -47,10 +42,10 @@ public HiveClientPool(int poolSize, Configuration conf) {
}
@Override
- protected HiveMetaStoreClient newClient() {
+ protected IMetaStoreClient newClient() {
try {
try {
- return CLIENT_CTOR.newInstance(hiveConf);
+ return RetryingMetaStoreClient.getProxy(hiveConf, null, HiveMetaStoreClient.class.getName());
Review comment:
Maybe I'm mistaken, but this is the CTOR that is @VisibleForTesting?
Can we use:
```
public static IMetaStoreClient getProxy(
Configuration hiveConf, boolean allowEmbedded)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
szehon-ho commented on pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804#issuecomment-915403020
@jhsenjaliya do you plan to continue working on it, or I can also help? Probably we need to get rid of the extra retry at ClientPool layer and refactor more codes to get it to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on pull request #2804: [Issue-2741] use RetryingMetaStoreClient
Posted by GitBox <gi...@apache.org>.
szehon-ho commented on pull request #2804:
URL: https://github.com/apache/iceberg/pull/2804#issuecomment-914669941
Actually on second look, it looks like ClientPoolImpl already does the retry logic (though no configurable retries), so if we replace it it will need to be a bigger replacement..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org