You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/10/10 15:58:55 UTC
[GitHub] [iceberg] sumeetgajjar opened a new pull request, #5952: Spark: Perform nullability check on metastore and spark instance before invoking stop
sumeetgajjar opened a new pull request, #5952:
URL: https://github.com/apache/iceberg/pull/5952
In this PR, we fix `SparkTestBase#stopMetastoreAndSpark`.
When `SparkTestBase#startMetastoreAndSpark` throws an exception while starting embedded HMS or while creating `SparkSession`, the respective `metastore` or `spark` instance would be null. Thus when `stop` method is invoked on either of those, an NPE is thrown simply adding noise to the logs.
Adding a nullability check before invoking stop to prevent NPE noise in the test logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] sumeetgajjar commented on pull request #5952: Spark: Perform nullability check on metastore and spark instances before invoking stop
Posted by GitBox <gi...@apache.org>.
sumeetgajjar commented on PR #5952:
URL: https://github.com/apache/iceberg/pull/5952#issuecomment-1376416677
Hi @pvary @nastra,
Thanks a ton for the review, can we please merge this PR since it is already approved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #5952: Spark: Perform nullability check on metastore and spark instances before invoking stop
Posted by GitBox <gi...@apache.org>.
pvary commented on PR #5952:
URL: https://github.com/apache/iceberg/pull/5952#issuecomment-1376785336
@sumeetgajjar: Sorry for the delay! Thanks for the PR
Merge to the master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] sumeetgajjar commented on pull request #5952: Spark: Perform nullability check on metastore and spark instance before invoking stop
Posted by GitBox <gi...@apache.org>.
sumeetgajjar commented on PR #5952:
URL: https://github.com/apache/iceberg/pull/5952#issuecomment-1273528765
Hi @Fokko @nastra @pvary,
Can you please review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] sumeetgajjar commented on pull request #5952: Spark: Perform nullability check on metastore and spark instances before invoking stop
Posted by GitBox <gi...@apache.org>.
sumeetgajjar commented on PR #5952:
URL: https://github.com/apache/iceberg/pull/5952#issuecomment-1276723402
The check failure is due to unrelated reasons and not because of this change.
```
org.apache.iceberg.spark.TestFileRewriteCoordinator > testBinPackRewrite[catalogName = testhive, implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=hive, default-namespace=default}] FAILED
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
at app//org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:113)
at app//org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:225)
at app//org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:137)
at app//org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:127)
at app//org.apache.spark.sql.internal.SharedState.globalTempViewManager$lzycompute(SharedState.scala:157)
at app//org.apache.spark.sql.internal.SharedState.globalTempViewManager(SharedState.scala:155)
at app//org.apache.spark.sql.hive.HiveSessionStateBuilder.$anonfun$catalog$2(HiveSessionStateBuilder.scala:60)
```
Pushing an empty commit to retrigger the checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary merged pull request #5952: Spark: Perform nullability check on metastore and spark instances before invoking stop
Posted by GitBox <gi...@apache.org>.
pvary merged PR #5952:
URL: https://github.com/apache/iceberg/pull/5952
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org