You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/11 11:20:35 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
AngersZhuuuu opened a new pull request #34241:
URL: https://github.com/apache/spark/pull/34241
### What changes were proposed in this pull request?
Currently we collect hive client calls metrics from `withHiveState`, it's not correct that in one call `withHiveState`, there may be many hive client calls. we need to split such behaviors.
### Why are the changes needed?
Make hive client calls metrics more accrate
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existed UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949243242
**[Test build #144525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948440226
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949258708
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943460103
**[Test build #144266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144266/testReport)** for PR 34241 at commit [`3db6f60`](https://github.com/apache/spark/commit/3db6f607644e0ac09ed3292ff432f700bc950853).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940965182
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144145/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940023160
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940019808
**[Test build #144083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144083/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940964033
**[Test build #144145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144145/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943303992
**[Test build #144258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144258/testReport)** for PR 34241 at commit [`1042b03`](https://github.com/apache/spark/commit/1042b035d5a393386c8c641e48f3fffec16310c4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943314850
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48737/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952814471
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144653/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] codecov-commenter commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952799438
# [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#34241](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1099fd3) into [master](https://codecov.io/gh/apache/spark/commit/1020c6d8e51d3dc135cc012645950e9a8eea8376?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1020c6d) will **decrease** coverage by `30.70%`.
> The diff coverage is `73.42%`.
> :exclamation: Current head 1099fd3 differs from pull request most recent head 80ffe42. Consider uploading reports for the commit 80ffe42 to get more accurate results
[](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #34241 +/- ##
===========================================
- Coverage 90.19% 59.48% -30.71%
===========================================
Files 288 197 -91
Lines 61391 38377 -23014
Branches 9021 6853 -2168
===========================================
- Hits 55372 22829 -32543
- Misses 4670 14385 +9715
+ Partials 1349 1163 -186
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `59.48% <73.42%> (-30.69%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [python/pyspark/context.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvY29udGV4dC5weQ==) | `44.67% <ø> (-40.87%)` | :arrow_down: |
| [python/pyspark/ml/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvY29tbW9uLnB5) | `51.42% <ø> (-24.29%)` | :arrow_down: |
| [python/pyspark/ml/fpm.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvZnBtLnB5) | `47.05% <ø> (-36.98%)` | :arrow_down: |
| [python/pyspark/mllib/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWxsaWIvY29tbW9uLnB5) | `54.87% <ø> (-34.15%)` | :arrow_down: |
| [python/pyspark/pandas/\_typing.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL190eXBpbmcucHk=) | `71.42% <0.00%> (ø)` | |
| [python/pyspark/pandas/extensions.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL2V4dGVuc2lvbnMucHk=) | `88.46% <0.00%> (ø)` | |
| [python/pyspark/pandas/ml.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL21sLnB5) | `86.36% <0.00%> (-4.55%)` | :arrow_down: |
| [python/pyspark/pandas/plot/plotly.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3Bsb3QvcGxvdGx5LnB5) | `90.90% <0.00%> (ø)` | |
| [python/pyspark/pandas/spark/accessors.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3NwYXJrL2FjY2Vzc29ycy5weQ==) | `90.44% <0.00%> (-2.25%)` | :arrow_down: |
| [python/pyspark/pandas/strings.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3N0cmluZ3MucHk=) | `97.92% <0.00%> (ø)` | |
| ... and [240 more](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [b1f1575...80ffe42](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952807015
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49123/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949283729
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733407580
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -398,7 +402,9 @@ private[hive] class HiveClientImpl(
client.getDatabasesByPattern(pattern).asScala.toSeq
}
- private def getRawTableOption(dbName: String, tableName: String): Option[HiveTable] = {
+ private def getRawTableOption(
+ dbName: String,
+ tableName: String): Option[HiveTable] = withHiveState {
Review comment:
IIUC `getRawTableOption` is always called within `withHiveState`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940023160
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940066999
**[Test build #144083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144083/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943508542
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48747/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940067463
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144083/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-939938603
**[Test build #144081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144081/testReport)** for PR 34241 at commit [`8c46c03`](https://github.com/apache/spark/commit/8c46c034ed5f3fd4b1b00f23e8b0eb77511e07e4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940667772
**[Test build #144116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144116/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940879029
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940856466
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48613/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943502290
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48745/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952703667
**[Test build #144653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144653/testReport)** for PR 34241 at commit [`80ffe42`](https://github.com/apache/spark/commit/80ffe42f3127bb0393579cc6dda0cd2093fcc1a2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952751690
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49123/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949843040
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940011223
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144081/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-939976798
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940023116
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943311624
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144258/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943463698
**[Test build #144268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144268/testReport)** for PR 34241 at commit [`ebc1f81`](https://github.com/apache/spark/commit/ebc1f816e223e4c537ececd90e6a341ac32905bc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952813105
**[Test build #144653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144653/testReport)** for PR 34241 at commit [`80ffe42`](https://github.com/apache/spark/commit/80ffe42f3127bb0393579cc6dda0cd2093fcc1a2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] codecov-commenter edited a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952799438
# [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#34241](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1099fd3) into [master](https://codecov.io/gh/apache/spark/commit/1020c6d8e51d3dc135cc012645950e9a8eea8376?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1020c6d) will **decrease** coverage by `16.54%`.
> The diff coverage is `78.95%`.
> :exclamation: Current head 1099fd3 differs from pull request most recent head 80ffe42. Consider uploading reports for the commit 80ffe42 to get more accurate results
[](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #34241 +/- ##
===========================================
- Coverage 90.19% 73.64% -16.55%
===========================================
Files 288 206 -82
Lines 61391 48039 -13352
Branches 9021 7693 -1328
===========================================
- Hits 55372 35380 -19992
- Misses 4670 11657 +6987
+ Partials 1349 1002 -347
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `73.62% <78.95%> (-16.54%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [python/pyspark/context.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvY29udGV4dC5weQ==) | `45.43% <ø> (-40.11%)` | :arrow_down: |
| [python/pyspark/ml/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvY29tbW9uLnB5) | `51.42% <ø> (-24.29%)` | :arrow_down: |
| [python/pyspark/ml/fpm.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvZnBtLnB5) | `47.05% <ø> (-36.98%)` | :arrow_down: |
| [python/pyspark/mllib/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWxsaWIvY29tbW9uLnB5) | `54.87% <ø> (-34.15%)` | :arrow_down: |
| [python/pyspark/pandas/\_typing.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL190eXBpbmcucHk=) | `71.42% <0.00%> (ø)` | |
| [python/pyspark/pandas/extensions.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL2V4dGVuc2lvbnMucHk=) | `88.46% <0.00%> (ø)` | |
| [python/pyspark/pandas/ml.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL21sLnB5) | `90.90% <0.00%> (ø)` | |
| [python/pyspark/pandas/plot/plotly.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3Bsb3QvcGxvdGx5LnB5) | `90.90% <0.00%> (ø)` | |
| [python/pyspark/pandas/spark/accessors.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3NwYXJrL2FjY2Vzc29ycy5weQ==) | `92.69% <0.00%> (ø)` | |
| [python/pyspark/pandas/strings.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3N0cmluZ3MucHk=) | `97.92% <0.00%> (ø)` | |
| ... and [211 more](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [b1f1575...80ffe42](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943314850
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48737/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943311401
**[Test build #144258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144258/testReport)** for PR 34241 at commit [`1042b03`](https://github.com/apache/spark/commit/1042b035d5a393386c8c641e48f3fffec16310c4).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733406754
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -372,13 +374,15 @@ private[hive] class HiveClientImpl(
CatalogUtils.URIToString(database.locationUri),
(props -- Seq(PROP_OWNER)).asJava)
props.get(PROP_OWNER).orElse(userName).foreach { ownerName =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
shim.setDatabaseOwnerName(hiveDb, ownerName)
}
hiveDb
}
override def getDatabase(dbName: String): CatalogDatabase = withHiveState {
Option(client.getDatabase(dbName)).map { d =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
why not use `withHiveState(2)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948561622
**[Test build #144505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144505/testReport)** for PR 34241 at commit [`dba5e06`](https://github.com/apache/spark/commit/dba5e0633e309d894b9ed901ae77d5c50f513fe6).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733409636
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -963,36 +975,45 @@ private[hive] class HiveClientImpl(
clientLoader.createClient().asInstanceOf[HiveClientImpl]
}
- def reset(): Unit = withHiveState {
+ def reset(): Unit = withHiveState(2) {
val allTables = client.getAllTables("default")
- val (mvs, others) = allTables.asScala.map(t => client.getTable("default", t))
- .partition(_.getTableType.toString.equals("MATERIALIZED_VIEW"))
+ val (mvs, others) = allTables.asScala.map(t => {
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
+ client.getTable("default", t)
+ }).partition(_.getTableType.toString.equals("MATERIALIZED_VIEW"))
// Remove materialized view first, otherwise caused a violation of foreign key constraint.
mvs.foreach { table =>
val t = table.getTableName
logDebug(s"Deleting materialized view $t")
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
It's really messy to see this code in random places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948578409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940895574
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940677381
@HyukjinKwon There is still some GA oom issue, https://github.com/apache/spark/pull/34241/checks?check_run_id=3859514406 And I have saw @srowen have increate the xmx value to 6G, seems not work? need to restart GA or any other operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-941063891
ping @cloud-fan Since some API call hive client with a uncertain times so add a split function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940867019
**[Test build #144145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144145/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943458807
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48741/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948485880
**[Test build #144505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144505/testReport)** for PR 34241 at commit [`dba5e06`](https://github.com/apache/spark/commit/dba5e0633e309d894b9ed901ae77d5c50f513fe6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948392519
**[Test build #144498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144498/testReport)** for PR 34241 at commit [`517fef1`](https://github.com/apache/spark/commit/517fef1edec6d08bc89ddfe376abc15f5132882f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949273919
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949752868
**[Test build #144541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144541/testReport)** for PR 34241 at commit [`3af7e14`](https://github.com/apache/spark/commit/3af7e148c3b97e408531228bbc89e83a9938a5e9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949849022
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144541/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu closed pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu closed pull request #34241:
URL: https://github.com/apache/spark/pull/34241
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943461608
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943314826
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48737/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-939938603
**[Test build #144081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144081/testReport)** for PR 34241 at commit [`8c46c03`](https://github.com/apache/spark/commit/8c46c034ed5f3fd4b1b00f23e8b0eb77511e07e4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940615457
**[Test build #144116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144116/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940965182
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144145/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940668764
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144116/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940857481
**[Test build #144136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144136/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949284457
**[Test build #144525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949285303
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144525/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949847762
**[Test build #144541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144541/testReport)** for PR 34241 at commit [`3af7e14`](https://github.com/apache/spark/commit/3af7e148c3b97e408531228bbc89e83a9938a5e9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949849022
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144541/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r735702215
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -634,6 +636,9 @@ private[hive] class HiveClientImpl(
val matchingParts =
specs.flatMap { s =>
assert(s.values.forall(_.nonEmpty), s"partition spec '$s' is invalid")
+ // Since Spark can't know the actual num of matching partitions,
+ // here Spark update hive client calls one by one.
Review comment:
I think it's clearer to say:
```
// For each partition spec, we need to do a Hive client call
// to get the matching partitions/
```
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -634,6 +636,9 @@ private[hive] class HiveClientImpl(
val matchingParts =
specs.flatMap { s =>
assert(s.values.forall(_.nonEmpty), s"partition spec '$s' is invalid")
+ // Since Spark can't know the actual num of matching partitions,
+ // here Spark update hive client calls one by one.
Review comment:
I think it's clearer to say:
```
// For each partition spec, we need to do a Hive client call
// to get the matching partitions.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] codecov-commenter edited a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952799438
# [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#34241](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1099fd3) into [master](https://codecov.io/gh/apache/spark/commit/1020c6d8e51d3dc135cc012645950e9a8eea8376?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1020c6d) will **decrease** coverage by `8.11%`.
> The diff coverage is `86.63%`.
> :exclamation: Current head 1099fd3 differs from pull request most recent head 80ffe42. Consider uploading reports for the commit 80ffe42 to get more accurate results
[](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #34241 +/- ##
==========================================
- Coverage 90.19% 82.07% -8.12%
==========================================
Files 288 249 -39
Lines 61391 56384 -5007
Branches 9021 8527 -494
==========================================
- Hits 55372 46279 -9093
- Misses 4670 8913 +4243
+ Partials 1349 1192 -157
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `82.05% <86.63%> (-8.12%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [python/pyspark/context.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvY29udGV4dC5weQ==) | `57.86% <ø> (-27.67%)` | :arrow_down: |
| [python/pyspark/ml/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvY29tbW9uLnB5) | `51.42% <ø> (-24.29%)` | :arrow_down: |
| [python/pyspark/ml/fpm.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWwvZnBtLnB5) | `47.05% <ø> (-36.98%)` | :arrow_down: |
| [python/pyspark/mllib/classification.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWxsaWIvY2xhc3NpZmljYXRpb24ucHk=) | `91.87% <ø> (ø)` | |
| [python/pyspark/mllib/common.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvbWxsaWIvY29tbW9uLnB5) | `89.02% <ø> (ø)` | |
| [python/pyspark/pandas/\_typing.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL190eXBpbmcucHk=) | `71.42% <0.00%> (ø)` | |
| [python/pyspark/pandas/extensions.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL2V4dGVuc2lvbnMucHk=) | `88.46% <0.00%> (ø)` | |
| [python/pyspark/pandas/ml.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL21sLnB5) | `90.90% <0.00%> (ø)` | |
| [python/pyspark/pandas/plot/plotly.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3Bsb3QvcGxvdGx5LnB5) | `90.90% <0.00%> (ø)` | |
| [python/pyspark/pandas/spark/accessors.py](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHl0aG9uL3B5c3BhcmsvcGFuZGFzL3NwYXJrL2FjY2Vzc29ycy5weQ==) | `92.69% <0.00%> (ø)` | |
| ... and [145 more](https://codecov.io/gh/apache/spark/pull/34241/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [b1f1575...80ffe42](https://codecov.io/gh/apache/spark/pull/34241?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952703667
**[Test build #144653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144653/testReport)** for PR 34241 at commit [`80ffe42`](https://github.com/apache/spark/commit/80ffe42f3127bb0393579cc6dda0cd2093fcc1a2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940059902
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48561/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733435747
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -372,13 +374,15 @@ private[hive] class HiveClientImpl(
CatalogUtils.URIToString(database.locationUri),
(props -- Seq(PROP_OWNER)).asJava)
props.get(PROP_OWNER).orElse(userName).foreach { ownerName =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
shim.setDatabaseOwnerName(hiveDb, ownerName)
}
hiveDb
}
override def getDatabase(dbName: String): CatalogDatabase = withHiveState {
Option(client.getDatabase(dbName)).map { d =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
Here is no rpC call, should remove this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940019808
**[Test build #144083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144083/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943392816
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48741/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733435006
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -372,13 +374,15 @@ private[hive] class HiveClientImpl(
CatalogUtils.URIToString(database.locationUri),
(props -- Seq(PROP_OWNER)).asJava)
props.get(PROP_OWNER).orElse(userName).foreach { ownerName =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
Make a mistake here, remove this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733406087
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -372,13 +374,15 @@ private[hive] class HiveClientImpl(
CatalogUtils.URIToString(database.locationUri),
(props -- Seq(PROP_OWNER)).asJava)
props.get(PROP_OWNER).orElse(userName).foreach { ownerName =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
how does it increase the hive client calls? `shim.setDatabaseOwnerName` does not send RPC request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949796249
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948485085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949843040
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952814471
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144653/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948560996
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48977/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940864105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943458807
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940668764
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144116/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943311624
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144258/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940951696
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940095367
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48561/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940011223
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144081/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940067463
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144083/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940939016
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943348304
**[Test build #144262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144262/testReport)** for PR 34241 at commit [`b8cde6d`](https://github.com/apache/spark/commit/b8cde6d4b3e19681b9f30353bd16ca0379a59ef9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r733433538
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -372,13 +374,15 @@ private[hive] class HiveClientImpl(
CatalogUtils.URIToString(database.locationUri),
(props -- Seq(PROP_OWNER)).asJava)
props.get(PROP_OWNER).orElse(userName).foreach { ownerName =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
shim.setDatabaseOwnerName(hiveDb, ownerName)
}
hiveDb
}
override def getDatabase(dbName: String): CatalogDatabase = withHiveState {
Option(client.getDatabase(dbName)).map { d =>
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
> why not use `withHiveState(2)`?
```
public Database getDatabase(String dbName) throws HiveException {
try {
return this.getMSC().getDatabase(dbName);
} catch (NoSuchObjectException var3) {
return null;
} catch (Exception var4) {
throw new HiveException(var4);
}
}
```
If return null, won't have the second call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948459864
**[Test build #144498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144498/testReport)** for PR 34241 at commit [`517fef1`](https://github.com/apache/spark/commit/517fef1edec6d08bc89ddfe376abc15f5132882f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948525412
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48977/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948485880
**[Test build #144505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144505/testReport)** for PR 34241 at commit [`dba5e06`](https://github.com/apache/spark/commit/dba5e0633e309d894b9ed901ae77d5c50f513fe6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948476254
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948392519
**[Test build #144498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144498/testReport)** for PR 34241 at commit [`517fef1`](https://github.com/apache/spark/commit/517fef1edec6d08bc89ddfe376abc15f5132882f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948578409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-945391900
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949828340
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49013/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949283729
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949752868
**[Test build #144541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144541/testReport)** for PR 34241 at commit [`3af7e14`](https://github.com/apache/spark/commit/3af7e148c3b97e408531228bbc89e83a9938a5e9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940634833
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48592/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940112366
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48561/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940112366
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48561/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940615457
**[Test build #144116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144116/testReport)** for PR 34241 at commit [`353429d`](https://github.com/apache/spark/commit/353429d040b7a8f44cf0b5d0c8069a85e1765cf0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-939984217
**[Test build #144081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144081/testReport)** for PR 34241 at commit [`8c46c03`](https://github.com/apache/spark/commit/8c46c034ed5f3fd4b1b00f23e8b0eb77511e07e4).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940599518
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940867019
**[Test build #144145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144145/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940769615
**[Test build #144136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144136/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943457961
**[Test build #144262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144262/testReport)** for PR 34241 at commit [`b8cde6d`](https://github.com/apache/spark/commit/b8cde6d4b3e19681b9f30353bd16ca0379a59ef9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943348304
**[Test build #144262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144262/testReport)** for PR 34241 at commit [`b8cde6d`](https://github.com/apache/spark/commit/b8cde6d4b3e19681b9f30353bd16ca0379a59ef9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940860949
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943303992
**[Test build #144258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144258/testReport)** for PR 34241 at commit [`1042b03`](https://github.com/apache/spark/commit/1042b035d5a393386c8c641e48f3fffec16310c4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940864105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949243242
**[Test build #144525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144525/testReport)** for PR 34241 at commit [`cabe392`](https://github.com/apache/spark/commit/cabe392635a9113ca58d4fee4929191f3cd19667).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952807015
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49123/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-948485085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-952788425
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49123/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r735702643
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -662,6 +667,10 @@ private[hive] class HiveClientImpl(
|======================
""".stripMargin)
throw e
+ } finally {
+ // Since Spark can't know the actual num of matching partitions,
+ // here Spark update hive client calls one by one.
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
Review comment:
Why not do it right before `shim.dropPartition`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r735703034
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -671,17 +680,23 @@ private[hive] class HiveClientImpl(
db: String,
table: String,
specs: Seq[TablePartitionSpec],
- newSpecs: Seq[TablePartitionSpec]): Unit = withHiveState {
+ newSpecs: Seq[TablePartitionSpec]): Unit = withHiveState(0) {
require(specs.size == newSpecs.size, "number of old and new partition specs differ")
val catalogTable = getTable(db, table)
val hiveTable = toHiveTable(catalogTable, Some(userName))
specs.zip(newSpecs).foreach { case (oldSpec, newSpec) =>
+ // Since Spark may throw exception when call client.getPartition(),
+ // Spark need to update hive client calls one by one.
Review comment:
ditto, let's refine the comment
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -671,17 +680,23 @@ private[hive] class HiveClientImpl(
db: String,
table: String,
specs: Seq[TablePartitionSpec],
- newSpecs: Seq[TablePartitionSpec]): Unit = withHiveState {
+ newSpecs: Seq[TablePartitionSpec]): Unit = withHiveState(0) {
require(specs.size == newSpecs.size, "number of old and new partition specs differ")
val catalogTable = getTable(db, table)
val hiveTable = toHiveTable(catalogTable, Some(userName))
specs.zip(newSpecs).foreach { case (oldSpec, newSpec) =>
+ // Since Spark may throw exception when call client.getPartition(),
+ // Spark need to update hive client calls one by one.
+ HiveCatalogMetrics.incrementHiveClientCalls(1)
if (client.getPartition(hiveTable, newSpec.asJava, false) != null) {
throw new PartitionAlreadyExistsException(db, table, newSpec)
}
val hivePart = getPartitionOption(catalogTable, oldSpec)
.map { p => toHivePartition(p.copy(spec = newSpec), hiveTable) }
.getOrElse { throw new NoSuchPartitionException(db, table, oldSpec) }
+ // Since Spark may throw exception when call client.getPartition(),
+ // Spark need to update hive client calls one by one.
Review comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34241:
URL: https://github.com/apache/spark/pull/34241#discussion_r735703871
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -786,6 +800,9 @@ private[hive] class HiveClientImpl(
} catch {
case _: UnsupportedOperationException =>
// Fallback to filter logic if getTablesByType not supported.
+ // Since Spark can't know if it will throw exception when call shim.getTablesByType(),
+ // here Spark update hive client calls once.
Review comment:
```suggestion
// here we need to make an extra Hive client call.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Correct the hive client calls‘s metrics in HiveClientImpl
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-949285303
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144525/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940769615
**[Test build #144136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144136/testReport)** for PR 34241 at commit [`a50fdd0`](https://github.com/apache/spark/commit/a50fdd0468e6fb606844c22a54d1ad8a1891b3e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940637932
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48592/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940618798
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48592/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940951696
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940820448
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48613/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [WIP][SPARK-36975][SQL] Split hive client call collection from withHiveState()
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-940637932
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48592/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943459428
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144262/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34241: [SPARK-36975][SQL] Split hive client call collection from withHiveState() and make metrics correct
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34241:
URL: https://github.com/apache/spark/pull/34241#issuecomment-943450378
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48741/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org