You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/12/06 03:12:04 UTC
[GitHub] [incubator-kyuubi] zwangsheng opened a new pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
zwangsheng opened a new pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
1. fix when using `-t` may cause build fail, because tag invalid reference format
2. add new option `-w` help put spark into kyuubi image by using `Dockerfile FROM` way
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (387590d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.09%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 387590d differs from pull request most recent head 765dc28. Consider uploading reports for the commit 765dc28 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.21% +0.09%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12192 +140
Branches 1478 1496 +18
============================================
+ Hits 7125 7219 +94
- Misses 4323 4359 +36
- Partials 604 614 +10
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `81.57% <0.00%> (-2.64%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...765dc28](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0b43985) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.12%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.24% +0.12%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12195 +143
Branches 1478 1496 +18
============================================
+ Hits 7125 7225 +100
- Misses 4323 4358 +35
- Partials 604 612 +8
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...0b43985](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762764468
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,23 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${WITHSPARKIMAGE}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
can we also do `cp -r "$SPARK_HOME/" "$KYUUBI_ROOT/spark-binary/` here ? so we can make `SPARK_HOME`
and `SPARK_BINARY` consisent in `dockerfile`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762686180
##########
File path: bin/docker-image-tool.sh
##########
@@ -118,11 +120,17 @@ function build {
# else use builtin spark
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
Review comment:
why we need delete it, if it is not existed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762718382
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
However, Dockerfile seems have not ability to user `if else` to chose copy or not. As I envision it, spark-binary will have no effect on the inherent Spark in the mirror when building with the Spark image as the base image.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a775006) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.28%`.
> The diff coverage is `n/a`.
> :exclamation: Current head a775006 differs from pull request most recent head 1486b1e. Consider uploading reports for the commit 1486b1e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.40% +0.28%
Complexity 172 172
============================================
Files 235 237 +2
Lines 12052 12168 +116
Branches 1478 1492 +14
============================================
+ Hits 7125 7228 +103
- Misses 4323 4330 +7
- Partials 604 610 +6
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `79.13% <0.00%> (-1.05%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetColumns.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0Q29sdW1ucy5zY2FsYQ==) | `98.52% <0.00%> (-0.03%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetSchemas.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0U2NoZW1hcy5zY2FsYQ==) | `90.00% <0.00%> (ø)` | |
| [...he/kyuubi/engine/spark/operation/GetCatalogs.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0Q2F0YWxvZ3Muc2NhbGE=) | `83.33% <0.00%> (ø)` | |
| ... and [7 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...1486b1e](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8f2420c) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.25%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 8f2420c differs from pull request most recent head 387590d. Consider uploading reports for the commit 387590d to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.37% +0.25%
Complexity 172 172
============================================
Files 235 238 +3
Lines 12052 12168 +116
Branches 1478 1491 +13
============================================
+ Hits 7125 7225 +100
- Misses 4323 4331 +8
- Partials 604 612 +8
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `81.57% <0.00%> (-2.64%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `79.13% <0.00%> (-1.05%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetColumns.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0Q29sdW1ucy5zY2FsYQ==) | `98.52% <0.00%> (-0.03%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetSchemas.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0U2NoZW1hcy5zY2FsYQ==) | `90.00% <0.00%> (ø)` | |
| ... and [10 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...387590d](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7bc5d94) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.11%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 7bc5d94 differs from pull request most recent head 0b43985. Consider uploading reports for the commit 0b43985 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.23% +0.11%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12195 +143
Branches 1478 1496 +18
============================================
+ Hits 7125 7224 +99
- Misses 4323 4358 +35
- Partials 604 613 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `81.57% <0.00%> (-2.64%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...0b43985](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762764870
##########
File path: bin/docker-image-tool.sh
##########
@@ -240,6 +244,7 @@ do
X) CROSS_BUILD=1;;
u) KYUUBI_UID=${OPTARG};;
s) SPARK_HOME=${OPTARG};;
+ w) WITHSPARKIMAGE=1;;
Review comment:
how about `SPARK_PROVIDED`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762689323
##########
File path: bin/docker-image-tool.sh
##########
@@ -118,11 +120,17 @@ function build {
# else use builtin spark
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
Review comment:
The logic is if this dir exists, it is considered that the spark was used last time. The spark version that was used last time is saved. Delete spark first and do copy or nothing according to `-w` parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986580526
Local test success, please retest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764529945
##########
File path: docker/Dockerfile
##########
@@ -24,15 +24,24 @@
# -t the target repo and tag name
# more options can be found with -h
-ARG BASE_IMAGE=8-jre-slim
-FROM openjdk:${BASE_IMAGE}
+ARG BASE_IMAGE=openjdk:8-jre-slim
+ARG spark_provided="spark_builtin"
+ARG spark_home="/opt/spark"
Review comment:
Nice catch, i will check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764525456
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,26 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ local BUILD_ARGS=(${BUILD_PARAMS})
+
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${SPARK_PROVIDED}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
I will try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764547190
##########
File path: bin/docker-image-tool.sh
##########
@@ -240,6 +247,7 @@ do
X) CROSS_BUILD=1;;
u) KYUUBI_UID=${OPTARG};;
s) SPARK_HOME=${OPTARG};;
+ d) SPARK_HOME_IN_DOCKER=${OPTARG};;
Review comment:
how about `S` ? `d` looks strange. And please update this option in `usage`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762766656
##########
File path: bin/docker-image-tool.sh
##########
@@ -240,6 +244,7 @@ do
X) CROSS_BUILD=1;;
u) KYUUBI_UID=${OPTARG};;
s) SPARK_HOME=${OPTARG};;
+ w) WITHSPARKIMAGE=1;;
Review comment:
Good advice, modify later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cfmcgrady commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r763598113
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
FYI: https://stackoverflow.com/questions/31528384/conditional-copy-add-in-dockerfile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r763690189
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
Nice advice, i will try use `as build`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764576737
##########
File path: bin/docker-image-tool.sh
##########
@@ -240,6 +247,7 @@ do
X) CROSS_BUILD=1;;
u) KYUUBI_UID=${OPTARG};;
s) SPARK_HOME=${OPTARG};;
+ d) SPARK_HOME_IN_DOCKER=${OPTARG};;
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8f2420c) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.25%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.37% +0.25%
Complexity 172 172
============================================
Files 235 238 +3
Lines 12052 12168 +116
Branches 1478 1491 +13
============================================
+ Hits 7125 7225 +100
- Misses 4323 4331 +8
- Partials 604 612 +8
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `81.57% <0.00%> (-2.64%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `79.13% <0.00%> (-1.05%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetColumns.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0Q29sdW1ucy5zY2FsYQ==) | `98.52% <0.00%> (-0.03%)` | :arrow_down: |
| [...che/kyuubi/engine/spark/operation/GetSchemas.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0U2NoZW1hcy5zY2FsYQ==) | `90.00% <0.00%> (ø)` | |
| ... and [10 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...8f2420c](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f98a7c2) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
> :exclamation: Current head f98a7c2 differs from pull request most recent head 1486b1e. Consider uploading reports for the commit 1486b1e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
=========================================
Coverage 59.11% 59.12%
Complexity 172 172
=========================================
Files 235 235
Lines 12052 12055 +3
Branches 1478 1478
=========================================
+ Hits 7125 7127 +2
Misses 4323 4323
- Partials 604 605 +1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.31% <0.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `65.00% <0.00%> (+1.36%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...1486b1e](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764496994
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,26 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ local BUILD_ARGS=(${BUILD_PARAMS})
+
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${SPARK_PROVIDED}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
A little confuse here. the `SPARK_HOME` means the directory inside docker container. But in general, the `SPARK_HOME` means the directory in real world.
So should the name be `SPARK_HOME_IN_DOCKER` ? and we don't need the flag `SPARK_PROVIDED` any more since we can detect `SPARK_HOME_IN_DOCKER` if existed.
Also we should document the `SPARK_HOME_IN_DOCKER`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cfmcgrady commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764497347
##########
File path: docker/Dockerfile
##########
@@ -24,15 +24,24 @@
# -t the target repo and tag name
# more options can be found with -h
-ARG BASE_IMAGE=8-jre-slim
-FROM openjdk:${BASE_IMAGE}
+ARG BASE_IMAGE=openjdk:8-jre-slim
+ARG spark_provided="spark_builtin"
+ARG spark_home="/opt/spark"
Review comment:
Does this ARG `spark_home` work?
From the docker official documents:
> An ARG instruction goes out of scope at the end of the build stage where it was defined. To use an arg in multiple stages, each stage must include the ARG instruction.
FYI: https://docs.docker.com/engine/reference/builder/#scope
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762832262
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,23 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${WITHSPARKIMAGE}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
so this `spark_home` is the directory in docker rather than in real world ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r763653621
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
ah, I like the idea using two options of `FROM` + `ONBUILD`, @zwangsheng can you try it ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762951485
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
cc @cfmcgrady do you have good idea ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7bc5d94) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.11%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.23% +0.11%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12195 +143
Branches 1478 1496 +18
============================================
+ Hits 7125 7224 +99
- Misses 4323 4358 +35
- Partials 604 613 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `81.57% <0.00%> (-2.64%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...7bc5d94](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r763846365
##########
File path: docker/Dockerfile
##########
@@ -24,15 +24,25 @@
# -t the target repo and tag name
# more options can be found with -h
-ARG BASE_IMAGE=8-jre-slim
-FROM openjdk:${BASE_IMAGE}
+ARG BASE_IMAGE=openjdk:8-jre-slim
+ARG spark_provided="spark_builtin"
-ARG kyuubi_uid=10009
+FROM ${BASE_IMAGE} as builder_spark_provided
+
+FROM ${BASE_IMAGE} as builder_spark_builtin
+
+ONBUILD ENV SPARK_BINARY /opt/spark
+ONBUILD RUN mkdir -p ${SPARK_BINARY}
+ONBUILD COPY spark-binary ${SPARK_BINARY}
Review comment:
do you need `SPARK_BINARY` ? seems we can just `COPY spark-binary ${spark_home}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (765dc28) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.10%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.21% +0.10%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12192 +140
Branches 1478 1496 +18
============================================
+ Hits 7125 7220 +95
- Misses 4323 4359 +36
- Partials 604 613 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...765dc28](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7b68a8b) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.11%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 7b68a8b differs from pull request most recent head 7bc5d94. Consider uploading reports for the commit 7bc5d94 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.22% +0.11%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12195 +143
Branches 1478 1496 +18
============================================
+ Hits 7125 7223 +98
- Misses 4323 4359 +36
- Partials 604 613 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...7bc5d94](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (765dc28) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.10%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 765dc28 differs from pull request most recent head 7b68a8b. Consider uploading reports for the commit 7b68a8b to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
============================================
+ Coverage 59.11% 59.21% +0.10%
Complexity 172 172
============================================
Files 235 241 +6
Lines 12052 12192 +140
Branches 1478 1496 +18
============================================
+ Hits 7125 7220 +95
- Misses 4323 4359 +36
- Partials 604 613 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...cala/org/apache/kyuubi/session/SessionHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25IYW5kbGUuc2NhbGE=) | `46.42% <0.00%> (-34.83%)` | :arrow_down: |
| [.../org/apache/kyuubi/operation/OperationHandle.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uSGFuZGxlLnNjYWxh) | `67.44% <0.00%> (-29.23%)` | :arrow_down: |
| [...kyuubi/engine/spark/session/SparkSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU2Vzc2lvbkltcGwuc2NhbGE=) | `94.73% <0.00%> (-5.27%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `82.50% <0.00%> (-4.17%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `84.76% <0.00%> (-2.86%)` | :arrow_down: |
| [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `78.78% <0.00%> (-1.40%)` | :arrow_down: |
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `75.86% <0.00%> (-0.81%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/operation/GetFunctions.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0RnVuY3Rpb25zLnNjYWxh) | `94.59% <0.00%> (-0.15%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.50% <0.00%> (-0.13%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/operation/GetTables.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vR2V0VGFibGVzLnNjYWxh) | `97.36% <0.00%> (-0.07%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...7b68a8b](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764547190
##########
File path: bin/docker-image-tool.sh
##########
@@ -240,6 +247,7 @@ do
X) CROSS_BUILD=1;;
u) KYUUBI_UID=${OPTARG};;
s) SPARK_HOME=${OPTARG};;
+ d) SPARK_HOME_IN_DOCKER=${OPTARG};;
Review comment:
how about `S` ? `d` looks strange. And please update this option in `usage` and PR description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng removed a comment on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng removed a comment on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986589069
OK to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762689349
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
if user specify the `${spark_home}`, seems we don't need `COPY` the `spark-binary`. So here are tow options:
* user specify the `${spark_home}`; we just do `ENV SPARK_HOME ${spark_home}` without mkdir and `COPY` the `spark-binary`
* user doesn't specify the `${spark_home}`; we should mkdir and `COPY` the `spark-binary` and `ENV SPARK_HOME SPARK_BINARY`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762771238
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,23 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${WITHSPARKIMAGE}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
Here, meams kyuubi use spark which from BASE_IMAGE. Spark already in Image, we can not move here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986499397
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1495](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f98a7c2) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/7f3f8af902fda3a8d1d422212bed3465a914df96?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f3f8af) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
> :exclamation: Current head f98a7c2 differs from pull request most recent head a775006. Consider uploading reports for the commit a775006 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1495 +/- ##
=========================================
Coverage 59.11% 59.12%
Complexity 172 172
=========================================
Files 235 235
Lines 12052 12055 +3
Branches 1478 1478
=========================================
+ Hits 7125 7127 +2
Misses 4323 4323
- Partials 604 605 +1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.31% <0.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `65.00% <0.00%> (+1.36%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7f3f8af...a775006](https://codecov.io/gh/apache/incubator-kyuubi/pull/1495?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-986589069
OK to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r762839191
##########
File path: bin/docker-image-tool.sh
##########
@@ -113,16 +115,23 @@ function build {
KYUUBI_ROOT="$CTX_DIR/base"
fi
- # cp spark for kyuubi as submit client
- # if user set -s(spark-provider), use if
- # else use builtin spark
+ # mkdir spark-binary to cache spark
+ # clean cache if spark-binary exists
if [[ ! -d "$KYUUBI_ROOT/spark-binary" ]]; then
mkdir "$KYUUBI_ROOT/spark-binary"
+ else
+ rm -rf "$KYUUBI_ROOT/spark-binary/*"
fi
- if [[ ! -d "$SPARK_HOME" ]]; then
- error "Cannot found dir $SPARK_HOME, you must configure SPARK_HOME correct."
+ # if with spark image means, kyuubi won't copy spark from local.
+ # In this case, we just pass SPARK_HOME without copy.
+ if [[ "${WITHSPARKIMAGE}" != "false" ]]; then
+ BUILD_ARGS+=(--build-arg spark_home=$SPARK_HOME)
Review comment:
We can think of it as having two `SPARK_HOME` on the outside and inside the container. SPARK_HOME inside the container is `/opt/spark` by default, which is the folder inside the container that you COPY to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-988647227
thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
ulysses-you closed pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-987764577
pass local test, cc @cfmcgrady @ulysses-you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#issuecomment-988658737
Thanks for help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] zwangsheng commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
zwangsheng commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r764493487
##########
File path: docker/Dockerfile
##########
@@ -24,15 +24,25 @@
# -t the target repo and tag name
# more options can be found with -h
-ARG BASE_IMAGE=8-jre-slim
-FROM openjdk:${BASE_IMAGE}
+ARG BASE_IMAGE=openjdk:8-jre-slim
+ARG spark_provided="spark_builtin"
-ARG kyuubi_uid=10009
+FROM ${BASE_IMAGE} as builder_spark_provided
+
+FROM ${BASE_IMAGE} as builder_spark_builtin
+
+ONBUILD ENV SPARK_BINARY /opt/spark
+ONBUILD RUN mkdir -p ${SPARK_BINARY}
+ONBUILD COPY spark-binary ${SPARK_BINARY}
Review comment:
OK, good idea. I will advance the `ARG spark_home`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cfmcgrady commented on a change in pull request #1495: [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on a change in pull request #1495:
URL: https://github.com/apache/incubator-kyuubi/pull/1495#discussion_r763597633
##########
File path: docker/Dockerfile
##########
@@ -42,12 +44,12 @@ RUN set -ex && \
apt-get update && \
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && \
useradd -u ${kyuubi_uid} -g root kyuubi && \
- mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_HOME} && \
+ mkdir -p ${KYUUBI_HOME} ${KYUUBI_LOG_DIR} ${KYUUBI_PID_DIR} ${KYUUBI_WORK_DIR_ROOT} ${SPARK_BINARY} && \
chmod ug+rw -R ${KYUUBI_HOME} && \
chmod a+rwx -R ${KYUUBI_WORK_DIR_ROOT} && \
rm -rf /var/cache/apt/*
-COPY spark-binary ${SPARK_HOME}
+COPY spark-binary ${SPARK_BINARY}
Review comment:
How about
```Dockerfile
ARG BASE_IMAGE=openjdk:8-jre-slim
ARG BUILD_ENV=copy
FROM ${BASE_IMAGE} as build_copy
ONBUILD RUN echo "copy user specify spark binary...."
ONBUILD COPY user_specify_spark_path /path_to_spark
ONBUILD ENV SPARK_HOME /path_to_spark
FROM ${BASE_IMAGE} as build_no_copy
ONBUILD RUN echo "I don't copy"
ONBUILD ARG BUILDIN_SPARK_HOME="/opt/spark"
ONBUILD ENV SPARK_HOME ${BUILDIN_SPARK_HOME}
FROM build_${BUILD_ENV}
```
- build w/ copy user specify spark binary package
```shell
docker build --build-arg BUILD_ENV=copy -t ${your_image_tag} -f docker/Dockerfile .
```
- build w/o copy spark binary package
```shell
docker build --build-arg BUILD_ENV=no_copy --build-arg BUILDIN_SPARK_HOME="/path/to/spark_home" -t ${your_image_tag} -f docker/Dockerfile .
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org