You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 08:11:58 UTC
[PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]
zuston opened a new pull request, #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577
### What changes were proposed in this pull request?
Provide the uniffle server deploy guide for those machines without hadoop env.
### Why are the changes needed?
Leveraging from the #1379 and #1370 , we could setup uniffle shuffle-server without hadoop env.
This will simplify the quick start process.
Fix: #1576
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1996945555
## Test Results
2 340 files ±0 2 340 suites ±0 4h 34m 6s :stopwatch: + 2m 10s
908 tests ±0 907 :white_check_mark: ±0 1 :zzz: ±0 0 :x: ±0
10 541 runs ±0 10 527 :white_check_mark: ±0 14 :zzz: ±0 0 :x: ±0
Results for commit 451cb9ad. ± Comparison against base commit f6569b04.
[test-results]:data:application/gzip;base64,H4sIANS98mUC/03MTQ7CIBCG4as0rF0w7QDiZQxQSIj9MRRWxrs7baB2+T0D74eFOPmNPbp+QH7r2FZivu6xJJPjupCARJREdM/7C83vbT234txB6k+v+N5/nRBMnAj4CT6lNVVJZdmbwAVCnS1K1qvTahUbXKrHvkbdOs8x02AowFltxqDl4EZlhRC9B9BeGlABvbB00V6z7w/iyFPgEgEAAA==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-2003002199
> > > I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
> >
> >
> > It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.
>
> Not this. I will worry about this way. They don't modify any config options but they upgrade the jar directly, the server can't start.
Make sense. Let me revert this config default val
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-2002766168
> > I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
>
> It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.
Not this. I will worry about this way. They don't modify any config options but they upgrade the jar directly, the server can't start.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1999031284
I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on code in PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#discussion_r1524623000
##########
common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java:
##########
@@ -198,7 +198,7 @@ public class RssClientConf {
public static final ConfigOption<Boolean> RSS_CLIENT_REMOTE_STORAGE_USE_LOCAL_CONF_ENABLED =
ConfigOptions.key("rss.client.remote.storage.useLocalConfAsDefault")
.booleanType()
- .defaultValue(false)
+ .defaultValue(true)
Review Comment:
Why do we need to change the default value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on code in PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#discussion_r1524772016
##########
common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java:
##########
@@ -198,7 +198,7 @@ public class RssClientConf {
public static final ConfigOption<Boolean> RSS_CLIENT_REMOTE_STORAGE_USE_LOCAL_CONF_ENABLED =
ConfigOptions.key("rss.client.remote.storage.useLocalConfAsDefault")
.booleanType()
- .defaultValue(false)
+ .defaultValue(true)
Review Comment:
I hope so, that will make shuffle server without hadoop env by default
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1996828615
## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 54.01%. Comparing base [(`f6569b0`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/f6569b046d2a1c0ec95fb390fe36f17c43aec052?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`dc9b3d2`)](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #1577 +/- ##
============================================
+ Coverage 53.99% 54.01% +0.01%
+ Complexity 2862 2293 -569
============================================
Files 438 360 -78
Lines 24819 16063 -8756
Branches 2111 1465 -646
============================================
- Hits 13402 8677 -4725
+ Misses 10578 6884 -3694
+ Partials 839 502 -337
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1999122187
> I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org