You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 08:11:58 UTC

[PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]

zuston opened a new pull request, #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577

   
   ### What changes were proposed in this pull request?
   
   Provide the uniffle server deploy guide for those machines without hadoop env.
   
   ### Why are the changes needed?
   
   Leveraging from the #1379 and #1370 , we could setup uniffle shuffle-server without hadoop env.
   This will simplify the quick start process.
   
   Fix: #1576
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Existing tests
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1996945555

   ## Test Results
    2 340 files  ±0   2 340 suites  ±0   4h 34m 6s :stopwatch: + 2m 10s
      908 tests ±0     907 :white_check_mark: ±0   1 :zzz: ±0  0 :x: ±0 
   10 541 runs  ±0  10 527 :white_check_mark: ±0  14 :zzz: ±0  0 :x: ±0 
   
   Results for commit 451cb9ad. ± Comparison against base commit f6569b04.
   
   [test-results]:data:application/gzip;base64,H4sIANS98mUC/03MTQ7CIBCG4as0rF0w7QDiZQxQSIj9MRRWxrs7baB2+T0D74eFOPmNPbp+QH7r2FZivu6xJJPjupCARJREdM/7C83vbT234txB6k+v+N5/nRBMnAj4CT6lNVVJZdmbwAVCnS1K1qvTahUbXKrHvkbdOs8x02AowFltxqDl4EZlhRC9B9BeGlABvbB00V6z7w/iyFPgEgEAAA==
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-2003002199

   > > > I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
   > > 
   > > 
   > > It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.
   > 
   > Not this. I will worry about this way. They don't modify any config options but they upgrade the jar directly, the server can't start.
   
   Make sense. Let me revert this config default val


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-2002766168

   > > I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
   > 
   > It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.
   
   Not this. I will worry about this way. They don't modify any config options but they upgrade the jar directly, the server can't start.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1999031284

   I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on code in PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#discussion_r1524623000


##########
common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java:
##########
@@ -198,7 +198,7 @@ public class RssClientConf {
   public static final ConfigOption<Boolean> RSS_CLIENT_REMOTE_STORAGE_USE_LOCAL_CONF_ENABLED =
       ConfigOptions.key("rss.client.remote.storage.useLocalConfAsDefault")
           .booleanType()
-          .defaultValue(false)
+          .defaultValue(true)

Review Comment:
   Why do we need to change the default value?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on code in PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#discussion_r1524772016


##########
common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java:
##########
@@ -198,7 +198,7 @@ public class RssClientConf {
   public static final ConfigOption<Boolean> RSS_CLIENT_REMOTE_STORAGE_USE_LOCAL_CONF_ENABLED =
       ConfigOptions.key("rss.client.remote.storage.useLocalConfAsDefault")
           .booleanType()
-          .defaultValue(false)
+          .defaultValue(true)

Review Comment:
   I hope so, that will make shuffle server without hadoop env by default



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1996828615

   ## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Project coverage is 54.01%. Comparing base [(`f6569b0`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/f6569b046d2a1c0ec95fb390fe36f17c43aec052?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`dc9b3d2`)](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1577      +/-   ##
   ============================================
   + Coverage     53.99%   54.01%   +0.01%     
   + Complexity     2862     2293     -569     
   ============================================
     Files           438      360      -78     
     Lines         24819    16063    -8756     
     Branches       2111     1465     -646     
   ============================================
   - Hits          13402     8677    -4725     
   + Misses        10578     6884    -3694     
   + Partials        839      502     -337     
   ```
   
   
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1577?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on PR #1577:
URL: https://github.com/apache/incubator-uniffle/pull/1577#issuecomment-1999122187

   > I hesitate for this pull request. If users have already used external hadoop package, will it affect them if they don't modify the config option.
   
   It won't. We just use the same config both for the read and write. BTW, this has been applied in our internal cluster.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org