You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/09/20 06:56:06 UTC
[GitHub] [incubator-uniffle] kaijchen opened a new issue, #232: Minimize required configs to work out of the box
kaijchen opened a new issue, #232:
URL: https://github.com/apache/incubator-uniffle/issues/232
Provide optimized default values when a field in config file is missing,
so it's easier for inexperienced users to try and experiment with this project.
For example:
If the `rss.storage.type` field is missing , it may defaults to `MEMORY_LOCALFILE_HDFS`.
Currently, it throws `NullPointerException: Name is null`.
In summary, let's try make the default settings work out of the box.
Please refer to how easy it is to setup a single node cluster for HDFS or Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jiangbiao910 commented on issue #232: Minimize required configs to work out of the box
Posted by GitBox <gi...@apache.org>.
jiangbiao910 commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1277044995
@kaijchen @zuston @jerqi When I deploy Coordinator Server from master , it throws `[ERROR] 2022-10-13 11:22:04,116 main CoordinatorServer <init> - Errors on initializing coordinator server.
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
at org.apache.uniffle.coordinator.ApplicationManager.getStorageHost(ApplicationManager.java:278)
at org.apache.uniffle.coordinator.ApplicationManager.addRemoteStorageMetrics(ApplicationManager.java:263)
at org.apache.uniffle.coordinator.ApplicationManager.refreshRemoteStorage(ApplicationManager.java:109)
at org.apache.uniffle.coordinator.ClientConfManager.updateClientConfInternal(ClientConfManager.java:130)
at org.apache.uniffle.coordinator.ClientConfManager.init(ClientConfManager.java:73)
at org.apache.uniffle.coordinator.ClientConfManager.<init>(ClientConfManager.java:58)
at org.apache.uniffle.coordinator.CoordinatorServer.initialization(CoordinatorServer.java:142)
at org.apache.uniffle.coordinator.CoordinatorServer.<init>(CoordinatorServer.java:61)
at org.apache.uniffle.coordinator.CoordinatorServer.main(CoordinatorServer.java:79)
`
How to solve the problem？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jiangbiao910 commented on issue #232: Minimize required configs to work out of the box
Posted by GitBox <gi...@apache.org>.
jiangbiao910 commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1278489509
@zuston conf shoud be set
rss.coordinator.remote.storage.path hdfs://nameservice1/tmp/path1,hdfs://nameservice1/tmp/path2
no
rss.coordinator.remote.storage.path hdfs:///tmp/path1,hdfs:///tmp/path2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] smlHao commented on issue #232: Minimize required configs to work out of the box
Posted by "smlHao (via GitHub)" <gi...@apache.org>.
smlHao commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1649401211
> Ok, we can give more configurations a default value.
@jerqi Hi, I have a question , there are 3 conf file can set rss.storage.type ，
(1) spark-default.conf as uniffle client
(2) $RSS_HOME/conf/server.conf
（3 $RSS_HOME/conf/server.conf
where can I set the path on hdfs if rss.storage.type is MEMORY_LOCALFILE_HDFS ? rss.storage.basePath is local disk path the shuffle server be deployed ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #232: Minimize required configs to work out of the box
Posted by GitBox <gi...@apache.org>.
zuston commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1253500821
Make sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #232: Minimize required configs to work out of the box
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1253123780
Ok, we can give more configurations a default value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #232: Minimize required configs to work out of the box
Posted by GitBox <gi...@apache.org>.
zuston commented on issue #232:
URL: https://github.com/apache/incubator-uniffle/issues/232#issuecomment-1277085147
Please attach your coordinator.conf file @jiangbiao910
The conf of `rss.coordinator.dynamicClientConf.enabled` default value is false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org