You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by jackylk <gi...@git.apache.org> on 2018/05/01 12:27:58 UTC
[GitHub] carbondata pull request #2256: [CARBONDATA-2422] Search mode Master port sho...
GitHub user jackylk opened a pull request:
https://github.com/apache/carbondata/pull/2256
[CARBONDATA-2422] Search mode Master port should be dynamic
In SDV test, sometimes search mode testcase failed because Master port is occupied. This PR adds support for dynamic master port to avoid port binding failure
- [X] Any interfaces changed?
No
- [X] Any backward compatibility impacted?
No
- [X] Document update required?
No
- [X] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
No
- [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jackylk/incubator-carbondata master-port
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2256.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2256
----
commit a54ff923443b9abd5ef9decc1d93bd29e4a2a36c
Author: Jacky Li <ja...@...>
Date: 2018-05-01T12:25:16Z
support dynamic master port
----
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4397/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2256
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4682/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2256
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4655/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5596/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4425/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2256
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4690/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5559/
---
[GitHub] carbondata pull request #2256: [CARBONDATA-2422] Search mode Master port sho...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2256
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5586/
---
[GitHub] carbondata pull request #2256: [CARBONDATA-2422] Search mode Master port sho...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2256#discussion_r186953529
--- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
@@ -64,22 +64,39 @@ class Master(sparkConf: SparkConf, port: Int) {
private val scheduler: Scheduler = new Scheduler
- def this(sparkConf: SparkConf) = {
- this(sparkConf, CarbonProperties.getSearchMasterPort)
- }
-
/** start service and listen on port passed in constructor */
def startService(): Unit = {
if (rpcEnv == null) {
new Thread(new Runnable {
override def run(): Unit = {
val hostAddress = InetAddress.getLocalHost.getHostAddress
- val config = RpcEnvConfig(
- sparkConf, "registry-service", hostAddress, "", CarbonProperties.getSearchMasterPort,
- new SecurityManager(sparkConf), clientMode = false)
- rpcEnv = new NettyRpcEnvFactory().create(config)
+ var port = CarbonProperties.getSearchMasterPort
--- End diff --
It's better to ensure there are no intersection between master port and worker port, for example. the default master and worker port is 10100, 10200, not 10020,10021. Because if master use 10020 fail, master will use 10021, and the worker will failed to use 10021.
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2256
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4435/
---
[GitHub] carbondata issue #2256: [CARBONDATA-2422] Search mode Master port should be ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2256
LGTM
---