You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by MrSandmanRUS <gi...@git.apache.org> on 2018/09/06 21:22:48 UTC
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
GitHub user MrSandmanRUS opened a pull request:
https://github.com/apache/phoenix/pull/343
PHOENIX-4892 Unable to start load balancer with queryserver
Added phoenix_loadbalancer_jar path to phoenix_utils.py,
added phoenix_loadbalancer_jar to classpath in queryserver.py,
added service for registry in meta-inf
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MrSandmanRUS/phoenix PHOENIX-4892
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/phoenix/pull/343.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #343
----
commit b906e545eeaf6787192f5952a19506d379361c40
Author: Vitaliy <vi...@...>
Date: 2018-09-06T21:20:12Z
PHOENIX-4892 Unable to start load balancer with queryserver
Added phoenix_loadbalancer_jar path to phoenix_utils.py,
added phoenix_loadbalancer_jar to classpath in queryserver.py,
added service for registry in meta-inf
----
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/343
Ok, this looks good to go. Thanks for the quick changes @MrSandmanRUS.
Want to file another Jira issue to make sure the load balance jars make it into the normal phoenix-assembly tarball (and then, re-add the check inside of lib/)?
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218503306
--- Diff: bin/phoenix_utils.py ---
@@ -71,6 +71,7 @@ def setPath():
PHOENIX_CLIENT_JAR_PATTERN = "phoenix-*-client.jar"
PHOENIX_THIN_CLIENT_JAR_PATTERN = "phoenix-*-thin-client.jar"
PHOENIX_QUERYSERVER_JAR_PATTERN = "phoenix-*-queryserver.jar"
+ PHOENIX_LOADBALANCER_JAR_PATTERN = "phoenix-load-balancer-*[!t][!e][!s][!t][!s].jar"
--- End diff --
I think all of the extra square-brackets are unnecessary here. Wouldn't `phoenix-load-balancer-*[!tests].jar` be sufficient?
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218529331
--- Diff: bin/phoenix_utils.py ---
@@ -162,6 +163,13 @@ def setPath():
if phoenix_queryserver_jar == "":
phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, ".."))
+ global phoenix_loadbalancer_jar
+ phoenix_loadbalancer_jar = find(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-loadbalancer", "target", "*"))
+ if phoenix_loadbalancer_jar == "":
+ phoenix_loadbalancer_jar = findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
--- End diff --
@joshelser Load balancer is presented as a separate jar-file and the query-server.jar is uses for connection to load-balancer special java-class - ServiceLoader, which is looking for the right jar-file with implementation of necessary class. For release you just need to rebuild load-balancer.jar with my fix ("added service for registry in meta-inf"). For this you can use pom.xml in the load-balancer package.
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218525645
--- Diff: bin/phoenix_utils.py ---
@@ -71,6 +71,7 @@ def setPath():
PHOENIX_CLIENT_JAR_PATTERN = "phoenix-*-client.jar"
PHOENIX_THIN_CLIENT_JAR_PATTERN = "phoenix-*-thin-client.jar"
PHOENIX_QUERYSERVER_JAR_PATTERN = "phoenix-*-queryserver.jar"
+ PHOENIX_LOADBALANCER_JAR_PATTERN = "phoenix-load-balancer-*[!t][!e][!s][!t][!s].jar"
--- End diff --
Unfortunately, it is impossible. Here used fnmatch (https://docs.python.org/2/library/fnmatch.html) and [!tests] means matches any character not in "tests" (just 1 character in the end), so if we will have phoenix-load-balancer-someversion-snapshot.jar we will not find it.
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on the issue:
https://github.com/apache/phoenix/pull/343
@joshelser can you review, please?
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on the issue:
https://github.com/apache/phoenix/pull/343
or is it better to create a new issue?
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on the issue:
https://github.com/apache/phoenix/pull/343
ok, thank you!
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on the issue:
https://github.com/apache/phoenix/pull/343
@joshelser yeah, I can do it, but can you help me, where I can find file with specification of libs inside phoenix-assembly tarball? Is it /phoenix/pom.xml? (main pom)
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/phoenix/pull/343
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218537424
--- Diff: bin/phoenix_utils.py ---
@@ -71,6 +71,7 @@ def setPath():
PHOENIX_CLIENT_JAR_PATTERN = "phoenix-*-client.jar"
PHOENIX_THIN_CLIENT_JAR_PATTERN = "phoenix-*-thin-client.jar"
PHOENIX_QUERYSERVER_JAR_PATTERN = "phoenix-*-queryserver.jar"
+ PHOENIX_LOADBALANCER_JAR_PATTERN = "phoenix-load-balancer-*[!t][!e][!s][!t][!s].jar"
--- End diff --
Bummer. Maybe we switch this over to just use normal Python `re` later. Fine for now -- thanks for clarifying!
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on the issue:
https://github.com/apache/phoenix/pull/343
Can I do it in current branch?
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218532590
--- Diff: bin/phoenix_utils.py ---
@@ -162,6 +163,13 @@ def setPath():
if phoenix_queryserver_jar == "":
phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, ".."))
+ global phoenix_loadbalancer_jar
+ phoenix_loadbalancer_jar = find(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-loadbalancer", "target", "*"))
+ if phoenix_loadbalancer_jar == "":
+ phoenix_loadbalancer_jar = findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
--- End diff --
Oh, I understand, yeah, I'll fix it
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/343
> or is it better to create a new issue?
Create a new issue in a little bit please. I'm already working through applying this change :)
---
[GitHub] phoenix issue #343: PHOENIX-4892 Unable to start load balancer with queryser...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/343
You need to add the phoenix-loadbalancer module as a dependency to phoenix-assembly/pom.xml. I think `phoenix-assembly/src/build/components/all-common-dependencies.xml` already has the necessary configuration in place to land the dependencies into lib/ for you.
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by MrSandmanRUS <gi...@git.apache.org>.
Github user MrSandmanRUS commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218535960
--- Diff: bin/phoenix_utils.py ---
@@ -162,6 +163,13 @@ def setPath():
if phoenix_queryserver_jar == "":
phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, ".."))
+ global phoenix_loadbalancer_jar
+ phoenix_loadbalancer_jar = find(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-loadbalancer", "target", "*"))
+ if phoenix_loadbalancer_jar == "":
+ phoenix_loadbalancer_jar = findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
--- End diff --
@joshelser ready :)
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218531961
--- Diff: bin/phoenix_utils.py ---
@@ -162,6 +163,13 @@ def setPath():
if phoenix_queryserver_jar == "":
phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, ".."))
+ global phoenix_loadbalancer_jar
+ phoenix_loadbalancer_jar = find(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-loadbalancer", "target", "*"))
+ if phoenix_loadbalancer_jar == "":
+ phoenix_loadbalancer_jar = findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
--- End diff --
Yes, I understand how the load-balancer is meant to work (I was involved in the reviews and stipulated it be this way ;)).
I'm trying to point out that the logic you've added here is useless on master because the jar doesn't exist in the lib directory of the tarball created by the phoenix-assembly module. Specifically, the following path is never valid because the JAR is not put there on normal tarball builds of Phoenix
```
findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
```
---
[GitHub] phoenix pull request #343: PHOENIX-4892 Unable to start load balancer with q...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/343#discussion_r218504342
--- Diff: bin/phoenix_utils.py ---
@@ -162,6 +163,13 @@ def setPath():
if phoenix_queryserver_jar == "":
phoenix_queryserver_jar = findFileInPathWithoutRecursion(PHOENIX_QUERYSERVER_JAR_PATTERN, os.path.join(current_dir, ".."))
+ global phoenix_loadbalancer_jar
+ phoenix_loadbalancer_jar = find(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "phoenix-loadbalancer", "target", "*"))
+ if phoenix_loadbalancer_jar == "":
+ phoenix_loadbalancer_jar = findFileInPathWithoutRecursion(PHOENIX_LOADBALANCER_JAR_PATTERN, os.path.join(current_dir, "..", "lib"))
--- End diff --
I'm just noticing in my 5.1.0-SNAPSHOT build that the phoenix-load-balancer jar isn't there at all. Does phoenix-assembly need to be updated as well?
---