You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@fluo.apache.org by GitBox <gi...@apache.org> on 2020/02/29 15:18:00 UTC
[GitHub] [fluo] jkosh44 opened a new pull request #1087: Add Oracle count
functionality
jkosh44 opened a new pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087
Add functionality to FluoAdminImpl to count the number of Oracles
started. This allows the 'fluo list' command to print the number
of Oracles for each app.
Fixes #895
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386060512
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
~~This might not actually be the best idea because it relies on the Curator LeaderLatch implementation.
The docs seem to advise against using the paths given to Curator Recipes: https://cwiki.apache.org/confluence/display/CURATOR/TN7~~
~~Another possible option could be maintaining some count in zookeeper that is incremented and decremented when the oracle starts and stops~~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r387850508
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +517,20 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public int numOracles() {
Review comment:
@keith-turner looking at the `oracleExists` method I wonder if we should switch `!curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).isEmpty()` with something like
`(new LeaderLatch(curator, ZookeeperPath.ORACLE_SERVER)).getParticipants().size() > 0`.
The current implementation is probably right but it does rely on the LeaderLatch implementation and uses the LeaderLatch's path which Curator advises against (https://cwiki.apache.org/confluence/display/CURATOR/TN7). They're probably more concerned about creating and deleting nodes so it's probably not a big deal, but may be slightly safer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386752709
##########
File path: modules/command/src/main/java/org/apache/fluo/command/FluoList.java
##########
@@ -91,7 +91,8 @@ private void listApp(FluoConfiguration config, String path) {
state = "RUNNING";
}
int numWorkers = admin.numWorkers();
- System.out.format("%-15s %-11s %4d\n", path, state, numWorkers);
+ int numOracles = admin.numOracles();
+ System.out.format("%-15s %-11s %4d %13d\n", path, state, numWorkers, numOracles);
Review comment:
I've attached some screenshots so you can see how the spacing looks.
As a note, if you call `fluo list` too soon after starting or stopping an oracle then the oracle count may not have been updated yet (you can see some examples of this in my screenshots). I see the same behavior with workers so I think it should be fine.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386060512
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
This might not actually be the best idea because it relies on the Curator LeaderLatch implementation.
The docs seem to advise against using the paths given to Curator Recipes: https://cwiki.apache.org/confluence/display/CURATOR/TN7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386034681
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
I'm a little concerned that the LeaderLatch may create some Ephemeral ZNodes during leader election that would cause this to be temporarily inaccurate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] keith-turner commented on a change in pull request #1087:
Add Oracle count functionality
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386486954
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +517,23 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
Review comment:
Seems like this method is only called in one place and could be inlined.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386070473
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
Never mind I'm pretty sure I found a much better way in the new commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386060512
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
This might not actually be the best idea because it relies on the Curator LeaderLatch implementation.
The docs seem to advise against using the paths given to Curator Recipes: https://cwiki.apache.org/confluence/display/CURATOR/TN7
Another possible option could be maintaining some count in zookeeper that is incremented and decremented when the oracle starts and stops
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386034681
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +516,22 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
+ int numOracles = 0;
+ if (oracleExists(curator)) {
+ try {
+ numOracles += curator.getChildren().forPath(ZookeeperPath.ORACLE_SERVER).size();
Review comment:
~~I'm a little concerned that the LeaderLatch may create some Ephemeral ZNodes during leader election that would cause this to be temporarily inaccurate.~~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] keith-turner commented on a change in pull request #1087:
Add Oracle count functionality
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386487390
##########
File path: modules/command/src/main/java/org/apache/fluo/command/FluoList.java
##########
@@ -91,7 +91,8 @@ private void listApp(FluoConfiguration config, String path) {
state = "RUNNING";
}
int numWorkers = admin.numWorkers();
- System.out.format("%-15s %-11s %4d\n", path, state, numWorkers);
+ int numOracles = admin.numOracles();
+ System.out.format("%-15s %-11s %4d %13d\n", path, state, numWorkers, numOracles);
Review comment:
This is a really neat change. Would you happen to have any example output from running? If not don't worry about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 merged pull request #1087: Add Oracle count
functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 merged pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] keith-turner commented on a change in pull request #1087:
Add Oracle count functionality
Posted by GitBox <gi...@apache.org>.
keith-turner commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r387876058
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +517,20 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public int numOracles() {
Review comment:
I think that would be a good change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [fluo] jkosh44 commented on a change in pull request #1087: Add
Oracle count functionality
Posted by GitBox <gi...@apache.org>.
jkosh44 commented on a change in pull request #1087: Add Oracle count functionality
URL: https://github.com/apache/fluo/pull/1087#discussion_r386755051
##########
File path: modules/core/src/main/java/org/apache/fluo/core/client/FluoAdminImpl.java
##########
@@ -516,6 +517,23 @@ public boolean oracleExists() {
return oracleExists(getAppCurator());
}
+ public static int numOracles(CuratorFramework curator) {
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services