You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by blrunner <gi...@git.apache.org> on 2015/04/30 05:43:04 UTC
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
GitHub user blrunner opened a pull request:
https://github.com/apache/tajo/pull/560
TAJO-1586: TajoMaster HA startup failure on Yarn.
I tested this successfully on production cluster and I fixed various bugs related TajoMaster HA. For the reference, I replaced master IP address to master host name on Tajo web interface.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/blrunner/tajo TAJO-1586
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/560.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #560
----
commit 58c144205637f7acc9145f8bcbdee0c5ca932cad
Author: JaeHwa Jung <bl...@apache.org>
Date: 2015-04-28T16:02:54Z
TAJO-1586: TajoMaster HA startup failure on Yarn.
commit 9c84e00480e03202b164925f3c363b305509a277
Author: JaeHwa Jung <bl...@apache.org>
Date: 2015-04-30T03:39:43Z
Fix various bugs
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by blrunner <gi...@git.apache.org>.
Github user blrunner closed the pull request at:
https://github.com/apache/tajo/pull/560
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by blrunner <gi...@git.apache.org>.
Github user blrunner commented on the pull request:
https://github.com/apache/tajo/pull/560#issuecomment-97720036
Hi @jinossy
I updated the patch. Could you check it again? :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on a diff in the pull request:
https://github.com/apache/tajo/pull/560#discussion_r29731769
--- Diff: tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java ---
@@ -184,15 +181,31 @@ private void createMasterFile(boolean isActive) throws IOException {
createMasterFile(false);
}
+ // Set active status and delete existing file.
if (isActive) {
isActiveStatus = true;
+ Path backupFile = new Path(backupPath, fileName);
+ if (fs.exists(backupFile)) {
+ fs.delete(backupFile, true);
+ }
} else {
isActiveStatus = false;
+ Path activeFile = new Path(activePath, fileName);
--- End diff --
Can you refactor the duplicate codes?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by blrunner <gi...@git.apache.org>.
Github user blrunner commented on a diff in the pull request:
https://github.com/apache/tajo/pull/560#discussion_r29412956
--- Diff: tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java ---
@@ -70,12 +70,45 @@
public HdfsServiceTracker(TajoConf conf) throws IOException {
this.conf = conf;
+ }
+
+ @Override
+ public void register() throws IOException {
initSystemDirectory();
InetSocketAddress socketAddress = conf.getSocketAddrVar(ConfVars.TAJO_MASTER_UMBILICAL_RPC_ADDRESS);
- this.masterName = socketAddress.getAddress().getHostAddress() + ":" + socketAddress.getPort();
+ this.masterName = socketAddress.getAddress().getHostName()+ ":" + socketAddress.getPort();
--- End diff --
Thanks for your review.
I also think that ip address would be more safety compare than host name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on a diff in the pull request:
https://github.com/apache/tajo/pull/560#discussion_r29731683
--- Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java ---
@@ -142,6 +142,8 @@ public static int setDateOrder(int dateOrder) {
// High availability configurations
TAJO_MASTER_HA_ENABLE("tajo.master.ha.enable", false, Validators.bool()),
TAJO_MASTER_HA_MONITOR_INTERVAL("tajo.master.ha.monitor.interval", 5 * 1000), // 5 sec
+ TAJO_MASTER_HA_CLIENT_RETRY_MAX_NUM("tajo.master.ha.client.read.retry.max-num", 20),
+ TAJO_MASTER_HA_CLIENT_RETRY_WAIT_TIME("tajo.master.ha.client.read.wait-time", 5 * 1000), // 5 sec
--- End diff --
IMO, 5 sec is too long. What do you think to change (500 ms * 120 retry) ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1586: TajoMaster HA startup failure on Yar...
Posted by jinossy <gi...@git.apache.org>.
Github user jinossy commented on a diff in the pull request:
https://github.com/apache/tajo/pull/560#discussion_r29403727
--- Diff: tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java ---
@@ -70,12 +70,45 @@
public HdfsServiceTracker(TajoConf conf) throws IOException {
this.conf = conf;
+ }
+
+ @Override
+ public void register() throws IOException {
initSystemDirectory();
InetSocketAddress socketAddress = conf.getSocketAddrVar(ConfVars.TAJO_MASTER_UMBILICAL_RPC_ADDRESS);
- this.masterName = socketAddress.getAddress().getHostAddress() + ":" + socketAddress.getPort();
+ this.masterName = socketAddress.getAddress().getHostName()+ ":" + socketAddress.getPort();
--- End diff --
If a hostname is not FQDN, it may be throw the unknown host exception.
In my opinion, ip address is more safety. How do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---