You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org> on 2016/10/21 21:07:15 UTC
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Michael Blow has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/1309
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Windows 10 Logs Constant Exceptions On CC/NC Death
- Backoff delays on IPC Listener Thread exceptions; use logger instead of
printStackTrace()
Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
---
M hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
1 file changed, 20 insertions(+), 9 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/09/1309/1
diff --git a/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java b/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
index b03b7e0..25b7fed 100644
--- a/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
+++ b/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
@@ -35,6 +35,7 @@
import java.util.Iterator;
import java.util.List;
import java.util.Map;
+import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
@@ -71,11 +72,11 @@
ServerSocket socket = serverSocketChannel.socket();
socket.bind(socketAddress);
address = new InetSocketAddress(socket.getInetAddress(), socket.getLocalPort());
- ipcHandleMap = new HashMap<InetSocketAddress, IPCHandle>();
- pendingConnections = new ArrayList<IPCHandle>();
- workingPendingConnections = new ArrayList<IPCHandle>();
- sendList = new ArrayList<Message>();
- workingSendList = new ArrayList<Message>();
+ ipcHandleMap = new HashMap<>();
+ pendingConnections = new ArrayList<>();
+ workingPendingConnections = new ArrayList<>();
+ sendList = new ArrayList<>();
+ workingSendList = new ArrayList<>();
}
InetSocketAddress getAddress() {
@@ -191,7 +192,8 @@
throw new RuntimeException(e);
}
BitSet unsentMessagesBitmap = new BitSet();
- List<Message> tempUnsentMessages = new ArrayList<Message>();
+ List<Message> tempUnsentMessages = new ArrayList<>();
+ int failingLoops = 0;
while (!stopped) {
try {
if (LOGGER.isLoggable(Level.FINE)) {
@@ -204,7 +206,7 @@
SocketChannel channel = SocketChannel.open();
channel.setOption(StandardSocketOptions.TCP_NODELAY, true);
channel.configureBlocking(false);
- SelectionKey cKey = null;
+ SelectionKey cKey;
if (channel.connect(handle.getRemoteAddress())) {
cKey = channel.register(selector, SelectionKey.OP_READ);
handle.setState(HandleState.CONNECT_SENT);
@@ -310,7 +312,7 @@
throw new Exception("Connection did not finish");
}
} catch (Exception e) {
- e.printStackTrace();
+ LOGGER.log(Level.WARNING, "Exception finishing connection", e);
handle.setState(HandleState.CONNECT_FAILED);
continue;
}
@@ -321,8 +323,17 @@
}
}
}
+ // reset failingLoops on a good loop
+ failingLoops = 0;
} catch (Exception e) {
- e.printStackTrace();
+ int sleepSecs = (int)Math.pow(2, Math.min(11, failingLoops++));
+ LOGGER.log(Level.WARNING, "Exception processing message; sleeping " + sleepSecs
+ + " seconds", e);
+ try {
+ Thread.sleep(TimeUnit.SECONDS.toMillis(sleepSecs));
+ } catch (InterruptedException e1) {
+ Thread.currentThread().interrupt();
+ }
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 1:
(1 comment)
Looks good, one question on the log message.
https://asterix-gerrit.ics.uci.edu/#/c/1309/1/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
File hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java:
Line 315: LOGGER.log(Level.WARNING, "Exception finishing connection", e);
This is a little confusing. "Finish connection" could mean a) "close connection" or b) "finish the process of connecting". I think that it is b) in this case. Is that right? If so, should be have a more explicit log message?
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: Yes
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1000/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3115/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/1309
to look at the new patch set (#2).
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Windows 10 Logs Constant Exceptions On CC/NC Death
- Backoff delays on IPC Listener Thread exceptions; use logger instead of
printStackTrace()
Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
---
M hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
1 file changed, 34 insertions(+), 14 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/09/1309/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Michael Blow (Code Review)" <do...@asterixdb.incubator.apache.org>.
Michael Blow has submitted this change and it was merged.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Windows 10 Logs Constant Exceptions On CC/NC Death
- Backoff delays on IPC Listener Thread exceptions; use logger instead of
printStackTrace()
Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1309
Sonar-Qube: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Till Westmann <ti...@apache.org>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
---
M hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
1 file changed, 34 insertions(+), 14 deletions(-)
Approvals:
Till Westmann: Looks good to me, approved
Jenkins: Verified; No violations found; Verified
diff --git a/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java b/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
index b03b7e0..9ee135b 100644
--- a/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
+++ b/hyracks-fullstack/hyracks/hyracks-ipc/src/main/java/org/apache/hyracks/ipc/impl/IPCConnectionManager.java
@@ -35,6 +35,7 @@
import java.util.Iterator;
import java.util.List;
import java.util.Map;
+import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
@@ -71,11 +72,11 @@
ServerSocket socket = serverSocketChannel.socket();
socket.bind(socketAddress);
address = new InetSocketAddress(socket.getInetAddress(), socket.getLocalPort());
- ipcHandleMap = new HashMap<InetSocketAddress, IPCHandle>();
- pendingConnections = new ArrayList<IPCHandle>();
- workingPendingConnections = new ArrayList<IPCHandle>();
- sendList = new ArrayList<Message>();
- workingSendList = new ArrayList<Message>();
+ ipcHandleMap = new HashMap<>();
+ pendingConnections = new ArrayList<>();
+ workingPendingConnections = new ArrayList<>();
+ sendList = new ArrayList<>();
+ workingSendList = new ArrayList<>();
}
InetSocketAddress getAddress() {
@@ -191,7 +192,8 @@
throw new RuntimeException(e);
}
BitSet unsentMessagesBitmap = new BitSet();
- List<Message> tempUnsentMessages = new ArrayList<Message>();
+ List<Message> tempUnsentMessages = new ArrayList<>();
+ int failingLoops = 0;
while (!stopped) {
try {
if (LOGGER.isLoggable(Level.FINE)) {
@@ -204,7 +206,7 @@
SocketChannel channel = SocketChannel.open();
channel.setOption(StandardSocketOptions.TCP_NODELAY, true);
channel.configureBlocking(false);
- SelectionKey cKey = null;
+ SelectionKey cKey;
if (channel.connect(handle.getRemoteAddress())) {
cKey = channel.register(selector, SelectionKey.OP_READ);
handle.setState(HandleState.CONNECT_SENT);
@@ -305,15 +307,11 @@
} else if (key.isConnectable()) {
SocketChannel channel = (SocketChannel) sc;
IPCHandle handle = (IPCHandle) key.attachment();
- try {
- if (!channel.finishConnect()) {
- throw new Exception("Connection did not finish");
- }
- } catch (Exception e) {
- e.printStackTrace();
+ if (!finishConnect(channel)) {
handle.setState(HandleState.CONNECT_FAILED);
continue;
}
+
handle.setState(HandleState.CONNECT_SENT);
registerHandle(handle);
key.interestOps(SelectionKey.OP_READ);
@@ -321,12 +319,34 @@
}
}
}
+ // reset failingLoops on a good loop
+ failingLoops = 0;
} catch (Exception e) {
- e.printStackTrace();
+ int sleepSecs = (int)Math.pow(2, Math.min(11, failingLoops++));
+ LOGGER.log(Level.WARNING, "Exception processing message; sleeping " + sleepSecs
+ + " seconds", e);
+ try {
+ Thread.sleep(TimeUnit.SECONDS.toMillis(sleepSecs));
+ } catch (InterruptedException e1) {
+ Thread.currentThread().interrupt();
+ }
}
}
}
+ private boolean finishConnect(SocketChannel channel) {
+ boolean connectFinished = false;
+ try {
+ connectFinished = channel.finishConnect();
+ if (!connectFinished) {
+ LOGGER.log(Level.WARNING, "Channel connect did not finish");
+ }
+ } catch (IOException e) {
+ LOGGER.log(Level.WARNING, "Exception finishing channel connect", e);
+ }
+ return connectFinished;
+ }
+
private void copyUnsentMessages(BitSet unsentMessagesBitmap, List<Message> tempUnsentMessages) {
assert tempUnsentMessages.isEmpty();
for (int i = unsentMessagesBitmap.nextSetBit(0); i >= 0; i = unsentMessagesBitmap.nextSetBit(i + 1)) {
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3113/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1000/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/997/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No
Change in asterixdb[master]: Windows 10 Logs Constant Exceptions On CC/NC Death
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Windows 10 Logs Constant Exceptions On CC/NC Death
......................................................................
Patch Set 2: Integration-Tests-1
Integration Tests Failed
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/997/ : UNSTABLE
--
To view, visit https://asterix-gerrit.ics.uci.edu/1309
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec7a3ce23ab99d04b0fc0e0f832cce1a461e2df8
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-HasComments: No