You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by prashanth-vasudev <gi...@git.apache.org> on 2017/01/19 00:17:54 UTC
[GitHub] incubator-trafodion pull request #917: [TRAFODION-2150] and [TRAFODION-2151]...
GitHub user prashanth-vasudev opened a pull request:
https://github.com/apache/incubator-trafodion/pull/917
[TRAFODION-2150] and [TRAFODION-2151] TM long prepare and chore thread Issues
Queries such as drop schema cascase involve multiple DDL operations that take some time to process. This cause transaction commits to timeout and cause transaction aborts. Fix is to detect long prepare scenarios and continue to commit the transaction.
Cleanup Chore thread calls were deprecated and required chore service constructs. This should help memory cleanup issues in end point co processor. This should also help some of the regression test failure we see in executor/TEST130
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/prashanth-vasudev/incubator-trafodion longPrepare
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/917.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #917
----
commit 30c8a83d1512667f5ecfe2c7384372f646f18ae3
Author: Prashant Vasudev <pr...@esgyn.com>
Date: 2017-01-19T00:11:33Z
[TRAFODION-2150] and [TRAFODION-2151] TM long prepare and chore thread issues.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #917: [TRAFODION-2150] and [TRAFODION-2151]...
Posted by prashanth-vasudev <gi...@git.apache.org>.
Github user prashanth-vasudev commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/917#discussion_r96957797
--- Diff: core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java ---
@@ -1108,9 +1108,28 @@ public void run() {
}
if (LOG.isDebugEnabled()) LOG.debug("TRAF RCOV THREAD: in-doubt transaction size " + transactionStates.size());
for (Map.Entry<Long, TransactionState> tsEntry : transactionStates.entrySet()) {
+ int isTransactionStillAlive = 0;
TransactionState ts = tsEntry.getValue();
Long txID = ts.getTransactionId();
// TransactionState ts = new TransactionState(txID);
+
+ //It is possible for long prepare situations that involve multiple DDL
+ //operations, multiple prompts from RS is received. Hence check to see if there
+ //is a TS object in main TS list and transaction is still active.
+ //Note that tsEntry is local TS object.
+ if (hbtx.mapTransactionStates.get(txID) != null) {
+ if (hbtx.mapTransactionStates.get(txID).getStatus().toString().contains("ACTIVE")) {
+ isTransactionStillAlive = 1;
+ }
+ if (LOG.isInfoEnabled())
+ LOG.info("TRAF RCOV THREAD: TID " + txID
+ + " still has ts object in TM memory with state "
+ + hbtx.mapTransactionStates.get(txID).getStatus().toString()
--- End diff --
Thanks for the review. I have updated the logging info to include TS object. Regarding committing/committed changes is part of separate Jira.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #917: [TRAFODION-2150] and [TRAFODION-2151]...
Posted by sbroeder <gi...@git.apache.org>.
Github user sbroeder commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/917#discussion_r96768810
--- Diff: core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java ---
@@ -1108,9 +1108,28 @@ public void run() {
}
if (LOG.isDebugEnabled()) LOG.debug("TRAF RCOV THREAD: in-doubt transaction size " + transactionStates.size());
for (Map.Entry<Long, TransactionState> tsEntry : transactionStates.entrySet()) {
+ int isTransactionStillAlive = 0;
TransactionState ts = tsEntry.getValue();
Long txID = ts.getTransactionId();
// TransactionState ts = new TransactionState(txID);
+
+ //It is possible for long prepare situations that involve multiple DDL
+ //operations, multiple prompts from RS is received. Hence check to see if there
+ //is a TS object in main TS list and transaction is still active.
+ //Note that tsEntry is local TS object.
+ if (hbtx.mapTransactionStates.get(txID) != null) {
+ if (hbtx.mapTransactionStates.get(txID).getStatus().toString().contains("ACTIVE")) {
+ isTransactionStillAlive = 1;
+ }
+ if (LOG.isInfoEnabled())
+ LOG.info("TRAF RCOV THREAD: TID " + txID
+ + " still has ts object in TM memory with state "
+ + hbtx.mapTransactionStates.get(txID).getStatus().toString()
--- End diff --
How do you feel about printing the entire TS object instead of just the state? Some other useful information could be printed
for example:
LOG.info("TRAF RCOV THREAD: TID " + txID + " still has ts object in TM memory " + hbtx.mapTransactionStates.get(txID).toString()
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #917: [TRAFODION-2150] and [TRAFODION-2151]...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/917
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #917: [TRAFODION-2150] and [TRAFODION-2151]...
Posted by shangshengtung <gi...@git.apache.org>.
Github user shangshengtung commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/917#discussion_r96775246
--- Diff: core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java ---
@@ -1108,9 +1108,28 @@ public void run() {
}
if (LOG.isDebugEnabled()) LOG.debug("TRAF RCOV THREAD: in-doubt transaction size " + transactionStates.size());
for (Map.Entry<Long, TransactionState> tsEntry : transactionStates.entrySet()) {
+ int isTransactionStillAlive = 0;
TransactionState ts = tsEntry.getValue();
Long txID = ts.getTransactionId();
// TransactionState ts = new TransactionState(txID);
+
+ //It is possible for long prepare situations that involve multiple DDL
+ //operations, multiple prompts from RS is received. Hence check to see if there
+ //is a TS object in main TS list and transaction is still active.
+ //Note that tsEntry is local TS object.
+ if (hbtx.mapTransactionStates.get(txID) != null) {
+ if (hbtx.mapTransactionStates.get(txID).getStatus().toString().contains("ACTIVE")) {
+ isTransactionStillAlive = 1;
+ }
+ if (LOG.isInfoEnabled())
+ LOG.info("TRAF RCOV THREAD: TID " + txID
+ + " still has ts object in TM memory with state "
+ + hbtx.mapTransactionStates.get(txID).getStatus().toString()
--- End diff --
Do we plan to move "committing/committed" changes next round?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---