You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/07/07 12:18:01 UTC
[GitHub] [hive] pvargacl opened a new pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
pvargacl opened a new pull request #1219:
URL: https://github.com/apache/hive/pull/1219
Fix flaky test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450823301
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -568,6 +568,7 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
*/
LOG.error("OpenTxnTimeOut exceeded commit duration {}, deleting transactionIds: {}", elapsedMillis, txnIds);
deleteInvalidOpenTransactions(dbConn, txnIds);
+ dbConn.commit();
Review comment:
Why is this change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450822761
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/txn/TestAcidTxnCleanerService.java
##########
@@ -148,16 +149,16 @@ public void cleansEmptyAbortedBatchTxns() throws Exception {
TxnStore.TIMED_OUT_TXN_ABORT_BATCH_SIZE + 50);
OpenTxnsResponse resp = txnHandler.openTxns(new OpenTxnRequest(
TxnStore.TIMED_OUT_TXN_ABORT_BATCH_SIZE + 50, "user", "hostname"));
+ txnHandler.setOpenTxnTimeOutMillis(1);
txnHandler.abortTxns(new AbortTxnsRequest(resp.getTxn_ids()));
GetOpenTxnsResponse openTxns = txnHandler.getOpenTxns();
Assert.assertEquals(TxnStore.TIMED_OUT_TXN_ABORT_BATCH_SIZE + 50 + 1, openTxns.getOpen_txnsSize());
- txnHandler.setOpenTxnTimeOutMillis(1);
underTest.run();
openTxns = txnHandler.getOpenTxns();
Assert.assertEquals(2, openTxns.getOpen_txnsSize());
- Assert.assertEquals(TxnStore.TIMED_OUT_TXN_ABORT_BATCH_SIZE + 50 + 1, getMaxTxnId());
+ Assert.assertTrue("The max txnId shoul be at least", getMaxTxnId() >= TxnStore.TIMED_OUT_TXN_ABORT_BATCH_SIZE + 50 + 1);
Review comment:
nit: should
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450844285
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -568,6 +568,7 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
*/
LOG.error("OpenTxnTimeOut exceeded commit duration {}, deleting transactionIds: {}", elapsedMillis, txnIds);
deleteInvalidOpenTransactions(dbConn, txnIds);
+ dbConn.commit();
Review comment:
Maybe in a follow-up jira...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450822656
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/txn/TestAcidTxnCleanerService.java
##########
@@ -135,7 +136,7 @@ public void cleansCommittedAndEmptyAbortedOnly() throws Exception {
// kept only the 5 non-empty aborted ones
Assert.assertEquals(5, getTxnCount());
- Assert.assertEquals(15, getMaxTxnId());
+ Assert.assertTrue("The max txnId shoul be at least 15", getMaxTxnId() >= 15);
Review comment:
nit: should
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #1219:
URL: https://github.com/apache/hive/pull/1219#issuecomment-654819192
Could you please validate your fix with the flaky test checker jenkins, and add the relevant url?
Thanks,
Peter
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvargacl commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvargacl commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450829187
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -568,6 +568,7 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
*/
LOG.error("OpenTxnTimeOut exceeded commit duration {}, deleting transactionIds: {}", elapsedMillis, txnIds);
deleteInvalidOpenTransactions(dbConn, txnIds);
+ dbConn.commit();
Review comment:
It was missing, the delete was not persisted. When I was testing manually, I didn't notice, because I was querying back the data on the same connection...
It does not cause a big problem, these transactions would be cleaned up eventually when they time out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvargacl commented on pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvargacl commented on pull request #1219:
URL: https://github.com/apache/hive/pull/1219#issuecomment-654826153
@pvary It is already running :) http://ci.hive.apache.org/job/hive-flaky-check/64/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvargacl commented on pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvargacl commented on pull request #1219:
URL: https://github.com/apache/hive/pull/1219#issuecomment-655935237
@pvary The flaky test run passed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1219:
URL: https://github.com/apache/hive/pull/1219#discussion_r450844111
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -568,6 +568,7 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
*/
LOG.error("OpenTxnTimeOut exceeded commit duration {}, deleting transactionIds: {}", elapsedMillis, txnIds);
deleteInvalidOpenTransactions(dbConn, txnIds);
+ dbConn.commit();
Review comment:
I thought close(null, stmt, dbConn) commits, but you are right, that it does not commit.
Shall we create a test case for this? :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary merged pull request #1219: HIVE-23525: TestAcidTxnCleanerService is unstable
Posted by GitBox <gi...@apache.org>.
pvary merged pull request #1219:
URL: https://github.com/apache/hive/pull/1219
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org