You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by GitBox <gi...@apache.org> on 2021/07/06 15:40:51 UTC
[GitHub] [qpid-broker-j] Dedeepya-T opened a new pull request #99: Qpid 8546 implement slow consumers fix
Dedeepya-T opened a new pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99
Consumer is slow on a BDB HA environment, due to synchronous execution of remove messages. Made it asynchronous by adding a new context variable to pass the transaction sync durability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665984594
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -346,19 +342,36 @@ public void commit(final Transaction tx, boolean syncCommit)
}
- @Override
- public <X> ListenableFuture<X> commitAsync(final Transaction tx, final X val)
+ public void commitInternal(final Transaction tx, final Durability realMessageStoreDurability)
Review comment:
For Unit tests i made it public.. however the unit testing seems less practical here. So making it private
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r669706637
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1206,75 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ @Test
+ public void testNodeCommitNoSync() throws Exception
+ {
+ DatabaseConfig createConfig = new DatabaseConfig();
+ createConfig.setAllowCreate(true);
+ createConfig.setTransactional(true);
+
+ TestStateChangeListener masterListener = new TestStateChangeListener();
+ ReplicatedEnvironmentFacade node1 = addNode(TEST_NODE_NAME, TEST_NODE_HOST_PORT, true, masterListener, new NoopReplicationGroupListener(),
+ "NO_SYNC,NO_SYNC,NONE");
+ assertTrue("Environment was not created", masterListener.awaitForStateChange(State.MASTER,
+ _timeout, TimeUnit.SECONDS));
+
+ String replicaNodeHostPort = "localhost:" + _portHelper.getNextAvailable();
+ String replicaName = TEST_NODE_NAME + 1;
+ ReplicatedEnvironmentFacade node2 = createReplica(replicaName, replicaNodeHostPort, new NoopReplicationGroupListener());
+
+ Database db = node1.openDatabase("mydb", createConfig);
+
+ // Put a record (using commitNoSync)
+ DatabaseEntry key = new DatabaseEntry();
+ DatabaseEntry data = new DatabaseEntry();
+
+ TransactionConfig transactionConfig = new TransactionConfig();
+ transactionConfig.setDurability(node1.getRealMessageStoreDurability());
+
+ Transaction txn = node1.beginTransaction(null);
+ IntegerBinding.intToEntry(1, key);
+ StringBinding.stringToEntry("value", data);
+
+ db.put(txn, key, data);
+ node1.commitNoSync(txn);
+ db.close();
+
+ // Stop node1
+ node1.close();
+ node2.close();
+
+ LOGGER.debug("RESTARTING " + TEST_NODE_NAME);
+
+ node1 = addNode(TEST_NODE_NAME, TEST_NODE_HOST_PORT, true, masterListener, new NoopReplicationGroupListener(),
+ null);
+ boolean awaitForStateChange = masterListener.awaitForStateChange(State.MASTER,
+ _timeout, TimeUnit.SECONDS);
+ LOGGER.debug("RESTARTING " + replicaName);
+ TestStateChangeListener node2StateChangeListener = new TestStateChangeListener();
+ node2 = addNode(replicaName, replicaNodeHostPort, false, node2StateChangeListener, new NoopReplicationGroupListener(),
+ null);
+ db = node1.openDatabase("mydb", DatabaseConfig.DEFAULT);
+ txn = node1.beginTransaction(transactionConfig);
Review comment:
Done
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1206,75 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ @Test
+ public void testNodeCommitNoSync() throws Exception
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r666017161
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commit(_tx, false);
+
+ }
+
+ public void testCommitNoSync() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commitNoSync(_tx);
Review comment:
Remove the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665982047
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/AbstractBDBMessageStore.java
##########
@@ -351,7 +351,7 @@ void removeMessage(long messageId, boolean sync) throws StoreException
getLogger().debug("Deleted content for message {}", messageId);
- getEnvironmentFacade().commit(tx, sync);
+ getEnvironmentFacade().commitNoSync(tx, sync);
Review comment:
Done!
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/EnvironmentFacade.java
##########
@@ -98,4 +98,6 @@
Map<String,Object> getDatabaseStatistics(String database, boolean reset);
void deleteDatabase(String databaseName);
+
+ void commitNoSync(final Transaction tx, boolean sync);
Review comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665983436
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/StandardEnvironmentFacade.java
##########
@@ -183,6 +188,12 @@ public void commit(com.sleepycat.je.Transaction tx, boolean syncCommit)
_committer.commit(tx, syncCommit);
}
+ @Override
+ public void commitNoSync(final Transaction tx, boolean sync)
Review comment:
Removed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665983704
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -327,16 +332,7 @@ public Transaction beginTransaction(TransactionConfig transactionConfig)
@Override
public void commit(final Transaction tx, boolean syncCommit)
{
- try
Review comment:
Removed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665985376
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -346,19 +342,36 @@ public void commit(final Transaction tx, boolean syncCommit)
}
- @Override
- public <X> ListenableFuture<X> commitAsync(final Transaction tx, final X val)
+ public void commitInternal(final Transaction tx, final Durability realMessageStoreDurability)
{
try
{
// Using commit() instead of commitNoSync() for the HA store to allow
// the HA durability configuration to influence resulting behaviour.
- tx.commit(_realMessageStoreDurability);
+ tx.commit(realMessageStoreDurability);
}
catch (DatabaseException de)
{
throw handleDatabaseException("Got DatabaseException on commit, closing environment", de);
}
+ }
+
+ @Override
+ public void commitNoSync(final Transaction tx, boolean sync)
Review comment:
removed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r666016930
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
Review comment:
Remove the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r665983295
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/StandardEnvironmentFacade.java
##########
@@ -167,6 +167,11 @@ public Transaction beginTransaction(TransactionConfig transactionConfig)
@Override
public void commit(com.sleepycat.je.Transaction tx, boolean syncCommit)
+ {
Review comment:
Removed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r669705848
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1315,10 +1406,14 @@ private ReplicatedEnvironmentFacade addNode(StateChangeListener stateChangeListe
}
private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName, String nodeHostPort, boolean designatedPrimary)
{
- return createReplicatedEnvironmentConfiguration(nodeName,nodeHostPort,designatedPrimary,false);
+ return createReplicatedEnvironmentConfiguration(nodeName, nodeHostPort, designatedPrimary, false, null);
}
- private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName, String nodeHostPort, boolean designatedPrimary,final boolean disableCoalescing)
+ private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName,
Review comment:
changed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] asfgit closed pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] alex-rufous commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
alex-rufous commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r664695504
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/EnvironmentFacade.java
##########
@@ -98,4 +98,6 @@
Map<String,Object> getDatabaseStatistics(String database, boolean reset);
void deleteDatabase(String databaseName);
+
+ void commitNoSync(final Transaction tx, boolean sync);
Review comment:
sync flag is redundant in methods commitNoSync and commit
the sync flag should be always 'false' for 'commitNoSync' and 'true' for commit
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/StandardEnvironmentFacade.java
##########
@@ -167,6 +167,11 @@ public Transaction beginTransaction(TransactionConfig transactionConfig)
@Override
public void commit(com.sleepycat.je.Transaction tx, boolean syncCommit)
+ {
Review comment:
syncCommit parameter requires removal
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/AbstractBDBMessageStore.java
##########
@@ -351,7 +351,7 @@ void removeMessage(long messageId, boolean sync) throws StoreException
getLogger().debug("Deleted content for message {}", messageId);
- getEnvironmentFacade().commit(tx, sync);
+ getEnvironmentFacade().commitNoSync(tx, sync);
Review comment:
sync parameter needs to be removed from both commitNoSync and removeMessage.
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/StandardEnvironmentFacade.java
##########
@@ -183,6 +188,12 @@ public void commit(com.sleepycat.je.Transaction tx, boolean syncCommit)
_committer.commit(tx, syncCommit);
}
+ @Override
+ public void commitNoSync(final Transaction tx, boolean sync)
Review comment:
sync parameter requires removal
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commit(_tx, false);
Review comment:
This test does not really a test commit functionality.
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commit(_tx, false);
+
+ }
+
+ public void testCommitNoSync() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commitNoSync(_tx);
Review comment:
the same as above. This test does not really a test commit functionality.
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -346,19 +342,36 @@ public void commit(final Transaction tx, boolean syncCommit)
}
- @Override
- public <X> ListenableFuture<X> commitAsync(final Transaction tx, final X val)
+ public void commitInternal(final Transaction tx, final Durability realMessageStoreDurability)
Review comment:
the visibility for the method should be private
please rename `realMessageStoreDurability` into `transactionDurability`
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -346,19 +342,36 @@ public void commit(final Transaction tx, boolean syncCommit)
}
- @Override
- public <X> ListenableFuture<X> commitAsync(final Transaction tx, final X val)
+ public void commitInternal(final Transaction tx, final Durability realMessageStoreDurability)
{
try
{
// Using commit() instead of commitNoSync() for the HA store to allow
// the HA durability configuration to influence resulting behaviour.
- tx.commit(_realMessageStoreDurability);
+ tx.commit(realMessageStoreDurability);
}
catch (DatabaseException de)
{
throw handleDatabaseException("Got DatabaseException on commit, closing environment", de);
}
+ }
+
+ @Override
+ public void commitNoSync(final Transaction tx, boolean sync)
Review comment:
sync parameter has to be removed
##########
File path: tatus
##########
@@ -0,0 +1,91 @@
+[1mdiff --git a/bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/EnvironmentFacade.java b/bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/EnvironmentFacade.java[m
Review comment:
this is an unexpected file. Please delete
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
Review comment:
Annotation @Test is missed
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -327,16 +332,7 @@ public Transaction beginTransaction(TransactionConfig transactionConfig)
@Override
public void commit(final Transaction tx, boolean syncCommit)
{
- try
Review comment:
Parameter syncCommit needs to be removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r666017054
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1202,36 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ public void testCommit() throws Exception
+ {
+ ReplicatedEnvironmentFacade master = createMaster();
+ assertEquals("Unexpected message store durability",
+ new Durability(Durability.SyncPolicy.NO_SYNC, Durability.SyncPolicy.NO_SYNC, Durability.ReplicaAckPolicy.SIMPLE_MAJORITY),
+ master.getRealMessageStoreDurability());
+ assertEquals("Unexpected durability", TEST_DURABILITY, master.getMessageStoreDurability());
+ assertTrue("Unexpected coalescing sync", master.isCoalescingSync());
+
+ doNothing().when(_tx).commit();
+ doNothing().when(_coalescingCommiter).commit(Mockito.any(), Mockito.anyBoolean());
+ master.commit(_tx, false);
Review comment:
Remove the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] alex-rufous commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
alex-rufous commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r667558511
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -103,6 +104,7 @@
private static final int DEFAULT_ENVIRONMENT_RESTART_RETRY_LIMIT = 3;
private static final int DEFAULT_EXECUTOR_SHUTDOWN_TIMEOUT = 5000;
private static final boolean DEFAULT_DISABLE_COALESCING_COMMITTER = false;
+ private static final String DEFAULT_NO_SYNC_TX_DURABILITY_PROPERTY_NAME = "NO_SYNC,NO_SYNC,NONE";
Review comment:
Dedeepya,
Do you know whether durability SYNC,NO_SYNC,NONE gives the same performance?
I think that using SYNC,NO_SYNC,NONE without coalescing committer and NO_SYNC,NO_SYNC,NONE with a coalescing committer might be a better approach, as a transaction would be at least synced locally...
What do you think?
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1206,75 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ @Test
+ public void testNodeCommitNoSync() throws Exception
Review comment:
please move code to populate and read test db into a separate method. Something like below
private void addTestKeyValue(Database db, Transaction txn, int key, String value)
private String getTestKeyValue(Database db, int key)
That should improve test readability..
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1315,10 +1406,14 @@ private ReplicatedEnvironmentFacade addNode(StateChangeListener stateChangeListe
}
private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName, String nodeHostPort, boolean designatedPrimary)
{
- return createReplicatedEnvironmentConfiguration(nodeName,nodeHostPort,designatedPrimary,false);
+ return createReplicatedEnvironmentConfiguration(nodeName, nodeHostPort, designatedPrimary, false, null);
}
- private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName, String nodeHostPort, boolean designatedPrimary,final boolean disableCoalescing)
+ private ReplicatedEnvironmentConfiguration createReplicatedEnvironmentConfiguration(String nodeName,
Review comment:
Please revert new durability parameter. I do not think we need to change it in any test...
IMHO, it should be sufficient to set it to NO_SYNC,NO_SYNC,NONE in createReplicatedEnvironmentConfiguration.
It is unlikely that another durability would be tested.
The changes caused by an introduction of durability parameter can be reverted as well...
##########
File path: bdbstore/src/test/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacadeTest.java
##########
@@ -1196,6 +1206,75 @@ public void onNoMajority()
masterListener.awaitForStateChange(State.MASTER, _timeout, TimeUnit.SECONDS));
}
+ @Test
+ public void testNodeCommitNoSync() throws Exception
+ {
+ DatabaseConfig createConfig = new DatabaseConfig();
+ createConfig.setAllowCreate(true);
+ createConfig.setTransactional(true);
+
+ TestStateChangeListener masterListener = new TestStateChangeListener();
+ ReplicatedEnvironmentFacade node1 = addNode(TEST_NODE_NAME, TEST_NODE_HOST_PORT, true, masterListener, new NoopReplicationGroupListener(),
+ "NO_SYNC,NO_SYNC,NONE");
+ assertTrue("Environment was not created", masterListener.awaitForStateChange(State.MASTER,
+ _timeout, TimeUnit.SECONDS));
+
+ String replicaNodeHostPort = "localhost:" + _portHelper.getNextAvailable();
+ String replicaName = TEST_NODE_NAME + 1;
+ ReplicatedEnvironmentFacade node2 = createReplica(replicaName, replicaNodeHostPort, new NoopReplicationGroupListener());
+
+ Database db = node1.openDatabase("mydb", createConfig);
+
+ // Put a record (using commitNoSync)
+ DatabaseEntry key = new DatabaseEntry();
+ DatabaseEntry data = new DatabaseEntry();
+
+ TransactionConfig transactionConfig = new TransactionConfig();
+ transactionConfig.setDurability(node1.getRealMessageStoreDurability());
+
+ Transaction txn = node1.beginTransaction(null);
+ IntegerBinding.intToEntry(1, key);
+ StringBinding.stringToEntry("value", data);
+
+ db.put(txn, key, data);
+ node1.commitNoSync(txn);
+ db.close();
+
+ // Stop node1
+ node1.close();
+ node2.close();
+
+ LOGGER.debug("RESTARTING " + TEST_NODE_NAME);
+
+ node1 = addNode(TEST_NODE_NAME, TEST_NODE_HOST_PORT, true, masterListener, new NoopReplicationGroupListener(),
+ null);
+ boolean awaitForStateChange = masterListener.awaitForStateChange(State.MASTER,
+ _timeout, TimeUnit.SECONDS);
+ LOGGER.debug("RESTARTING " + replicaName);
+ TestStateChangeListener node2StateChangeListener = new TestStateChangeListener();
+ node2 = addNode(replicaName, replicaNodeHostPort, false, node2StateChangeListener, new NoopReplicationGroupListener(),
+ null);
+ db = node1.openDatabase("mydb", DatabaseConfig.DEFAULT);
+ txn = node1.beginTransaction(transactionConfig);
Review comment:
This transaction is redundant. I think null can be passed into the get method as a transaction. Removal transaction creation/committing for a read operation should simplify the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
[GitHub] [qpid-broker-j] Dedeepya-T commented on a change in pull request #99: Qpid 8546 implement slow consumers fix
Posted by GitBox <gi...@apache.org>.
Dedeepya-T commented on a change in pull request #99:
URL: https://github.com/apache/qpid-broker-j/pull/99#discussion_r669611579
##########
File path: bdbstore/src/main/java/org/apache/qpid/server/store/berkeleydb/replication/ReplicatedEnvironmentFacade.java
##########
@@ -103,6 +104,7 @@
private static final int DEFAULT_ENVIRONMENT_RESTART_RETRY_LIMIT = 3;
private static final int DEFAULT_EXECUTOR_SHUTDOWN_TIMEOUT = 5000;
private static final boolean DEFAULT_DISABLE_COALESCING_COMMITTER = false;
+ private static final String DEFAULT_NO_SYNC_TX_DURABILITY_PROPERTY_NAME = "NO_SYNC,NO_SYNC,NONE";
Review comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org