[GitHub] [pulsar] codelipenghui commented on a diff in pull request #21151: [improve][txn] Take first snapshot before persisting the first transactional message

["codelipenghui (via GitHub)" <gi...@apache.org>]

codelipenghui commented on code in PR #21151:
URL: https://github.com/apache/pulsar/pull/21151#discussion_r1325297006


##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/buffer/TopicTransactionBufferTest.java:
##########
@@ -179,4 +180,32 @@ public void testCloseTransactionBufferWhenTimeout() throws Exception {
         Assert.assertTrue(f.isCompletedExceptionally());
     }
 
+    // 1. The state should be NoSnapshot instead of Ready after building producer.
+    // 2. The state should be Ready after sending first transactional message.
+    @Test
+    public void testWriteSnapshotWhenFirstTxnMessageSend() throws Exception {
+        String topic = "persistent://" + NAMESPACE1 + "/testWriteSnapshotWhenFirstTxnMessageSend";
+        admin.topics().createNonPartitionedTopic(topic);
+        PersistentTopic persistentTopic = (PersistentTopic) pulsarServiceList.get(0).getBrokerService()
+                .getTopic(topic, false)
+                .get()
+                .get();
+        persistentTopic.checkIfTransactionBufferRecoverCompletely(true).get();
+        TopicTransactionBuffer topicTransactionBuffer = (TopicTransactionBuffer) persistentTopic.getTransactionBuffer();
+        Assert.assertEquals(topicTransactionBuffer.getState(), TopicTransactionBufferState.State.NoSnapshot);
+        @Cleanup
+        Producer<byte[]> producer = pulsarClient.newProducer()
+        .topic(topic)
+        .create();
+        producer.newMessage().send();
+        Assert.assertEquals(topicTransactionBuffer.getState(), TopicTransactionBufferState.State.NoSnapshot);
+
+        Transaction transaction = pulsarClient.newTransaction()
+                .withTransactionTimeout(5, TimeUnit.HOURS)
+                .build()
+                .get();
+        producer.newMessage(transaction).send();
+        Assert.assertEquals(topicTransactionBuffer.getState(), TopicTransactionBufferState.State.Ready);

Review Comment:
   It's better to check the transaction buffer can still be recovered after the topic has been unloaded.
   
   ```java
           MessageIdAdv messageId = (MessageIdAdv) producer.newMessage(transaction).send();
           Assert.assertEquals(topicTransactionBuffer.getState(), TopicTransactionBufferState.State.Ready);
           transaction.commit();
           admin.topics().unload(topic);
           PersistentTopic persistentTopic2 = (PersistentTopic) pulsarServiceList.get(0).getBrokerService()
                   .getTopic(topic, false)
                   .get()
                   .get();
           TopicTransactionBuffer topicTransactionBuffer2 = (TopicTransactionBuffer) persistentTopic2.getTransactionBuffer();
           Awaitility.await().untilAsserted(() -> {
               Assert.assertEquals(topicTransactionBuffer2.getState(), TopicTransactionBufferState.State.Ready);
               Assert.assertTrue(topicTransactionBuffer2.getMaxReadPosition()
                       .compareTo(PositionImpl.get(messageId.getLedgerId(), messageId.getEntryId())) > 0);
           });
   ```



##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java:
##########
@@ -255,6 +230,38 @@ public long getCommittedTxnCount() {
 
     @Override
     public CompletableFuture<Position> appendBufferToTxn(TxnID txnId, long sequenceId, ByteBuf buffer) {
+        return transactionBufferFuture.thenCompose(ignore -> {
+            if (checkIfNoSnapshot()) {
+                CompletableFuture<Void> completableFuture = new CompletableFuture<>();
+                snapshotAbortedTxnProcessor.takeAbortedTxnsSnapshot(maxReadPosition).thenRun(() -> {
+                    if (changeToReadyStateFromNoSnapshot()) {
+                        timer.newTimeout(TopicTransactionBuffer.this,
+                                takeSnapshotIntervalTime, TimeUnit.MILLISECONDS);
+                    } else {
+                        //This case should not happen.
+                        log.error("[{} ]Failed to change state of transaction buffer to Ready from NoSnapshot",
+                                topic.getName());
+                        completableFuture.completeExceptionally(new BrokerServiceException.ServiceUnitNotReadyException(
+                                "Transaction Buffer take first snapshot failed, the current state is: " + getState()));
+                    }
+                    completableFuture.complete(null);

Review Comment:
   A completableFuture cannot be completed twice.



##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/TransactionBuffer.java:
##########
@@ -188,12 +188,14 @@ public interface TransactionBuffer {
     TransactionBufferStats getStats(boolean lowWaterMarks);
 
     /**
-     * Wait TransactionBuffer Recovers completely.
-     * Take snapshot after TB Recovers completely.
+     * Wait TransactionBuffer recovers completely.
      * @param isTxn
-     * @return a future which has completely if isTxn = false. Or a future return by takeSnapshot.
+     * @return a future whether the transaction buffer recover completely.
      */
-    CompletableFuture<Void> checkIfTBRecoverCompletely(boolean isTxn);
+    @Deprecated

Review Comment:
   Can we remove this method directly? It's not a user-facing public API.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org