You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "zengguan (via GitHub)" <gi...@apache.org> on 2023/06/12 14:46:19 UTC

[GitHub] [pulsar-client-go] zengguan opened a new pull request, #1029: [Issue 1027][producer] fix: add read-write lock to makesure pendingQueue sending order

zengguan opened a new pull request, #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029

   Fixes #1027
   
   ### Motivation
   
   fix send timeout error cause by reconnection failures
   
   ### Modifications
   
   After #691 , client avoid blocking the reconnection logic, but it also make reconnection and internalSend/internalFlush have concurrency issues。when the client connection is block, it run into `p.grabCnx()` to get new connection and resend pendingQueue messages. At the same time, the internalSend/internalFlush keep running by use pendingQueue and  send mssage by `p._getConn().WriteData`. Therefore, it received an ack larger than expected, and trigger the ConnectionClosed logic, then the client entry `p.reconnectToBroker()` and repeat the concurrency issues with internalSend/internalFlush, fall into reconnect-resend-
   
   ### Verifying this change
   
   - [x] Make sure that the change passes the CI checks.
   
   ### Does this pull request potentially affect one of the following parts:
   
   *If `yes` was chosen, please highlight the changes*
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API: (no)
     - The schema: (no)
     - The default values of configurations: (no)
     - The wire protocol: (no)
   
   ### Documentation
   
     - Does this pull request introduce a new feature? (no)
     - If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented)
     - If a feature is not applicable for documentation, explain why?
     - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] zengguan commented on pull request #1029: [Issue 1027][producer] fix: add read-write lock to avoid continuous reconnection

Posted by "zengguan (via GitHub)" <gi...@apache.org>.
zengguan commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1587559505

   BTW, I think @bschofield said in #697 is right.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] BewareMyPower commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1593052269

   Could you update your PR description since you don't use RWLock now?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] zengguan commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "zengguan (via GitHub)" <gi...@apache.org>.
zengguan commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1593972674

   > Could you update your PR description since you don't use RWLock now?
   
   done. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] zengguan commented on a diff in pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "zengguan (via GitHub)" <gi...@apache.org>.
zengguan commented on code in PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#discussion_r1230394670


##########
pulsar/producer_partition.go:
##########
@@ -150,7 +151,8 @@ func newPartitionProducer(client *client, topic string, options *ProducerOptions
 		log:              logger,
 		options:          options,
 		producerID:       client.rpcClient.NewProducerID(),
-		eventsChan:       make(chan interface{}, maxPendingMessages),
+		dataChan:         make(chan *sendRequest, maxPendingMessages),
+		cmdChan:          make(chan interface{}, 10),
 		connectClosedCh:  make(chan connectionClosed, 10),
 		closeCh:          make(chan struct{}),

Review Comment:
   yes. removed it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] BewareMyPower commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1593146982

   It seems the CI is broken.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] shibd commented on a diff in pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "shibd (via GitHub)" <gi...@apache.org>.
shibd commented on code in PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#discussion_r1230363891


##########
pulsar/producer_partition.go:
##########
@@ -150,7 +151,8 @@ func newPartitionProducer(client *client, topic string, options *ProducerOptions
 		log:              logger,
 		options:          options,
 		producerID:       client.rpcClient.NewProducerID(),
-		eventsChan:       make(chan interface{}, maxPendingMessages),
+		dataChan:         make(chan *sendRequest, maxPendingMessages),
+		cmdChan:          make(chan interface{}, 10),
 		connectClosedCh:  make(chan connectionClosed, 10),
 		closeCh:          make(chan struct{}),

Review Comment:
   In this way, we don't need `closeCh` anymore



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] shibd commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "shibd (via GitHub)" <gi...@apache.org>.
shibd commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1592266966

   @BewareMyPower @gaoran10 Can you help take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] zengguan commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "zengguan (via GitHub)" <gi...@apache.org>.
zengguan commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1590827040

   @shibd great idea ! now I split sendRequest and changed `eventsChan` to `dataChan` and `cmdChan` to handle `sendRequest` and `flushRequest`/`closeProducer` separately
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] shibd commented on pull request #1029: [Issue 1027][producer] fix: add read-write lock to avoid continuous reconnection

Posted by "shibd (via GitHub)" <gi...@apache.org>.
shibd commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1589370856

   @zengguan Thanks for your contribute.
   
   I have a question, As #687. Can we let the `sendRequest` event alone in one channel?
   
   In this way, Let `reconnectToBroker` and other operate are kept in the same coroutine.
   
   ```go
   	for {
   		select {
   		case sendRequest <-p.sendRequestChan:
   		    p.internalSend(sendRequest)
   		case i := <-p.eventsChan:
   			switch v := i.(type) {
   			case *flushRequest:
   				p.internalFlush(v)
   			case *closeProducer:
   				p.internalClose(v)
   				return
   			}
   		case <-p.connectClosedCh:
   			p.reconnectToBroker()
   		case <-p.batchFlushTicker.C:
   			if p.batchBuilder.IsMultiBatches() {
   				p.internalFlushCurrentBatches()
   			} else {
   				p.internalFlushCurrentBatch()
   			}
   		}
   	}
   ```
   
   I think we can discuss clearly here, using channels should avoid using locks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] BewareMyPower commented on pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029#issuecomment-1594127112

   Please rebase to master to have tests fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar-client-go] shibd merged pull request #1029: [Issue 1027][producer] fix: split sendRequest and make reconnectToBroker and other operate in the same coroutine

Posted by "shibd (via GitHub)" <gi...@apache.org>.
shibd merged PR #1029:
URL: https://github.com/apache/pulsar-client-go/pull/1029


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org