You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/05/16 10:50:46 UTC

[GitHub] [pulsar-client-go] WJL3333 opened a new pull request #249: feature: add send timeout (#242)

WJL3333 opened a new pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249


   Changes
   
   1. ProduceOptions
   add SendTimeout and SendTimeoutCheckInterval in ProducerOptions
   
   2. BlockingQueue
   unexported BlockingQueue.Iterator
   and add methods to avoid race condition.
   BlockingQueue.IterateIfNonEmpty and BlockingQueue.PollIfSatisfy
   
   3. partitionProducer
   add a ticker to check if pendingItem in pendingQueue has expired.
   
   Some explain on implementation:
   
   1. if message is send as batch, will only check if the first message timeout.
      if first message in batch timeout. will call all message send callback with an ErrSendTimeout.
   
   2. avoid use context to judge if send has been timeout.
      maybe one set different timeout on each message. this will introduce a lot complicity.
      and context is use for goroutine control. i think this is not suit for send timeout use case, because there is no goroutine at all.
   
   <--
   ### Contribution Checklist
     
     - Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where *XYZ* should be replaced by the actual issue number.
       Skip *Issue XYZ* if there is no associated github issue for this pull request.
       Skip *component* if you are unsure about which is the best component. E.g. `[docs] Fix typo in produce method`.
   
     - Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
     
     - Each pull request should address only one issue, not mix up code from multiple issues.
     
     - Each commit in the pull request has a meaningful commit message
   
     - Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
   
   **(The sections below can be removed for hotfixes of typos)**
   -->
   
   *(If this PR fixes a github issue, please add `Fixes #<xyz>`.)*
   
   Fixes #<xyz>
   
   *(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
   
   Master Issue: #<xyz>
   
   ### Motivation
   
   
   *Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
   
   ### Modifications
   
   *Describe the modifications you've done.*
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   *(Please pick either of the following options)*
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   *(or)*
   
   This change is already covered by existing tests, such as *(please describe tests)*.
   
   *(or)*
   
   This change added tests and can be verified as follows:
   
   *(example:)*
     - *Added integration tests for end-to-end deployment with large payloads (10MB)*
     - *Extended integration test for recovery after broker failure*
   
   ### Does this pull request potentially affect one of the following parts:
   
   *If `yes` was chosen, please highlight the changes*
   
     - Dependencies (does it add or upgrade a dependency): (yes / no)
     - The public API: (yes / no)
     - The schema: (yes / no / don't know)
     - The default values of configurations: (yes / no)
     - The wire protocol: (yes / no)
   
   ### Documentation
   
     - Does this pull request introduce a new feature? (yes / no)
     - If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented)
     - If a feature is not applicable for documentation, explain why?
     - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 edited a comment on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 edited a comment on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629684905


   i'm confused about the above use case, need some guide on the real popuse of passing an context here. and where do you prefer to check the context? just before send or both before send and time 
   wait for response.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629684905


   i'm confused about the above use case, need some guide on the real popuse of passing an context here. and where do you prefer to check the context? just before send or both before send and waiting for response.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 edited a comment on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 edited a comment on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629684905


   i'm confused about the above use case, need some guide on the real popuse of passing an context here. and where do you prefer to check the context? just before send or both before send and time 
    during wait for response.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629815523


   closed and goto https://github.com/apache/pulsar-client-go/pull/252


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] merlimat commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629654609


   The `Send()` is already taking a `Context` object. If the context has a timeout, it will be used.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] yarthur1 commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
yarthur1 commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629813082


   I hope that both before send and time wait for response should check timeout.I don't want to wait send func always,because I meet this situation where pulsar broker has something wrong and send func always wait @


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629678043


   yeah, i think it should be, but when user pass different context with different sendtimeout to Send and SendAsync , this will introduce a lot of complicity. the message is send as batch. and each message in batch with different timeout


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] WJL3333 closed pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
WJL3333 closed pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar-client-go] merlimat commented on pull request #249: [Issue 242][pulsar-client-go] feature: add send timeout

Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #249:
URL: https://github.com/apache/pulsar-client-go/pull/249#issuecomment-629654825


   I think the correct fix for #242 would be to also check the context  **before** attempting to publish the message.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org