You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2020/10/17 02:29:22 UTC
[GitHub] [rocketmq-client-python] tom0392 opened a new pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
tom0392 opened a new pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100
Resolves #99
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] tom0392 commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
tom0392 commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711840093
@ifplusor Hi, Could you gvie some specific reasons for further discussion?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] ifplusor removed a comment on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
ifplusor removed a comment on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711464119
@vongosling I think this is useless, why merge it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] tom0392 edited a comment on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
tom0392 edited a comment on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711867417
@messense Hi, this is indeed a special scene. However, in the production debugging and test environment, we encountered this problem repeatedly and we think the submitted code is one way to avoid the errors after the actual test verification. Of course, we look forward to other methods.
> First suggestion
When the producer sends a message successfully, the program will return SendStatus.OK=0 to the ret variable. So we think when the producer fails to send a message, the program should return an error code to the ret accordingly and not to throw an exception to prevent the thread from running the rest of code, especially the following step:
producer.shutdown()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] messense commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
messense commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711469836
No real world applications should do this kind of hack.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] codecov-io commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-710730897
# [Codecov](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=h1) Report
> Merging [#100](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=desc) into [master](https://codecov.io/gh/apache/rocketmq-client-python/commit/58397dc86d85d1b16a4d949c430e7499722050b5?el=desc) will **not change** coverage.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #100 +/- ##
=======================================
Coverage 82.14% 82.14%
=======================================
Files 4 4
Lines 532 532
=======================================
Hits 437 437
Misses 95 95
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=footer). Last update [58397dc...89ad6bd](https://codecov.io/gh/apache/rocketmq-client-python/pull/100?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] ifplusor commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
ifplusor commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711877456
send_sync is thread-safe, protect it with mutex is unnecessary.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] tom0392 commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
tom0392 commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711966230
@messense Thanks for your good suggestions! Our application scenario is to send data once every 5 minutes. Maybe this frequency is acceptable to start() and shutdown() every time. As long as the customer's business is running, it needs to keep sending data. So we use apscheduler module:
`from apscheduler.schedulers.background import BackgroundScheduler
`
and make the program a systemctl service. So far, the service is running stable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] messense commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
messense commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711877593
Producer should be reused, not start() and shutdown() repeatedly.
For example
```python
from concurrent.futures import ThreadPoolExecutor, wait
from rocketmq.client import Producer, Message
producer = Producer('PID-XXX')
producer.set_name_server_address('127.0.0.1:9876')
def send_msg():
msg = Message('YOUR-TOPIC')
msg.set_keys('XXX')
msg.set_tags('XXX')
msg.set_body('XXXX')
ret = producer.send_sync(msg)
print(ret.status, ret.msg_id, ret.offset)
def main():
executor = ThreadPoolExecutor()
# start producer
producer.start()
# submit tasks
futures = []
for _ in range(10):
futures.append(executor.submit(send_msg))
wait(futures)
producer.shutdown()
if __name__ == '__main__':
main()
````
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] tom0392 commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
tom0392 commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711972983
> send_sync is thread-safe, protect it with mutex is unnecessary.
@ifplusor Thank you for the same. We will test it without mutex.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] ifplusor commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
ifplusor commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711874793
You should report the error of sending, but call shutdown() is unrecommended.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] ifplusor commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
ifplusor commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711464119
@vongosling I think this is useless, why merge it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] ifplusor commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
ifplusor commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711870546
First, don't call start() and shutdown() under multi-threads; secondly, C++ SDK has retry logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] tom0392 commented on pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
tom0392 commented on pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100#issuecomment-711867417
This is indeed a special scene. However, in the production debugging and test environment, we encountered this problem repeatedly and we think the submitted code is one way to avoid the errors after the actual test verification. Of course, we look forward to other methods.
> First suggestion
When the producer sends a message successfully, the program will return SendStatus.OK=0 to the ret variable. So we think when the producer fails to send a message, the program should return an error code to the ret accordingly and not to throw an exception to prevent the thread from running the rest of code, especially the following step:
producer.shutdown()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-python] vongosling merged pull request #100: [ISSUE #99]Add a function which shows how to use rocketmq in multi-threaded scen…
Posted by GitBox <gi...@apache.org>.
vongosling merged pull request #100:
URL: https://github.com/apache/rocketmq-client-python/pull/100
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org