You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/04/09 06:29:04 UTC
[GitHub] [pulsar] congbobo184 opened a new pull request #6697: Fix
filesystem offload oom
congbobo184 opened a new pull request #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697
### Motivation
to fix filesystem offload oom #6692
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes
Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)
### Documentation
Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #6697: Fix filesystem
offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-611869211
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #6697: Fix filesystem
offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-611420831
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] pheecian commented on issue #6697: Fix filesystem offload
oom
Posted by GitBox <gi...@apache.org>.
pheecian commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-612391818
quote from @congbobo184 "so we maybe should control the read-ahead number" this is what my PR want to do
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #6697: Fix filesystem
offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-611918600
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 commented on issue #6697: Fix filesystem
offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-612358562
> @congbobo184 thanks for your pull request. I have applied the modification. However, the OOM still occurs.
> <img alt="Screen Shot 2020-04-10 at 3 10 44 PM" width="1429" src="https://user-images.githubusercontent.com/8122303/78971203-a2a27a00-7b3d-11ea-8722-c5be2c7ff64c.png">
I think when one entry be written and then release it, if
> @congbobo184 thanks for your pull request. I have applied the modification. However, the OOM still occurs.
> <img alt="Screen Shot 2020-04-10 at 3 10 44 PM" width="1429" src="https://user-images.githubusercontent.com/8122303/78971203-a2a27a00-7b3d-11ea-8722-c5be2c7ff64c.png">
I think you offload to hdfs is to slowly, and then you produce message is too fast. and then many ledger will offload int the same time. so we maybe should control the read-ahead number or the whole fileSystem offload task number
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #6697: Fix filesystem offload oom
Posted by GitBox <gi...@apache.org>.
tuteng commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-612709880
Add label release/2.5.1 and remove label release/2.5.2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 edited a comment on issue #6697: Fix
filesystem offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 edited a comment on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-612358562
> @congbobo184 thanks for your pull request. I have applied the modification. However, the OOM still occurs.
> <img alt="Screen Shot 2020-04-10 at 3 10 44 PM" width="1429" src="https://user-images.githubusercontent.com/8122303/78971203-a2a27a00-7b3d-11ea-8722-c5be2c7ff64c.png">
I think you offload to hdfs is too slowly, and then you produce message is too fast. and then many ledger will offload int the same time. so we maybe should control the read-ahead number or the whole fileSystem offload task number
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] pheecian commented on issue #6697: Fix filesystem offload
oom
Posted by GitBox <gi...@apache.org>.
pheecian commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-611921216
@congbobo184 thanks for your pull request. I have applied the modification. However, the OOM still occurs.
<img width="1429" alt="Screen Shot 2020-04-10 at 3 10 44 PM" src="https://user-images.githubusercontent.com/8122303/78971203-a2a27a00-7b3d-11ea-8722-c5be2c7ff64c.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] congbobo184 closed pull request #6697: Fix filesystem
offload oom
Posted by GitBox <gi...@apache.org>.
congbobo184 closed pull request #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on issue #6697: Fix filesystem offload oom
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #6697: Fix filesystem offload oom
URL: https://github.com/apache/pulsar/pull/6697#issuecomment-611702613
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services