You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/03/11 10:52:36 UTC
[GitHub] [hadoop-ozone] elek commented on issue #551: HDDS-2717. Handle
chunk increments in datanode
elek commented on issue #551: HDDS-2717. Handle chunk increments in datanode
URL: https://github.com/apache/hadoop-ozone/pull/551#issuecomment-597565621
I tested with this path with teragen (100GB) and compared the numbers with the current master.
1. I see high variance in the test (same test can provide +/- 10% performance at a different time)
2. During my first test (used an old version of master / old version of this branch) the patch was slower: https://hackmd.io/HbYNWTE2SXyOgZQ6CeXaXg
3. Yesterday I tested the final version and couldn't reproduce any slowness: https://hackmd.io/P3Hkf09fTLS78Ho4W82YbQ
* This patch always provided better performance than the master
4. Using load tests (freon) instead of teragen I have seen significant better performance. The current implementation uses a lot of unnecessary file.exists check which makes impossible to get better performance than 1.2 k / sec chunk write. This patch (even with the old layout) can easily provide > 4k without any adjustment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org