You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by GitBox <gi...@apache.org> on 2020/09/04 02:00:49 UTC
[GitHub] [kafka-site] ableegoldman opened a new pull request #301: Redo 2.6 docs commit
ableegoldman opened a new pull request #301:
URL: https://github.com/apache/kafka-site/pull/301
Just noticed that all of the docs I added just before 2.6 aren't actually showing up on the site. Seems like they were accidentally reverted by [this commit](https://github.com/apache/kafka-site/commit/35d3b804b86c249aace9a729c5abd9be4525ef3d)
So, this PR just redoes everything that was originally in https://github.com/apache/kafka-site/commit/4c19eab4f6315c91549be0bff08ad46173be4f64
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka-site] ableegoldman commented on a change in pull request #301: Redo 2.6 docs commit
Posted by GitBox <gi...@apache.org>.
ableegoldman commented on a change in pull request #301:
URL: https://github.com/apache/kafka-site/pull/301#discussion_r485154723
##########
File path: 26/streams/developer-guide/memory-mgmt.html
##########
@@ -202,7 +202,10 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#r
<span class="o">}</span>
<span class="o">}</span>
</div>
- <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>. </sup>
+ <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>.
+ NOTE: the boolean parameter in the cache constructor lets you control whether the cache should enforce a strict memory limit by failing the read or iteration in the rare cases where it might go larger than its capacity. Due to a
+ <a class="reference external" href="https://github.com/facebook/rocksdb/issues/6247">bug in RocksDB</a>, this option cannot be used
+ if the write buffer memory is also counted against the cache. If you set this to true, you should NOT pass the cache in to the <code>WriteBufferManager</code> and just control the write buffer and cache memory separately.</sup>
Review comment:
Nope, this is correct. My original commit had a few other fixes on the side in addition to the KIP-441/KIP-613 docs
As far as I know this has always been the case in every RocksDB version we've depended on but didn't discover the bug until just recently. (I believe it actually is fixed in some newer versions, so we may be able to remove this again once we upgrade)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka-site] vvcephei merged pull request #301: Redo 2.6 docs commit
Posted by GitBox <gi...@apache.org>.
vvcephei merged pull request #301:
URL: https://github.com/apache/kafka-site/pull/301
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka-site] vvcephei commented on a change in pull request #301: Redo 2.6 docs commit
Posted by GitBox <gi...@apache.org>.
vvcephei commented on a change in pull request #301:
URL: https://github.com/apache/kafka-site/pull/301#discussion_r485142442
##########
File path: 26/streams/developer-guide/memory-mgmt.html
##########
@@ -202,7 +202,10 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#r
<span class="o">}</span>
<span class="o">}</span>
</div>
- <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>. </sup>
+ <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>.
+ NOTE: the boolean parameter in the cache constructor lets you control whether the cache should enforce a strict memory limit by failing the read or iteration in the rare cases where it might go larger than its capacity. Due to a
+ <a class="reference external" href="https://github.com/facebook/rocksdb/issues/6247">bug in RocksDB</a>, this option cannot be used
+ if the write buffer memory is also counted against the cache. If you set this to true, you should NOT pass the cache in to the <code>WriteBufferManager</code> and just control the write buffer and cache memory separately.</sup>
Review comment:
It looks like this (and some other stuff in the config-streams file) has come along for the ride. We should make sure it's referencing behavior that was released, not behavior that will be released next. Can you double-check that this actually is in reference to 2.6.0, and not 2.6.1+?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka-site] ableegoldman commented on a change in pull request #301: Redo 2.6 docs commit
Posted by GitBox <gi...@apache.org>.
ableegoldman commented on a change in pull request #301:
URL: https://github.com/apache/kafka-site/pull/301#discussion_r485154723
##########
File path: 26/streams/developer-guide/memory-mgmt.html
##########
@@ -202,7 +202,10 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a class="headerlink" href="#r
<span class="o">}</span>
<span class="o">}</span>
</div>
- <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>. </sup>
+ <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of the block cache to set aside for "high priority" (aka index and filter) blocks, preventing them from being evicted by data blocks. See the full signature of the <a class="reference external" href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72">LRUCache constructor</a>.
+ NOTE: the boolean parameter in the cache constructor lets you control whether the cache should enforce a strict memory limit by failing the read or iteration in the rare cases where it might go larger than its capacity. Due to a
+ <a class="reference external" href="https://github.com/facebook/rocksdb/issues/6247">bug in RocksDB</a>, this option cannot be used
+ if the write buffer memory is also counted against the cache. If you set this to true, you should NOT pass the cache in to the <code>WriteBufferManager</code> and just control the write buffer and cache memory separately.</sup>
Review comment:
Nope, this is correct. As far as I know this has always been the case in every RocksDB version we've depended on but didn't discover the bug until just recently. (I believe it actually is fixed in some newer versions, so we may be able to remove this again once we upgrade)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org