You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jay Kreps (JIRA)" <ji...@apache.org> on 2013/01/29 04:21:12 UTC

[jira] [Created] (KAFKA-741) Improve log cleaning dedupe buffer efficiency

Jay Kreps created KAFKA-741:
-------------------------------

             Summary: Improve log cleaning dedupe buffer efficiency
                 Key: KAFKA-741
                 URL: https://issues.apache.org/jira/browse/KAFKA-741
             Project: Kafka
          Issue Type: Bug
            Reporter: Jay Kreps


Two good suggestions:
1. Use a probing scheme to increase density without increasing the collision rate
2. Only count unique updates to the offset map (i.e. if the key is all zero, don't count it) when computing the load. Dynamically choose the end offset based on when the map is full.

Would be good to investigate these things.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira