You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by mm...@apache.org on 2019/05/02 18:37:19 UTC
[bookkeeper] branch master updated: Use pure python implementation
of MurmurHash
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push:
new 973d2ab Use pure python implementation of MurmurHash
973d2ab is described below
commit 973d2ab0da91e5a24b1d30990ca6bb1484f07f7e
Author: Matteo Merli <mm...@apache.org>
AuthorDate: Thu May 2 11:37:14 2019 -0700
Use pure python implementation of MurmurHash
### Motivation
BK table python client is depending on `mmh3` library for MurmurHash. This library wraps a C based function but there are no binaries published on PyPI. Therefore users need to have a GCC installed in order to install the BK client lib, since it compiles it at install time. GCC is typically not available in Docker containers.
### Modifications
Include a pure python implementation of MurmurHash to use if the C based is not found.
Notes:
* I couldn't find any published pure-python MurmurHash implementations on PyPI
* Importing public-domain code is permitted by ASF
Reviewers: Ivan Kelly <iv...@apache.org>, Enrico Olivelli <eo...@gmail.com>
This closes #2069 from merlimat/mmh3
---
stream/clients/python/bookkeeper/common/router/router.py | 7 ++++++-
stream/clients/python/setup.py | 2 +-
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/stream/clients/python/bookkeeper/common/router/router.py b/stream/clients/python/bookkeeper/common/router/router.py
index de4f31d..f52ef64 100644
--- a/stream/clients/python/bookkeeper/common/router/router.py
+++ b/stream/clients/python/bookkeeper/common/router/router.py
@@ -10,7 +10,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-import mmh3
+try:
+ # Try with C based implemenation if available
+ import mmh3
+except ImportError:
+ # Fallback to pure python
+ import pymmh3 as mmh3
__SEED__ = 383242705
diff --git a/stream/clients/python/setup.py b/stream/clients/python/setup.py
index fb8a696..2b205dc 100644
--- a/stream/clients/python/setup.py
+++ b/stream/clients/python/setup.py
@@ -33,7 +33,7 @@ dependencies = [
'pytz',
'futures>=3.2.0;python_version<"3.2"',
'grpcio>=1.8.2',
- 'mmh3>=2.5.1'
+ 'pymmh3>=0.0.3'
]
extras = {
}