You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Blair Zajac <bl...@orcaware.com> on 2012/09/22 20:12:52 UTC
Re: svn commit: r1388816 - in /subversion/branches/10Gb/subversion:
include/private/svn_pseudo_md5.h libsvn_subr/cache-membuffer.c libsvn_subr/pseudo_md5.c
tests/libsvn_subr/checksum-test.c
On 09/22/2012 08:14 AM, stefan2@apache.org wrote:
> Author: stefan2
> Date: Sat Sep 22 15:14:25 2012
> New Revision: 1388816
>
> URL: http://svn.apache.org/viewvc?rev=1388816&view=rev
> Log:
> On the 10Gb branch: Introduce MD5-based hash functions optimized
> for short input lengths. Use these to speed up membuffer access.
How much faster is it than a plain MD5?
If we only need it for hashing, did you look at using a more well known
hashing function, e.g. FNV [1] or murmur [2]?
Also, can you include URLs where you downloaded the code from in the log
message and code.
Blair
[1] http://isthe.com/chongo/tech/comp/fnv/
[2] https://sites.google.com/site/murmurhash/
Re: svn commit: r1388816 - in /subversion/branches/10Gb/subversion:
include/private/svn_pseudo_md5.h libsvn_subr/cache-membuffer.c
libsvn_subr/pseudo_md5.c tests/libsvn_subr/checksum-test.c
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sun, Sep 23, 2012 at 2:58 PM, Stefan Sperling <st...@elego.de> wrote:
> On Sun, Sep 23, 2012 at 02:49:16PM +0200, Stefan Fuhrmann wrote:
> > I downloaded the original version from some admittedly
> > obscure location in the rather shady party of the interwebs:
> >
> >
> https://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.12/crypto/apr_md5.c
> >
> > -- Stefan^2.
>
> To comply with the licence of the code you've added you'll need to
> update the NOTICE file on your branch as well:
> https://svn.apache.org/repos/asf//subversion/branches/10Gb/NOTICE
>
> See https://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.12/NOTICE
>
Thanks for noticing. Committed in r1389044.
-- Stefan^2.
--
*
Join us this October at Subversion Live
2012<http://www.wandisco.com/svn-live-2012>
for two days of best practice SVN training, networking, live demos,
committer meet and greet, and more! Space is limited, so get signed up
today<http://www.wandisco.com/svn-live-2012>
!
*
Re: svn commit: r1388816 - in /subversion/branches/10Gb/subversion:
include/private/svn_pseudo_md5.h libsvn_subr/cache-membuffer.c
libsvn_subr/pseudo_md5.c tests/libsvn_subr/checksum-test.c
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Sep 23, 2012 at 02:49:16PM +0200, Stefan Fuhrmann wrote:
> I downloaded the original version from some admittedly
> obscure location in the rather shady party of the interwebs:
>
> https://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.12/crypto/apr_md5.c
>
> -- Stefan^2.
To comply with the licence of the code you've added you'll need to
update the NOTICE file on your branch as well:
https://svn.apache.org/repos/asf//subversion/branches/10Gb/NOTICE
See https://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.12/NOTICE
Re: svn commit: r1388816 - in /subversion/branches/10Gb/subversion:
include/private/svn_pseudo_md5.h libsvn_subr/cache-membuffer.c
libsvn_subr/pseudo_md5.c tests/libsvn_subr/checksum-test.c
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sat, Sep 22, 2012 at 8:12 PM, Blair Zajac <bl...@orcaware.com> wrote:
> On 09/22/2012 08:14 AM, stefan2@apache.org wrote:
>
>> Author: stefan2
>> Date: Sat Sep 22 15:14:25 2012
>> New Revision: 1388816
>>
>> URL: http://svn.apache.org/viewvc?**rev=1388816&view=rev<http://svn.apache.org/viewvc?rev=1388816&view=rev>
>> Log:
>> On the 10Gb branch: Introduce MD5-based hash functions optimized
>> for short input lengths. Use these to speed up membuffer access.
>>
>
> How much faster is it than a plain MD5?
>
The 16-byte version is twice as fast as the MD5 core
due to the fact that we know much of the input to be 0
and can hard-code it as such. In addition to that, the
APR implementation has a ~100% overhead (setting
up the context etc.) for strings that fit into a single
encoding block. In total, the pseudo-MD5 code is
3..4 times as fast as apr_md5.
> If we only need it for hashing, did you look at using a more well known
> hashing function, e.g. FNV [1] or murmur [2]?
>
FNV-128 would be just as slow as pseudo-MD5 as it
takes one iteration per byte and about 17(?) operations
per iteration.
murmur is not exactly "well-known" as it is quite new.
However, non of that really matters. The key is that we
need cryptographic strength for the hashes we use in
membuffer because they are the *only* identification for
any object stored therein. Basically the same scheme
as the SHA-1 usage in our working copy.
> Also, can you include URLs where you downloaded the code from in the log
> message and code.
>
So, you did not read the code ;) Simply read the first
65 lines of pseudo_md5.c
I downloaded the original version from some admittedly
obscure location in the rather shady party of the interwebs:
https://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.12/crypto/apr_md5.c
-- Stefan^2.
--
*
Join us this October at Subversion Live
2012<http://www.wandisco.com/svn-live-2012>
for two days of best practice SVN training, networking, live demos,
committer meet and greet, and more! Space is limited, so get signed up
today<http://www.wandisco.com/svn-live-2012>
!
*