You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Greg Bowyer (JIRA)" <ji...@apache.org> on 2014/03/07 07:49:47 UTC
[jira] [Comment Edited] (LUCENE-3178) Native MMapDir
[ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564880#comment-13564880 ]
Greg Bowyer edited comment on LUCENE-3178 at 3/7/14 6:48 AM:
-------------------------------------------------------------
So I was going to shut this down today, and just to make sure I ran the benchmark on the simplest code possible
... and suddenly I got good results, this is idiopathic :S
{code}
Report after iter 19:
TaskQPS baseline StdDevQPS mmap_tests StdDev Pct diff
OrHighHigh 1.68 (11.2%) 1.73 (10.3%) 3.0% ( -16% - 27%)
PKLookup 129.89 (5.8%) 135.03 (6.0%) 4.0% ( -7% - 16%)
HighTerm 8.09 (13.6%) 8.43 (12.8%) 4.2% ( -19% - 35%)
OrHighMed 4.46 (10.4%) 4.67 (9.5%) 4.7% ( -13% - 27%)
OrHighLow 4.82 (10.6%) 5.09 (10.3%) 5.6% ( -13% - 29%)
HighSpanNear 0.92 (8.1%) 0.97 (7.3%) 5.9% ( -8% - 23%)
IntNRQ 2.51 (10.2%) 2.67 (9.9%) 6.6% ( -12% - 29%)
HighPhrase 0.30 (11.7%) 0.32 (12.8%) 6.7% ( -16% - 35%)
MedPhrase 2.93 (6.8%) 3.12 (8.2%) 6.7% ( -7% - 23%)
AndHighHigh 5.46 (6.6%) 5.86 (7.0%) 7.2% ( -5% - 22%)
Respell 19.68 (5.9%) 21.15 (6.6%) 7.5% ( -4% - 21%)
LowPhrase 0.46 (9.5%) 0.50 (10.2%) 7.6% ( -11% - 30%)
Prefix3 5.25 (8.2%) 5.66 (7.7%) 7.9% ( -7% - 25%)
HighSloppyPhrase 1.54 (8.0%) 1.67 (13.1%) 8.5% ( -11% - 32%)
MedSpanNear 5.25 (7.0%) 5.72 (8.2%) 9.0% ( -5% - 25%)
Wildcard 12.44 (5.7%) 13.59 (6.5%) 9.2% ( -2% - 22%)
MedSloppyPhrase 2.27 (7.2%) 2.49 (8.5%) 9.5% ( -5% - 27%)
MedTerm 28.16 (10.3%) 30.89 (9.9%) 9.7% ( -9% - 33%)
Fuzzy1 18.91 (6.0%) 20.82 (6.7%) 10.1% ( -2% - 24%)
Fuzzy2 19.69 (6.6%) 21.68 (7.5%) 10.1% ( -3% - 25%)
AndHighMed 7.79 (7.5%) 8.58 (6.1%) 10.1% ( -3% - 25%)
LowSpanNear 1.45 (5.7%) 1.60 (9.3%) 10.5% ( -4% - 27%)
LowSloppyPhrase 22.84 (7.7%) 25.45 (9.7%) 11.4% ( -5% - 31%)
LowTerm 46.46 (6.8%) 52.90 (7.6%) 13.9% ( 0% - 30%)
AndHighLow 35.92 (5.3%) 42.38 (7.1%) 18.0% ( 5% - 32%)
{code}
was (Author: gbowyer@fastmail.co.uk):
So I was going to shut this down today, and just to make sure I ran the benchmark on the simplest code possible
... and suddenly I got good results, this is idiopathic :S
https://gist.github.com/0f017853861d050c0b66
{code}
Report after iter 19:
TaskQPS baseline StdDevQPS mmap_tests StdDev Pct diff
OrHighHigh 1.68 (11.2%) 1.73 (10.3%) 3.0% ( -16% - 27%)
PKLookup 129.89 (5.8%) 135.03 (6.0%) 4.0% ( -7% - 16%)
HighTerm 8.09 (13.6%) 8.43 (12.8%) 4.2% ( -19% - 35%)
OrHighMed 4.46 (10.4%) 4.67 (9.5%) 4.7% ( -13% - 27%)
OrHighLow 4.82 (10.6%) 5.09 (10.3%) 5.6% ( -13% - 29%)
HighSpanNear 0.92 (8.1%) 0.97 (7.3%) 5.9% ( -8% - 23%)
IntNRQ 2.51 (10.2%) 2.67 (9.9%) 6.6% ( -12% - 29%)
HighPhrase 0.30 (11.7%) 0.32 (12.8%) 6.7% ( -16% - 35%)
MedPhrase 2.93 (6.8%) 3.12 (8.2%) 6.7% ( -7% - 23%)
AndHighHigh 5.46 (6.6%) 5.86 (7.0%) 7.2% ( -5% - 22%)
Respell 19.68 (5.9%) 21.15 (6.6%) 7.5% ( -4% - 21%)
LowPhrase 0.46 (9.5%) 0.50 (10.2%) 7.6% ( -11% - 30%)
Prefix3 5.25 (8.2%) 5.66 (7.7%) 7.9% ( -7% - 25%)
HighSloppyPhrase 1.54 (8.0%) 1.67 (13.1%) 8.5% ( -11% - 32%)
MedSpanNear 5.25 (7.0%) 5.72 (8.2%) 9.0% ( -5% - 25%)
Wildcard 12.44 (5.7%) 13.59 (6.5%) 9.2% ( -2% - 22%)
MedSloppyPhrase 2.27 (7.2%) 2.49 (8.5%) 9.5% ( -5% - 27%)
MedTerm 28.16 (10.3%) 30.89 (9.9%) 9.7% ( -9% - 33%)
Fuzzy1 18.91 (6.0%) 20.82 (6.7%) 10.1% ( -2% - 24%)
Fuzzy2 19.69 (6.6%) 21.68 (7.5%) 10.1% ( -3% - 25%)
AndHighMed 7.79 (7.5%) 8.58 (6.1%) 10.1% ( -3% - 25%)
LowSpanNear 1.45 (5.7%) 1.60 (9.3%) 10.5% ( -4% - 27%)
LowSloppyPhrase 22.84 (7.7%) 25.45 (9.7%) 11.4% ( -5% - 31%)
LowTerm 46.46 (6.8%) 52.90 (7.6%) 13.9% ( 0% - 30%)
AndHighLow 35.92 (5.3%) 42.38 (7.1%) 18.0% ( 5% - 32%)
{code}
> Native MMapDir
> --------------
>
> Key: LUCENE-3178
> URL: https://issues.apache.org/jira/browse/LUCENE-3178
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/store
> Reporter: Michael McCandless
> Labels: gsoc2014
> Attachments: LUCENE-3178-Native-MMap-implementation.patch, LUCENE-3178-Native-MMap-implementation.patch, LUCENE-3178-Native-MMap-implementation.patch
>
>
> Spinoff from LUCENE-2793.
> Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO flags depending on the IOContext, we could in theory do something similar with MMapDir.
> The problem is MMap is apparently quite hairy... and to pass the flags the native code would need to invoke mmap (I think?), unlike UnixDir where the code "only" has to open the file handle.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org