You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Steve Rowe (JIRA)" <ji...@apache.org> on 2017/05/27 13:18:04 UTC
[jira] [Reopened] (LUCENE-7845) spatial RPT optimization when query
by point or common date range
[ https://issues.apache.org/jira/browse/LUCENE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Rowe reopened LUCENE-7845:
--------------------------------
My Jenkins found a reproducing failure that {{git bisect}} blames on commit {{d4f87b4a36}}, on this issue:
{noformat}
Checking out Revision bc973ecdcfacf39440da06b86139c77935e1e92e (refs/remotes/origin/master)
[...]
[junit4] Suite: org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest
[junit4] 2> máj. 27, 2017 5:12:44 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:3,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:44 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:5,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)))
[junit4] 2> máj. 27, 2017 5:12:44 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:44 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:44 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:1,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pointsOnly)
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:1,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:3,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:45 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pointsOnly,pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:1,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:1,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)),pointsOnly,pruneLeafyBranches)
[junit4] 2> Ig:Rect(minX=128.0,maxX=256.0,minY=0.0,maxY=128.0) Qg:Rect(minX=128.0,maxX=256.0,minY=0.0,maxY=128.0)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RandomSpatialOpFuzzyPrefixTreeTest -Dtests.method=testIntersects -Dtests.seed=F4FA997B1727B3C9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu-HU -Dtests.timezone=Europe/Tirane -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.05s J5 | RandomSpatialOpFuzzyPrefixTreeTest.testIntersects { seed=[F4FA997B1727B3C9:8990A22D10916476]} <<<
[junit4] > Throwable #1: java.lang.AssertionError: Should have matched I#0:Pt(x=209.0,y=63.0) Q:Pt(x=209.0,y=63.0)
[junit4] > at __randomizedtesting.SeedInfo.seed([F4FA997B1727B3C9:8990A22D10916476]:0)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.fail(RandomSpatialOpFuzzyPrefixTreeTest.java:399)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.doTest(RandomSpatialOpFuzzyPrefixTreeTest.java:386)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testIntersects(RandomSpatialOpFuzzyPrefixTreeTest.java:138)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:5,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)),pointsOnly,pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:3,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:5,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)),pruneLeafyBranches)
[junit4] 2> máj. 27, 2017 5:12:46 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:2,ctx:SpatialContext.GEO)))
[junit4] 2> máj. 27, 2017 5:12:47 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:5,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> máj. 27, 2017 5:12:47 DE org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/jobs/Lucene-Solr-Nightly-master/workspace/lucene/build/spatial-extras/test/J5/temp/lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest_F4FA997B1727B3C9-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {RandomSpatialOpFuzzyPrefixTreeTest=DFR I(ne)2}, locale=hu-HU, timezone=Europe/Tirane
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 (64-bit)/cpus=16,threads=1,free=387737760,total=514850816
[junit4] 2> NOTE: All tests run in this JVM: [RandomSpatialOpFuzzyPrefixTreeTest]
[junit4] Completed [6/23 (1!)] on J5 in 3.30s, 34 tests, 1 failure <<< FAILURES!
{noformat}
> spatial RPT optimization when query by point or common date range
> -----------------------------------------------------------------
>
> Key: LUCENE-7845
> URL: https://issues.apache.org/jira/browse/LUCENE-7845
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/spatial-extras
> Reporter: David Smiley
> Assignee: David Smiley
> Fix For: master (7.0)
>
> Attachments: LUCENE_7845_query_by_point_optimization.patch
>
>
> If the query to an RPT index is a 2D point, or if using NumerBrangePrefixTreeStrategy / DateRangePrefixTree (Solr DateRangeField) if the query is a grid cell (a common date range unit like some particular day), then we can do some optimizations, especially if the data is pointsOnly. If the data is pointsOnly the strategy can return a TermQuery, if the data isn't then we can at least tweak the prefixGridScanLevel. This is motivated by two scenarios:
> * indexing polygons and doing lookups by a point (AKA reverse geocoding)
> * indexing date instances and doing date range faceting. Solr's code for this has a fast path for a TermQuery, although more is needed beyond this issue to get there.
> _This development was funded by the Harvard Center for Geographic Analysis as part of the HHypermap project_
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org