You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Robert Muir (Jira)" <ji...@apache.org> on 2020/02/01 13:44:00 UTC
[jira] [Comment Edited] (LUCENE-9191) Fix linefiledocs compression
or replace in tests
[ https://issues.apache.org/jira/browse/LUCENE-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028089#comment-17028089 ]
Robert Muir edited comment on LUCENE-9191 at 2/1/20 1:43 PM:
-------------------------------------------------------------
{noformat}
./gradlew -p lucene/core test --tests TestFSTs -Dtests.linedocsfile=/bogus
...
org.apache.lucene.util.fst.TestFSTs > testRealTerms FAILED
java.nio.file.NoSuchFileException: /bogus
at __randomizedtesting.SeedInfo.seed([C215F686A314AB6B:DC33C40B44C45E59]:0)
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1842)
at java.base/java.nio.file.Files.size(Files.java:2459)
at org.apache.lucene.util.LineFileDocs.open(LineFileDocs.java:89)
at org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:63)
at org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:67)
{noformat}
was (Author: rcmuir):
./gradlew -p lucene/core test --tests TestFSTs -Dtests.linedocsfile=/bogus
...
org.apache.lucene.util.fst.TestFSTs > testRealTerms FAILED
java.nio.file.NoSuchFileException: /bogus
at __randomizedtesting.SeedInfo.seed([C215F686A314AB6B:DC33C40B44C45E59]:0)
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1842)
at java.base/java.nio.file.Files.size(Files.java:2459)
at org.apache.lucene.util.LineFileDocs.open(LineFileDocs.java:89)
at org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:63)
at org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:67)
{noformat}
> Fix linefiledocs compression or replace in tests
> ------------------------------------------------
>
> Key: LUCENE-9191
> URL: https://issues.apache.org/jira/browse/LUCENE-9191
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Robert Muir
> Assignee: Michael McCandless
> Priority: Major
>
> LineFileDocs(random) is very slow, even to open. It does a very slow "random skip" through a gzip compressed file.
> For the analyzers tests, in LUCENE-9186 I simply removed its usage, since TestUtil.randomAnalysisString is superior, and fast. But we should address other tests using it, since LineFileDocs(random) is slow!
> I think it is also the case that every lucene test has probably tested every LineFileDocs line many times now, whereas randomAnalysisString will invent new ones.
> Alternatively, we could "fix" LineFileDocs(random), e.g. special compression options (in blocks)... deflate supports such stuff. But it would make it even hairier than it is now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org