[jira] [Created] (LUCENE-7477) ExternalRefSorter should use OfflineSorter's actual writer for writing the input file

["Dawid Weiss (JIRA)" <ji...@apache.org>]

Dawid Weiss created LUCENE-7477:
-----------------------------------

             Summary: ExternalRefSorter should use OfflineSorter's actual writer for writing the input file
                 Key: LUCENE-7477
                 URL: https://issues.apache.org/jira/browse/LUCENE-7477
             Project: Lucene - Core
          Issue Type: Bug
            Reporter: Dawid Weiss
            Assignee: Dawid Weiss
            Priority: Minor
             Fix For: 6.x, master (7.0)


Consider this constructor in ExternalRefSorter:
{code}
  public ExternalRefSorter(OfflineSorter sorter) throws IOException {
    this.sorter = sorter;
    this.input = sorter.getDirectory().createTempOutput(sorter.getTempFileNamePrefix(), "RefSorterRaw", IOContext.DEFAULT);
    this.writer = new OfflineSorter.ByteSequencesWriter(this.input);
  }
{code}

The problem with it is that the writer for the initial input file is written with the default {{OfflineSorter.ByteSequencesWriter}}, but the instance of {{OfflineSorter}} may be unable to read it if it overrides {{getReader}} to use something else than the default.

While this works now, it should be cleaned up (I think). It'd be probably ideal to allow {{OfflineSorter}} to generate its own temporary file and just return the ByteSequencesWriter it chooses to use, so the above snippet would read:

{code}
  public ExternalRefSorter(OfflineSorter sorter) throws IOException {
    this.sorter = sorter;
    this.writer = sorter.newUnsortedPartition();
  }
{code}

This could be also extended so that {{OfflineSorter}} is in charge of managing its own (sorted and unsorted) partitions. Then {{sort(String file)}} would simply become {{ByteSequenceIterator sort()}} (or even {{Stream<BytesRef> sort()}} as Stream is conveniently {{AutoCloseable}}). If we made {{OfflineSorter}} implement {{Closeable}} it could also take care of cleaning up any resources it opens in the directory we pass to it.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org