[jira] [Resolved] (LUCENE-9652) DataInput.readFloats to be used by Lucene90VectorReader

["Michael Sokolov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-9652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Sokolov resolved LUCENE-9652.
-------------------------------------
    Resolution: Fixed

> DataInput.readFloats to be used by Lucene90VectorReader
> -------------------------------------------------------
>
>                 Key: LUCENE-9652
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9652
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael Sokolov
>            Priority: Major
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Benchmarking shows a substantial performance gain can be realized by avoiding the additional memory copy we must do today when converting from {{byte[]}} read using {{IndexInput}} into {{float[]}} returned by {{Lucene90VectorReader}}. We have a model for how to handle the various alignments, and buffer underflow when a value spans buffers, in {{readLELongs}}.
> I think we should only support little-endian floats from the beginning here. We're planning to move towards switching the whole IndexInput to that endianness, right?
> Lucene90VectorWriter relies on {{VectorValues.binaryValue()}} to return bytes in the format expected by the reader, and its javadocs don't currently specify their endianness. In fact the order has been the default supplied by {{ByteBuffer.allocate(int)}}, which I now realize is big-endian, so this issue also proposes to change the index format. That would mean a backwards-incompatible index change, but I think if we're still unreleased and in an experimental class that should be OK?
> Also, we don't need a corresponding {{DataOutput.writeFloats}} to support the current usage for vectors, since there we rely on {{VectorValues}} to do the conversion, so I don't plan to implement that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org