RE: Migration Lucene 4.7.0 -->6.0.1 - NumericUtils

[Ludovic Bertin <l....@lombardodier.com>]

Hi Mike,

Sorry for the delay.
For some use cases, Fields are implemented by numeric values, but distinct values are not so big (<50 terms).
We have a Search GUI, where user can add criteria on predefined fields. For those fields, where the number of terms is limited, then we proposed the available values in a dropdown.

What are your recommendations ? Using String instead of Numeric Values ?

Thanks.
Ludovic



-----Original Message-----
From: Michael McCandless [mailto:lucene@mikemccandless.com]
Sent: jeudi, 22. septembre 2016 22:34
To: Lucene Users; Ludovic Bertin
Subject: Re: Migration Lucene 4.7.0 -->6.0.1 - NumericUtils

LegacyNumericUtils is the right solution for your index for now, but
longer term you should migrate to dimensional points instead, which
are a more efficient way to index and range search numerics.

But: why do you need all distinct values of a field?  In general this
is a very dangerous method to offer.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Sep 20, 2016 at 9:02 AM, Ludovic Bertin
<l....@lombardodier.com> wrote:
> Hi there,
>
> I'm migrating an application from Lucene 4.7.0 to Lucene 6.0.1.
> I'm facing a problem with this piece of code :
>
> public List<Object> getDistinctValues(IndexReader reader, EventField field) throws IOException {
>
>     List<Object> values = new ArrayList<Object>();
>     Fields fields = MultiFields.getFields(reader);
>     if (fields == null) return values;
>
>     Terms terms = fields.terms(field.name());
>     if (terms == null) return values;
>
>     TermsEnum iterator = terms.iterator();
>
>     Class<? extends Object> type = field.getJavaType();
>     BytesRef value = iterator.next();
>
>     while (value != null) {
>         if (type == Long.class) {
>             values.add(LegacyNumericUtils.prefixCodedToLong(value));
>         } else if (type == Integer.class) {
>             values.add(LegacyNumericUtils.prefixCodedToInt(value));
>         } else if (type == Boolean.class) {
>             values.add(LegacyNumericUtils.prefixCodedToInt(value) == 1 ? TRUE : FALSE);
>         } else if (type == Date.class) {
>             values.add(new Date(LegacyNumericUtils.prefixCodedToLong(value)));
>         } else if (type == String.class) {
>             values.add(value.utf8ToString());
>         } else {
>             log.warn("getDistinctValues: ignoring field " + field + " of type " + type);
>         }
>
>         value = iterator.next();
>     }
>
>     return values;
> }
>
> The aim of this method is to get all terms present in a field directly in the correct java type. I'm seeing just now that NumericUtils is an internal class, so I should not have used it.
> For now, I can use LegacyNumericUtils, but that's a very temporary solution.
>
> What is the best approach to achieve my goal ?
>
> Thanks in advance for any help.
> Ludovic
>
> ************************ DISCLAIMER ************************
> This message is intended only for use by the person to
> whom it is addressed. It may contain information that is
> privileged and confidential. Its content does not constitute
> a formal commitment by Bank Lombard Odier & Co Ltd or any
> of its branches or affiliates. If you are not the intended recipient
> of this message, kindly notify the sender immediately and
> destroy this message. Thank You.
> *****************************************************************
[[ rethink everything. ]]<http://www.lombardodier.com>

DISCLAIMER **********************************************
This message is intended only for use by the person to
whom it is addressed. It may contain information that is
privileged and confidential. Its content does not constitute
a formal commitment by Bank Lombard Odier & Co Ltd
or any of its branches or affiliates. If you are not the
intended recipient of this message, kindly notify the sender
immediately and destroy this message. Thank You.
***************************************************************