relaxed vs. improved validation in solr.TrieDateField

[Uwe Reh <re...@hebis.uni-frankfurt.de>]

Hi,

doing some migration tests (4.10 to 6.0) I recognized a improved 
validation of TrieDateField.
Syntactical correct but impossible days are rejected now. (stack trace 
at the end of the mail)

Examples:
- '1997-02-29T00:00:00Z'
- '2006-06-31T00:00:00Z'
- '2000-00-00T00:00:00Z'
The first two dates are formal ok, but the Date does not exist. The 
third date is more suspicions, but was also accepted by Solr 4.10.

I appreciate this improvement in principle, but I have to respect the 
original data. The dates might be intentionally wrong.

Is there an easy way to get the weaker validation back?

Regards
Uwe


> Invalid Date in Date Math String:'1997-02-29T00:00:00Z'
>         at org.apache.solr.util.DateMathParser.parseMath(DateMathParser.java:254)
>         at org.apache.solr.schema.TrieField.createField(TrieField.java:726)
>         at org.apache.solr.schema.TrieField.createFields(TrieField.java:763)
>         at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)

Re: relaxed vs. improved validation in solr.TrieDateField

[David Smiley <da...@gmail.com>]

Sorry to hear that Uwe Reh.

If this is just in your input/index data, then this could be handled with
an URP, maybe evan an existing URP.
See ParseDateFieldUpdateProcessorFactory which uses the Joda-time API.  I
am not sure if that will work, I'm a little doubtful in fact since Solr now
uses the Java 8 time API which was taken, more or less, from Joda-time.
But it's worth a shot, any way.  If it doesn't work, let me know and I'll
give you a snippet of JavaScript you can use in your URP chain.

~ David

On Fri, Apr 29, 2016 at 4:07 AM Uwe Reh <re...@hebis.uni-frankfurt.de> wrote:

> Hi,
>
> doing some migration tests (4.10 to 6.0) I recognized a improved
> validation of TrieDateField.
> Syntactical correct but impossible days are rejected now. (stack trace
> at the end of the mail)
>
> Examples:
> - '1997-02-29T00:00:00Z'
> - '2006-06-31T00:00:00Z'
> - '2000-00-00T00:00:00Z'
> The first two dates are formal ok, but the Date does not exist. The
> third date is more suspicions, but was also accepted by Solr 4.10.
>
> I appreciate this improvement in principle, but I have to respect the
> original data. The dates might be intentionally wrong.
>
> Is there an easy way to get the weaker validation back?
>
> Regards
> Uwe
>
>
> > Invalid Date in Date Math String:'1997-02-29T00:00:00Z'
> >         at
> org.apache.solr.util.DateMathParser.parseMath(DateMathParser.java:254)
> >         at
> org.apache.solr.schema.TrieField.createField(TrieField.java:726)
> >         at
> org.apache.solr.schema.TrieField.createFields(TrieField.java:763)
> >         at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com