DIH and UUIDProcessorFactory

[Dmitri Maziuk <dm...@gmail.com>]

Hi everyone,

is there an easy way to use the stock UUID generator with DIH? We have a 
hand-written one-liner class we use as DIH entity transformer but I 
wonder if there's a way to use the built-in UUID generator class instead.

 From the TFM it looks like there isn't, is that correct?

TIA,
Dmitri

Re: DIH and UUIDProcessorFactory

[Alexandre Rafalovitch <ar...@gmail.com>]

Why not? You should be able to put an URP chain after DIH, the usual way.

Is that something about UUID that is special?

Regards,
    Alex

On Sat., Dec. 12, 2020, 2:55 p.m. Dmitri Maziuk, <dm...@gmail.com>
wrote:

> Hi everyone,
>
> is there an easy way to use the stock UUID generator with DIH? We have a
> hand-written one-liner class we use as DIH entity transformer but I
> wonder if there's a way to use the built-in UUID generator class instead.
>
>  From the TFM it looks like there isn't, is that correct?
>
> TIA,
> Dmitri
>

Re: DIH and UUIDProcessorFactory

[Dmitri Maziuk <dm...@gmail.com>]

On 12/17/2020 4:05 PM, Alexandre Rafalovitch wrote:
> Try with the explicit URP chain too. It may work as well.

Actually in this case we're just making sure uniqueKey is in fact unique 
in all documents, so default is what we want.

For this particular dataset I may at some future point look into 
generating ID as a hash of some unique tuple or other, but then I expect 
we'll still want to keep the UUID fallback.

Dima

Re: DIH and UUIDProcessorFactory

[Alexandre Rafalovitch <ar...@gmail.com>]

Try with the explicit URP chain too. It may work as well.

Regards,
   Alex.

On Thu, 17 Dec 2020 at 16:51, Dmitri Maziuk <dm...@gmail.com> wrote:
>
> On 12/12/2020 4:36 PM, Shawn Heisey wrote:
> > On 12/12/2020 2:30 PM, Dmitri Maziuk wrote:
> >> Right, ```Every update request received by Solr is run through a chain
> >> of plugins known as Update Request Processors, or URPs.```
> >>
> >> The part I'm missing is whether DIH's '<requestHandler
> >> name="/dataimport"' counts as an "Update Request", my reading is it
> >> doesn't and URP chain applies only to '<updateHandler ...'
> >
> > If you define an update chain as default, then it will be used for all
> > updates made where a different chain is not specifically requested.
> >
> > I have used this personally to have my custom update chain apply even
> > when the indexing comes from DIH.  I know for sure that this works on
> > 4.x and 5.x versions; it should work on newer versions as well.
> >
>
> Confirmed w/ 8.7.0: I finally got to importing the one DB where I need
> this, and UUIDs are there with the default URP chain.
>
> Thank you
> Dima
>
>

Re: DIH and UUIDProcessorFactory

[Dmitri Maziuk <dm...@gmail.com>]

On 12/12/2020 4:36 PM, Shawn Heisey wrote:
> On 12/12/2020 2:30 PM, Dmitri Maziuk wrote:
>> Right, ```Every update request received by Solr is run through a chain 
>> of plugins known as Update Request Processors, or URPs.```
>>
>> The part I'm missing is whether DIH's '<requestHandler 
>> name="/dataimport"' counts as an "Update Request", my reading is it 
>> doesn't and URP chain applies only to '<updateHandler ...'
> 
> If you define an update chain as default, then it will be used for all 
> updates made where a different chain is not specifically requested.
> 
> I have used this personally to have my custom update chain apply even 
> when the indexing comes from DIH.  I know for sure that this works on 
> 4.x and 5.x versions; it should work on newer versions as well.
> 

Confirmed w/ 8.7.0: I finally got to importing the one DB where I need 
this, and UUIDs are there with the default URP chain.

Thank you
Dima

Re: DIH and UUIDProcessorFactory

[Shawn Heisey <ap...@elyograg.org>]

On 12/12/2020 2:30 PM, Dmitri Maziuk wrote:
> Right, ```Every update request received by Solr is run through a chain 
> of plugins known as Update Request Processors, or URPs.```
> 
> The part I'm missing is whether DIH's '<requestHandler 
> name="/dataimport"' counts as an "Update Request", my reading is it 
> doesn't and URP chain applies only to '<updateHandler ...'

If you define an update chain as default, then it will be used for all 
updates made where a different chain is not specifically requested.

I have used this personally to have my custom update chain apply even 
when the indexing comes from DIH.  I know for sure that this works on 
4.x and 5.x versions; it should work on newer versions as well.

Thanks,
Shawn

Re: DIH and UUIDProcessorFactory

[Dmitri Maziuk <dm...@gmail.com>]

On 12/12/2020 2:50 PM, Shawn Heisey wrote:

> The only way I know of to use an update processor chain with DIH is to 
> set 'default="true"' when defining the chain.
> 
> I did manage to find an example with the default attribute, in javadocs:
> 
> https://lucene.apache.org/solr/5_0_0/solr-core/org/apache/solr/update/processor/UpdateRequestProcessorChain.html 

Right, ```Every update request received by Solr is run through a chain 
of plugins known as Update Request Processors, or URPs.```

The part I'm missing is whether DIH's '<requestHandler 
name="/dataimport"' counts as an "Update Request", my reading is it 
doesn't and URP chain applies only to '<updateHandler ...'

Dmitri

Re: DIH and UUIDProcessorFactory

[Shawn Heisey <ap...@elyograg.org>]

On 12/12/2020 12:54 PM, Dmitri Maziuk wrote:
> is there an easy way to use the stock UUID generator with DIH? We have a 
> hand-written one-liner class we use as DIH entity transformer but I 
> wonder if there's a way to use the built-in UUID generator class instead.
> 
>  From the TFM it looks like there isn't, is that correct?

The only way I know of to use an update processor chain with DIH is to 
set 'default="true"' when defining the chain.

I did manage to find an example with the default attribute, in javadocs:

https://lucene.apache.org/solr/5_0_0/solr-core/org/apache/solr/update/processor/UpdateRequestProcessorChain.html

If there is another way to specify the chain to use with DIH, I do not 
know about it.  I am always learning new things, something might exist 
that I have never seen.

Thanks,
Shawn