You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Peter PiĊĦljar <pe...@gmail.com> on 2014/05/15 13:15:27 UTC

DataImportHandler atomic updates

Hello,

i am trying to import data from my db into solr.
in db i have two tables
- orders [order_id, user_id, created_at, store]
- order_items [order_id, item_id] (one - to - many relation )

i would like to import this into my solr collection
- orders [user_id (unique), item_id (multivalue) ]

i select all the orders from my table and process them
i select all the items for each order and process them,.

now the problem is, that without atomic updates, each order for the user id
'10' will overwrite its previous orders. (instead just add the item_ids)

so i tried to use scripttransformer to set the "add" parameter for atomic
update of item_id field.

its still not working correctly.
1) i dont get all the items, but there are more than just from the last
order ... lets say last 5 orders
2) first few items are saved as add:ID, the others are ok.
(lets say that i have 20 itrems (1...20) for my user_id '10', something
like this would get in the index: [ add:14, add:15, 16, 17,18,19,20 ] (note
that items from 1...13 are missing, and item 14 and 15 have "add:" infront
of them.

here is my full script: http://pastebin.com/6EnW8Heg

please note that above i simplified the description a little bit, but still
everything applies.

i would really appreciate any kind of help.
-------------------