Custom DIH: FileDataSource with additional business logic?

[Giovanni De Stefano <gi...@gmail.com>]

Hello,

here I am with another question.

I am using DIH to index a DB. Additionally I also have to index some files
containing Java serialized objects (and I cannot change this... :-( ).

I currently have implemented a standalone Java app with the following
features:

1) read all files from a given folder
2) deserialize the files into lists of items
3) convert the list of items into lists of SolrInputDocument(s)
4) post the lists of SolrInputDocument(s) to Solr

All this is done using SolrJ. So far so good.

I would like to use a DIH with a FileDataSource to do 1) and 4), and I would
like to "squeeze" in my implementation for 2) and 3).

Is this possible? Any hint?

Thank you all in advance.

Cheers,
Giovanni

Re: Custom DIH: FileDataSource with additional business logic?

[Noble Paul നോബിള്‍ नोब्ळ् <no...@gmail.com>]

FileDataSource is of type Reader . means getData() returns
ajava.io.Reader.That is not very suitable for you.

your best bet is to write a simple DataSource  which returns an
Iterator<Map<String,Object>> after reading the serialized Objects
.This is what JdbcdataSource does. Then you can use it with
SqlEntityProcessor

On Thu, Apr 9, 2009 at 9:42 PM, Giovanni De Stefano
<gi...@gmail.com> wrote:
> Hello,
>
> here I am with another question.
>
> I am using DIH to index a DB. Additionally I also have to index some files
> containing Java serialized objects (and I cannot change this... :-( ).
>
> I currently have implemented a standalone Java app with the following
> features:
>
> 1) read all files from a given folder
> 2) deserialize the files into lists of items
> 3) convert the list of items into lists of SolrInputDocument(s)
> 4) post the lists of SolrInputDocument(s) to Solr
>
> All this is done using SolrJ. So far so good.
>
> I would like to use a DIH with a FileDataSource to do 1) and 4), and I would
> like to "squeeze" in my implementation for 2) and 3).
>
> Is this possible? Any hint?
>
> Thank you all in advance.
>
> Cheers,
> Giovanni
>



-- 
--Noble Paul