external values source

[Maciej Liżewski <ma...@3e.pl>]

I need some explanation on how ValuesSource and related classes work.

There are already implemented ExternalFileField, example on how to load data
from database (
<http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.
html>
http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.h
tml)

But they all fetch ALL data into memory which may consume large amounts of
this resource. Also documents are referenced by 'doc' integer value.

 

My questions:

1)      Is the 'doc' value pointing to document in whole index? If so - how
to get value of such documents field (for example: field named 'id')?

2)      Is there possibility to create ValuesSource, FieldType (or similar
interface which will provide external data to sort and in query results)
which will work only on some subset of documents and use external source
capabilities to fetch document related data?

3)      How does it all work (memory consumption, hashtable access speed,
etc), when there is a lot of documents in index (tens of millions for
example)?

4)      Are there any other examples on loading external data from database
(I want to have numerical 'rate' from simple table having two columns:
'document unique key' string, 'rate' integer/float) which are not just proof
of concept but real-life examples?

 

Any help and hints appreciated

TIA

 

--

Maciek