Is there any particular reason why ExternalFileField is read from data directory

[apoorvqwerty <ap...@unbxd.com>]

Hi,
As per the documentation for ExternalFileField we need to put external_field
with the map in parallel with the data directory on all the shards.
Is it possible to read the file from a central location or zookeeper?



--
View this message in context: http://lucene.472066.n3.nabble.com/Is-there-any-particular-reason-why-ExternalFileField-is-read-from-data-directory-tp4343374.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Is there any particular reason why ExternalFileField is read from data directory

[apoorvqwerty <ap...@unbxd.com>]

Thanks a lot,
for now I've written a listener to read from redis instead.
But might not scale well since the map is kept in memory.



--
View this message in context: http://lucene.472066.n3.nabble.com/Is-there-any-particular-reason-why-ExternalFileField-is-read-from-data-directory-tp4343374p4344151.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Is there any particular reason why ExternalFileField is read from data directory

[Erick Erickson <er...@gmail.com>]

Should be OK if you put it in your conf directory of your configset.
Do note that each and every time you start Solr, each and every
replica will download it. So if it's a large file you'll have
problems. The default limit for jute.maxbuffer is 1M so you'd also
have to bump that up.

Frankly, though, ExternalFileField is a blunt instrument, if you're
putting single numeric values in there consider updateable docValues
instead.

Best,
Erick

On Thu, Jun 29, 2017 at 1:17 AM, apoorvqwerty <ap...@unbxd.com> wrote:
> Hi,
> As per the documentation for ExternalFileField we need to put external_field
> with the map in parallel with the data directory on all the shards.
> Is it possible to read the file from a central location or zookeeper?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-there-any-particular-reason-why-ExternalFileField-is-read-from-data-directory-tp4343374.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Is there any particular reason why ExternalFileField is read from data directory

[Koji Sekiguchi <ko...@rondhuit.com>]

Hi,

ExternalFileField was introduced via SOLR-351.

https://issues.apache.org/jira/browse/SOLR-351

The author thought values could optionally be updated often...
I think it describes why it is read from not config, but datadir.

Koji


On 2017/06/29 17:17, apoorvqwerty wrote:
> Hi,
> As per the documentation for ExternalFileField we need to put external_field
> with the map in parallel with the data directory on all the shards.
> Is it possible to read the file from a central location or zookeeper?
> 
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-there-any-particular-reason-why-ExternalFileField-is-read-from-data-directory-tp4343374.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>