Solr index writing to s3

[Naveen M <na...@gmail.com>]

hi,

My requirement is to write the index data into S3, we have solr installed
on aws instances. Please let me know if there is any documentation on how
to achieve writing the index data to s3.

Thanks

Re: Solr index writing to s3

[Hendrik Haddorp <he...@gmx.net>]

Theoretically you should be able to use the HDFS backend, which you can 
configure to use s3. Last time I tried that it did however not work for 
some reason. Here is an example for that, which also seems to have 
ultimately failed: 
https://community.plm.automation.siemens.com/t5/Developer-Space/Running-Solr-on-S3/td-p/449360

On 16.01.2019 19:39, Naveen M wrote:
> hi,
>
> My requirement is to write the index data into S3, we have solr installed
> on aws instances. Please let me know if there is any documentation on how
> to achieve writing the index data to s3.
>
> Thanks
>

Re: Solr index writing to s3

[Mikhail Khludnev <mk...@apache.org>]

There is some experience on backup to s3
https://issues.apache.org/jira/browse/SOLR-9952 iirc, it lacks
performance.
Jörn, it's not a point, but literally s3 consistency might be enough, since
s3 provides read-after-write for PUT and Lucene index writer is
append-only.

On Thu, Jan 17, 2019 at 10:15 AM Jörn Franke <jo...@gmail.com> wrote:

> This is not a requirement. This is a statement to a problem where there
> could be other solutions. s3 is only eventually consistent and I am not
> sure Solr works properly in this case. You may also need to check the S3
> consistency to be applied.
>
> > Am 16.01.2019 um 19:39 schrieb Naveen M <na...@gmail.com>:
> >
> > hi,
> >
> > My requirement is to write the index data into S3, we have solr installed
> > on aws instances. Please let me know if there is any documentation on how
> > to achieve writing the index data to s3.
> >
> > Thanks
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Solr index writing to s3

[Jörn Franke <jo...@gmail.com>]

This is not a requirement. This is a statement to a problem where there could be other solutions. s3 is only eventually consistent and I am not sure Solr works properly in this case. You may also need to check the S3 consistency to be applied.

> Am 16.01.2019 um 19:39 schrieb Naveen M <na...@gmail.com>:
> 
> hi,
> 
> My requirement is to write the index data into S3, we have solr installed
> on aws instances. Please let me know if there is any documentation on how
> to achieve writing the index data to s3.
> 
> Thanks