You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Fiz N <fi...@gmail.com> on 2020/06/07 17:19:29 UTC
Indexing PDF on SOLR 8.5
Hello SOLR Experts,
I am working on a POC to Index millions of PDF documents present in
Multiple Folder in fileshare.
Could you please let me the best practices and step to implement it.
Thanks
Fiz Nadiyal.
Re: Indexing PDF on SOLR 8.5
Posted by Fiz N <fi...@gmail.com>.
Thanks Erick...
On Sun, Jun 7, 2020 at 1:50 PM Erick Erickson <er...@gmail.com>
wrote:
> https://lucidworks.com/post/indexing-with-solrj/
>
>
> > On Jun 7, 2020, at 3:22 PM, Fiz N <fi...@gmail.com> wrote:
> >
> > Thanks Jorn and Erick.
> >
> > Hi Erick, looks like the skeletal SOLRJ program attachment is missing.
> >
> > Thanks
> > Fiz
> >
> > On Sun, Jun 7, 2020 at 12:20 PM Erick Erickson <er...@gmail.com>
> > wrote:
> >
> >> Here’s a skeletal SolrJ program using Tika as another alternative.
> >>
> >> Best,
> >> Erick
> >>
> >>> On Jun 7, 2020, at 2:06 PM, Jörn Franke <jo...@gmail.com> wrote:
> >>>
> >>> You have to write an external application that creates multiple
> threads,
> >> parses the PDFs and index them in Solr. Ideally you parse the PDFs once
> and
> >> store the resulting text on some file system and then index it. Reason
> is
> >> that if you upgrade to two major versions of Solr you might need to
> reindex
> >> again. Then you can save time because you don’t need to parse the PDFs
> >> again.
> >>> It can be also useful in case you are not sure yet about the final
> >> schema and need to index several times in different schemas etc
> >>>
> >>> You can also use Apache manifoldCF.
> >>>
> >>>
> >>>
> >>>> Am 07.06.2020 um 19:19 schrieb Fiz N <fi...@gmail.com>:
> >>>>
> >>>> Hello SOLR Experts,
> >>>>
> >>>> I am working on a POC to Index millions of PDF documents present in
> >>>> Multiple Folder in fileshare.
> >>>>
> >>>> Could you please let me the best practices and step to implement it.
> >>>>
> >>>> Thanks
> >>>> Fiz Nadiyal.
> >>
> >>
>
>
Re: Indexing PDF on SOLR 8.5
Posted by Erick Erickson <er...@gmail.com>.
https://lucidworks.com/post/indexing-with-solrj/
> On Jun 7, 2020, at 3:22 PM, Fiz N <fi...@gmail.com> wrote:
>
> Thanks Jorn and Erick.
>
> Hi Erick, looks like the skeletal SOLRJ program attachment is missing.
>
> Thanks
> Fiz
>
> On Sun, Jun 7, 2020 at 12:20 PM Erick Erickson <er...@gmail.com>
> wrote:
>
>> Here’s a skeletal SolrJ program using Tika as another alternative.
>>
>> Best,
>> Erick
>>
>>> On Jun 7, 2020, at 2:06 PM, Jörn Franke <jo...@gmail.com> wrote:
>>>
>>> You have to write an external application that creates multiple threads,
>> parses the PDFs and index them in Solr. Ideally you parse the PDFs once and
>> store the resulting text on some file system and then index it. Reason is
>> that if you upgrade to two major versions of Solr you might need to reindex
>> again. Then you can save time because you don’t need to parse the PDFs
>> again.
>>> It can be also useful in case you are not sure yet about the final
>> schema and need to index several times in different schemas etc
>>>
>>> You can also use Apache manifoldCF.
>>>
>>>
>>>
>>>> Am 07.06.2020 um 19:19 schrieb Fiz N <fi...@gmail.com>:
>>>>
>>>> Hello SOLR Experts,
>>>>
>>>> I am working on a POC to Index millions of PDF documents present in
>>>> Multiple Folder in fileshare.
>>>>
>>>> Could you please let me the best practices and step to implement it.
>>>>
>>>> Thanks
>>>> Fiz Nadiyal.
>>
>>
Re: Indexing PDF on SOLR 8.5
Posted by Fiz N <fi...@gmail.com>.
Thanks Jorn and Erick.
Hi Erick, looks like the skeletal SOLRJ program attachment is missing.
Thanks
Fiz
On Sun, Jun 7, 2020 at 12:20 PM Erick Erickson <er...@gmail.com>
wrote:
> Here’s a skeletal SolrJ program using Tika as another alternative.
>
> Best,
> Erick
>
> > On Jun 7, 2020, at 2:06 PM, Jörn Franke <jo...@gmail.com> wrote:
> >
> > You have to write an external application that creates multiple threads,
> parses the PDFs and index them in Solr. Ideally you parse the PDFs once and
> store the resulting text on some file system and then index it. Reason is
> that if you upgrade to two major versions of Solr you might need to reindex
> again. Then you can save time because you don’t need to parse the PDFs
> again.
> > It can be also useful in case you are not sure yet about the final
> schema and need to index several times in different schemas etc
> >
> > You can also use Apache manifoldCF.
> >
> >
> >
> >> Am 07.06.2020 um 19:19 schrieb Fiz N <fi...@gmail.com>:
> >>
> >> Hello SOLR Experts,
> >>
> >> I am working on a POC to Index millions of PDF documents present in
> >> Multiple Folder in fileshare.
> >>
> >> Could you please let me the best practices and step to implement it.
> >>
> >> Thanks
> >> Fiz Nadiyal.
>
>
Re: Indexing PDF on SOLR 8.5
Posted by Erick Erickson <er...@gmail.com>.
Here’s a skeletal SolrJ program using Tika as another alternative.
Best,
Erick
> On Jun 7, 2020, at 2:06 PM, Jörn Franke <jo...@gmail.com> wrote:
>
> You have to write an external application that creates multiple threads, parses the PDFs and index them in Solr. Ideally you parse the PDFs once and store the resulting text on some file system and then index it. Reason is that if you upgrade to two major versions of Solr you might need to reindex again. Then you can save time because you don’t need to parse the PDFs again.
> It can be also useful in case you are not sure yet about the final schema and need to index several times in different schemas etc
>
> You can also use Apache manifoldCF.
>
>
>
>> Am 07.06.2020 um 19:19 schrieb Fiz N <fi...@gmail.com>:
>>
>> Hello SOLR Experts,
>>
>> I am working on a POC to Index millions of PDF documents present in
>> Multiple Folder in fileshare.
>>
>> Could you please let me the best practices and step to implement it.
>>
>> Thanks
>> Fiz Nadiyal.
Re: Indexing PDF on SOLR 8.5
Posted by Jörn Franke <jo...@gmail.com>.
You have to write an external application that creates multiple threads, parses the PDFs and index them in Solr. Ideally you parse the PDFs once and store the resulting text on some file system and then index it. Reason is that if you upgrade to two major versions of Solr you might need to reindex again. Then you can save time because you don’t need to parse the PDFs again.
It can be also useful in case you are not sure yet about the final schema and need to index several times in different schemas etc
You can also use Apache manifoldCF.
> Am 07.06.2020 um 19:19 schrieb Fiz N <fi...@gmail.com>:
>
> Hello SOLR Experts,
>
> I am working on a POC to Index millions of PDF documents present in
> Multiple Folder in fileshare.
>
> Could you please let me the best practices and step to implement it.
>
> Thanks
> Fiz Nadiyal.