Can a data import handler grab all pages of an RSS feed?

[eShard <zi...@yahoo.com>]

Good morning,
I have an IBM Portal atom feed that spans multiple pages.
Is there a way to instruct the DIH to grab all available pages?
I can put a huge range in but that can be extremely slow with large amounts
of XML data.
I'm currently using Solr 4.0 final.

Thanks,



--
View this message in context: http://lucene.472066.n3.nabble.com/Can-a-data-import-handler-grab-all-pages-of-an-RSS-feed-tp4086635.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Can a data import handler grab all pages of an RSS feed?

[Alexandre Rafalovitch <ar...@gmail.com>]

Have you tried using $hasMore and $nextUrl? You can inject it with a custom
transformer. It is not documented very well, but is mentioned on the Wiki.

Regards,
   Alex.

Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)


On Mon, Aug 26, 2013 at 9:49 PM, eShard <zi...@yahoo.com> wrote:

> Good morning,
> I have an IBM Portal atom feed that spans multiple pages.
> Is there a way to instruct the DIH to grab all available pages?
> I can put a huge range in but that can be extremely slow with large amounts
> of XML data.
> I'm currently using Solr 4.0 final.
>
> Thanks,
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Can-a-data-import-handler-grab-all-pages-of-an-RSS-feed-tp4086635.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>