You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@manifoldcf.apache.org by Arcadius Ahouansou <ar...@menelic.com> on 2014/10/30 15:47:04 UTC

Crawling a JSON REST API

Hello.

Can ManifoldCF be used to call/crawl an arbitrary JSON API (output is JSON
with multiple entities... bit like an RSS)  and map each entry to Solr
documents?

Thanks.

Arcadius.

Re: Crawling a JSON REST API

Posted by Karl Wright <da...@gmail.com>.

Hi Arcadius,

Yes, if you write an appropriate connector, MCF can crawl JSON and write
each entity as a component of the document.  You would need a distinct URL
each component, and a component identifier, but that is pretty much all.

Karl

On Thu, Oct 30, 2014 at 10:47 AM, Arcadius Ahouansou <ar...@menelic.com>
wrote:

>
> Hello.
>
> Can ManifoldCF be used to call/crawl an arbitrary JSON API (output is JSON
> with multiple entities... bit like an RSS)  and map each entry to Solr
> documents?
>
> Thanks.
>
> Arcadius.
>