Need help trying to pull from paged web api

[Dale King <da...@gmail.com>]

I am a newbie to Camel and am trying to set up Camel route(s) to tackle
this situation.

I want to update a local database with records that are made available
through a web API and periodically pull in new records to the database.

Unfortunately, the only API to access the records is a paged API where all
records are available from newest to oldest. The web API takes offset and
length parameters to specify where to offset into the list and how many
records to return, which it returns as XML.

So the logic, needs to be that I start at an offset of zero and load pages
of records until I get to a record that I already have in my database
(determined by the record ID). In the case of an initial load I would keep
loading pages until there is no more data (determined by getting an empty
result back).

Once I have all the new records I need to put them into the database as one
DB transaction.

I understand the pieces to schedule routes, and to actually do html call to
get back the XML results, but am struggling how to handle the looping
nature of the task.

I want to do the job as much as possible in camel.

Any suggestions would be appreciated

-- 
Dale King