You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Christian Müller <ch...@gmail.com> on 2011/04/04 17:13:54 UTC
Improvement of the Idempotent Repository
Hello list!
I'm thinking about to add a timestamp property to the idempotent repository
(at least to the JDBC and JPA based one). The reason is the following:
The individual records in our batch file doesn't have a unique id which we
can use as key for the idempotent consumer. Our solution was to get the
entire line and build a hash for the key.
The problems comes, after we have been asked to remove the entries in the
idempotent repository from the last processed file, because it should be
processed again. We could not do this, because we don't know which entries
(hashes) belongs to the last file we processed.
Our short time solution is to extend the database table we use and add a
'created_at' column which is filled by the database with a default value to
'sysdate' (Oracle). I'm wondering whether we should support this by default
in further versions of Camel (set the created_at timestamp) at least in the
JDBC and JPA based idempotent repositories.
What you are thinking?
Cheers,
Christian
Re: Improvement of the Idempotent Repository
Posted by Christian Mueller <ch...@gmail.com>.
Done: https://issues.apache.org/jira/browse/CAMEL-3975
--
View this message in context: http://camel.465427.n5.nabble.com/Improvement-of-the-Idempotent-Repository-tp4282010p4395346.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Improvement of the Idempotent Repository
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, May 10, 2011 at 11:20 AM, Christian Mueller
<ch...@gmail.com> wrote:
> No opinions about this?
>
I think its a good idea if we can do it using pure standard SQL so it
works with any database.
Feel free to create a JIRA ticket so we can track this.
> Best,
> christian
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Improvement-of-the-Idempotent-Repository-tp4282010p4384036.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Improvement of the Idempotent Repository
Posted by Christian Mueller <ch...@gmail.com>.
No opinions about this?
Best,
christian
--
View this message in context: http://camel.465427.n5.nabble.com/Improvement-of-the-Idempotent-Repository-tp4282010p4384036.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Improvement of the Idempotent Repository
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
We need to make it database agnostic so we can't have oracle specific
sql functions in there.
However we should be able to add a row with created_timstamp and use a
java.util.Date object type.
JPA should be able to map that to any database.
On Mon, Apr 4, 2011 at 5:13 PM, Christian Müller
<ch...@gmail.com> wrote:
> Hello list!
>
> I'm thinking about to add a timestamp property to the idempotent repository
> (at least to the JDBC and JPA based one). The reason is the following:
> The individual records in our batch file doesn't have a unique id which we
> can use as key for the idempotent consumer. Our solution was to get the
> entire line and build a hash for the key.
> The problems comes, after we have been asked to remove the entries in the
> idempotent repository from the last processed file, because it should be
> processed again. We could not do this, because we don't know which entries
> (hashes) belongs to the last file we processed.
> Our short time solution is to extend the database table we use and add a
> 'created_at' column which is filled by the database with a default value to
> 'sysdate' (Oracle). I'm wondering whether we should support this by default
> in further versions of Camel (set the created_at timestamp) at least in the
> JDBC and JPA based idempotent repositories.
>
> What you are thinking?
>
> Cheers,
> Christian
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/