You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Olivier.Roger" <ol...@bsb.com> on 2010/11/08 16:09:31 UTC
Re: [Aggregator] Persistence ?
Hello Camel!
I was wondering if someone had already implemented the
org.apache.camel.spi.AggregationRepository for JDBC database ?
If not I would like to give it a try, Is there something I need to know
before I begin?
I read Camel In Action, chap. 8 sec.2 and analyzed the HawtDB implementation
and here is was I understand. Please correct me if I am wrong.
1. I need to implement the RecoverableAggregationRepository for my route to
be transactional
2. Two index (would be table with jdbc) are used to store exchanges, a
normal and a complete one.
2a. The normal index use the correlation id as key
2b. The complete index uses the exchanged ID as key
3. The remove operation move an exchange from the normal to the completed
index.
4. the scan method is run at regular interval to reprocess failed outgoing
exchanges (every 5 sec by default).
I have one question concerning the HawtDB implementation.
I see the doStart() method uses the ServiceHelper. I do not see any
documentation about that on the website, not in the Book either. Would I
need it in my implementation ?
Thanks in advance,
Olivier
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3255168.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
I created a ticket for it on Camel's JIRA :
https://issues.apache.org/jira/browse/CAMEL-3502
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3329163.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Dec 23, 2010 at 5:23 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> The code is now cleaned and migrated to 2.6 (was 2.4) for donation.
> A patch will be provided in the first days of 2011.
>
Ah perfect. Glad you got OK from your company to donate it.
The Camel community continues to kick ass!
> Olivier
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3316667.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
The code is now cleaned and migrated to 2.6 (was 2.4) for donation.
A patch will be provided in the first days of 2011.
Olivier
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3316667.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
About the Codec class, I use an adapted version of the HawtDBCamelCodec to
(un)marshall the exchanges to and from byte arrays. It uses a simple
ObjectInputStream to decode the objects.
However, as in Camel-3120 the aggregator bundle should have a
<DynamicImport-Package>*<DynamicImport-Package>. This permits to decode
object from classes exposed by other bundle in the OSGI runtime.
Even with that I had an issue with some JMS headers added to the exchanges.
I had to use the ClassLoadingAwareObjectInputStream from ActiveMQ
(http://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/util/ClassLoadingAwareObjectInputStream.java)
to avoid the ClassNotFoundException.
As I understand, the class uses a different a simple ObjectInputStream but
not with the current classloader.
It is replaced by the current thread ContextClassLoader. But I don't know
where that classloader is set.
Any ideas ?
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3286263.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Claus,
I don't exactly see how to reuse the code already present in Camel.
I'm not sure, but from what I saw the HawtDB aggregator uses the transaction
manager from HawtDB. In our use cases, using an XA Transaction Manager
option.
However, I was thinking to use Spring TransactionTemplate with an injected
TransactionManager instead of annotation. I think this would solve problem
concerning transactions.
Concerning the code donation, I have already adapted all the test for an H2
database. Still waiting for an answer ;)
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3284785.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Nov 22, 2010 at 4:32 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> I am sorry, I am not sure to understand how transacted works on the
> aggregator component.
>
> Does adding .transacted() to the aggregator include the repository
> operations inside transactions?
> Then, is this necessary when the aggregator already is in a transacted route
> ? Or does it has to be specified so that the operation could join the
> current transaction ?
>
Nah you may be right that it should be more tightly integrated into
your custom repository.
But there should be code we can reuse / stand on top of from the code
in Camel already.
So its maybe a matter of depending on camel-spring, and then have the
TM injected into the repo.
And then take a look at how we do in camel-spring in the
TransactedErrorHandler where we can kinda do the logic in a spring
method.
Kinda like what we do for hawtdb as well.
Anyway if you manage to have the code shared / donate to Apache we can
work on getting those final pieces into places so we can have it
working with transactions.
Would be cool if you could prepare unit tests which can run using an
embedded database such as HSQLDB or something similar.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3275510.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
I am sorry, I am not sure to understand how transacted works on the
aggregator component.
Does adding .transacted() to the aggregator include the repository
operations inside transactions?
Then, is this necessary when the aggregator already is in a transacted route
? Or does it has to be specified so that the operation could join the
current transaction ?
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3275510.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Nov 22, 2010 at 3:51 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> That would be great. In fact I already asked my employer for approval. I
> should have an answer in the coming days.
>
> I had another question about the transactional aspect.
> At the moment I added annotation in the JdbcAggregationRepository class
> which implements RecoverableAggregationRepository.
>
> I am quite new to Spring Transaction and I think I used an old way to enable
> Spring to create a proxy with transactional behavior. Here is the code I add
> in the application Context:
>
> <!-- Transaction AOP with Annotation -->
> <bean
> class="org.springframework.aop.framework.autoproxy.DefaultAdvisorAutoProxyCreator"/>
>
> <bean
> class="org.springframework.transaction.interceptor.TransactionAttributeSourceAdvisor">
> <property name="transactionInterceptor" ref="txInterceptor"/>
> </bean>
>
> <bean id="txInterceptor"
> class="org.springframework.transaction.interceptor.TransactionInterceptor">
> <property name="transactionManager" ref="PlatformTransactionManager"/>
> <property name="transactionAttributeSource">
> <bean
> class="org.springframework.transaction.annotation.AnnotationTransactionAttributeSource"/>
> </property>
> </bean>
>
> This means, that the aggregator repository will only be transactional this
> boilerplate code is added.
>
> In our uses case, the aggregator should always be transactional. Therefore,
> I would like to be able to load that code in a spring context defined in the
> project, like META-INF/camel/contex.xml. The user would then only have to
> provide the transactionManager.
>
> Is that possible ? If not is there a way to archive the same result in
> another way ?
>
Yeah we ought to be able to leverage the .transacted stuff already in Camel.
In fact you should be able to add .transacted as first step in the aggregator.
aggregate(xxxxx)
.transacted()
.to("yyyy")
.to("zzzz")
.end()
> Thanks for your help on this
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3275451.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
That would be great. In fact I already asked my employer for approval. I
should have an answer in the coming days.
I had another question about the transactional aspect.
At the moment I added annotation in the JdbcAggregationRepository class
which implements RecoverableAggregationRepository.
I am quite new to Spring Transaction and I think I used an old way to enable
Spring to create a proxy with transactional behavior. Here is the code I add
in the application Context:
<!-- Transaction AOP with Annotation -->
<bean
class="org.springframework.aop.framework.autoproxy.DefaultAdvisorAutoProxyCreator"/>
<bean
class="org.springframework.transaction.interceptor.TransactionAttributeSourceAdvisor">
<property name="transactionInterceptor" ref="txInterceptor"/>
</bean>
<bean id="txInterceptor"
class="org.springframework.transaction.interceptor.TransactionInterceptor">
<property name="transactionManager" ref="PlatformTransactionManager"/>
<property name="transactionAttributeSource">
<bean
class="org.springframework.transaction.annotation.AnnotationTransactionAttributeSource"/>
</property>
</bean>
This means, that the aggregator repository will only be transactional this
boilerplate code is added.
In our uses case, the aggregator should always be transactional. Therefore,
I would like to be able to load that code in a spring context defined in the
project, like META-INF/camel/contex.xml. The user would then only have to
provide the transactionManager.
Is that possible ? If not is there a way to archive the same result in
another way ?
Thanks for your help on this
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3275451.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Nov 19, 2010 at 8:12 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> Good news, all test also pass on Oracle :)
>
> Only the lobHandler had to be overridden from DefaultLobHandler to the
> OracleLobHandler from Spring.
> --
Yeah usually each different DB may have their own CLOB/BLOB type you
need to use.
But cool you got it sorted.
Maybe you would consider donating your work to Apache so we have an
alternative persistent store in the kit?
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3272864.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Good news, all test also pass on Oracle :)
Only the lobHandler had to be overridden from DefaultLobHandler to the
OracleLobHandler from Spring.
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3272864.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Yes, If I made the changes correctly.
Out of the 34 tests, I removed 2 because
1- I don't use buffer like HawtDB does (HawtDBAggregateBufferSizeTest)
2- I don't have a Service implementation like HawtDBFile
(HawtDBAggregationRepositoryLoadExistingTest)
I could not see any advantage to have a Service implementation for the
database, but I might be missing something here. Open to suggestions ;)
At the moment, my implementation is only composed of 2 classes:
1- JdbcAggregationRepository, the RecoverableAggregationRepository
implementation
2- JdbcCamelCodec, with (un) marshall Exchange <--> byte[]
I will now test it with a Oracle database instead of the in-memory H2 I used
for the unit tests
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3272041.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
Ah
Glad you got it working. So all the tests now passes?
On Fri, Nov 19, 2010 at 9:32 AM, Olivier.Roger <ol...@bsb.com> wrote:
>
> It's my bad, I was not calling the super.setUp() method properly :-/
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3272008.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
It's my bad, I was not calling the super.setUp() method properly :-/
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3272008.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
I am still looking but don't understand why the AggregateProcessor do not
execute the doStart() method.
Is that not the default behavior ?
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3271088.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
By using the debugger, I clearly see that the AgreggateProcessor.doStart()
method is not called for my implementation of the repository whereas it is
called for hawtDB Aggregation Repository.
I think this is the root of my problem.
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3271032.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
You are right, I had the source but I was looking at the wrong spot.
I have finished adapting the tests to my aggregator repository. All pass
except 2 of them that require a recovery for the background task.
I do not know why but the doStart() method seems not to be called on my
aggregator which extends ServiceSupport and implements
RecoverableAggregationRepository.
When I run the test for HawtDB the logs shows :
[main ] INFO HawtDBAggregationRepository - On startup there
are no existing aggregate exchanges (not completed) in repository: repo1
[main ] INFO HawtDBAggregationRepository - On startup there
are no completed exchanges to be recovered in repository: repo1-completed
[main ] INFO AggregateProcessor - Using
RecoverableAggregationRepository by scheduling recover checker to run every
5000 millis.
The third line comes from the aggregatorProcessor (aggregator2
implementation). It should start the aggregatorrepository before that by
calling doStart(), which log the 2 first lines.
Is there something I could have missed to register the aggregator as a
service ?
Thanks for your help!
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3270979.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Nov 17, 2010 at 11:06 AM, Olivier.Roger <ol...@bsb.com> wrote:
>
> Well, I modified the HawtDB codec to remove all hawtbuf import and simply
> (un)marshall the exchange to/from byte[]. Those byte arrays are kept in Blob
> field in the database using the default LobHandler.
>
> Key are stored directly as String and therefore, do not need to be
> converted.
> With theses modifications, the repository appears to be functional. At least
> my few unit test still pass.
>
Great sounds like you are on a good start and that a JDBC based
persistent store is on the way :)
> I also used Spring to set all method as Transactional.
> Therefore I would like to add unit tests to validate each possible use case
> (message lost at any step of the process, SQL exception, ...)
>
> I hoped I could start by adapting the tests done for
> HawtDBAggregationRepository but I could not find any tests in the
> distributed Jar. Is there somewhere I could find sample tests for a
> Persistent Aggregation Repository?
You can look in the source code
http://camel.apache.org/source.html
For the camel-hawtdb component.
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3268799.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Well, I modified the HawtDB codec to remove all hawtbuf import and simply
(un)marshall the exchange to/from byte[]. Those byte arrays are kept in Blob
field in the database using the default LobHandler.
Key are stored directly as String and therefore, do not need to be
converted.
With theses modifications, the repository appears to be functional. At least
my few unit test still pass.
I also used Spring to set all method as Transactional.
Therefore I would like to add unit tests to validate each possible use case
(message lost at any step of the process, SQL exception, ...)
I hoped I could start by adapting the tests done for
HawtDBAggregationRepository but I could not find any tests in the
distributed Jar. Is there somewhere I could find sample tests for a
Persistent Aggregation Repository?
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3268799.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Nov 10, 2010 at 10:12 AM, Olivier.Roger <ol...@bsb.com> wrote:
>
> Hello,
>
> I made a wrong assumption about the DefaultExchange.
> Using the DefaultExchangeHolder makes it really easy.
>
Great thats the point with it. Its actually used by eg camel-jms if
you set option transferExchange=true
> I made a first test. It's still a bit messy but it appears to gives the
> expected results/behavior.
> About that, I saw that the HawtDB codec marshall the key into a Buffer.
> I did not do that since all the keys are always String, and therefore stored
> in a Varchar field in the database.
>
Well I can't remember if we enforced correlation keys to be String types.
But they would most often be that. I would actually state its a fair
requirement that the correlation keys are String based as well.
Then you can use VARCHAR for those keys which makes it easy to query
the database.
> Did I miss something here or is it really HawtDB related ?
> I suppose there is a reason why the index are parametrized with
> <Buffer,Buffer> and not <String,Buffer>.
>
HawtDB is a storing all its data as byte streams, hence it uses a
Buffer abstraction. And you need codec to map to/from it.
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3258350.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Hello,
I made a wrong assumption about the DefaultExchange.
Using the DefaultExchangeHolder makes it really easy.
I made a first test. It's still a bit messy but it appears to gives the
expected results/behavior.
About that, I saw that the HawtDB codec marshall the key into a Buffer.
I did not do that since all the keys are always String, and therefore stored
in a Varchar field in the database.
Did I miss something here or is it really HawtDB related ?
I suppose there is a reason why the index are parametrized with
<Buffer,Buffer> and not <String,Buffer>.
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3258350.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yes DefaultExchange is NOT serializable. This is on purpose as the
data it holds may or may not be serializable.
Its your duty to handle how it should be serialized to/from the database.
However there is a holder object you can use provided in camel-core.
Which is used by mina/netty/jms/hawtdb to do that.
See the org.apache.camel.impl.DefaultExchangeHolder class
On Tue, Nov 9, 2010 at 3:54 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> You're right, the tables should not be created by the component in production
> environment.
>
> About the Serialization in a Blob field, I saw the HawtDB component uses a
> codec object from the HAwtbuf Library to perform the convertion between
> Exchange <--> Buffer.
> I supposed it was a restriction impose by HawtDB, but in my first tests with
> a H2 in-memory database I got a NotSerializableException
> http://camel.465427.n5.nabble.com/file/n3256967/NotSerializableException.stack.txt
> (stack) .
>
> I suppose DefaultExchange is serializable, so the issue probably comes from
> the Message body, am I right ? Or do I also need to implement a codec to
> perform the conversion ?
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3256967.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
You're right, the tables should not be created by the component in production
environment.
About the Serialization in a Blob field, I saw the HawtDB component uses a
codec object from the HAwtbuf Library to perform the convertion between
Exchange <--> Buffer.
I supposed it was a restriction impose by HawtDB, but in my first tests with
a H2 in-memory database I got a NotSerializableException
http://camel.465427.n5.nabble.com/file/n3256967/NotSerializableException.stack.txt
(stack) .
I suppose DefaultExchange is serializable, so the issue probably comes from
the Message body, am I right ? Or do I also need to implement a codec to
perform the conversion ?
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3256967.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Nov 9, 2010 at 1:31 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> Hi Claus,
>
> That seems to be what I needed.
>
> I still have a few questions:
>
> 1 - In what type of database field should I persist the Exchange objects?
> I was thinking to save it as a Blob, would that be a good fit ?
>
Yeah you need something like BLOB as the Exchange can contain any kind
of data: XML, objects, binary or whatever.
> 2 - About the Service lifecycle, I am not sure what the aggregator
> repository would do on start and stop.
> Do you see a reason why I should use it ?
> Maybe verify the required table are created, if not create them ?
>
You should only use it if you need it. So only override doStart /
doStop if you need to do some logic.
For example maybe people would like to be able to pre check the
database connection works and that the table exists.
You can allow end users an option to configure this.
Auto creating the table is good for development. But in production the
DBA's most likely would like a create script but having them be in
control of creating the table, set the indexes and whatnot.
A side note many of the NoSQL database is a good fit for this as they
tend to be key / value based which is essentially what's stored on the
Exchange.
key = body
value = content of body
key = header A
value = content of header A
etc.
So that leaves room for a 3rd variation :) (Well HawtDB is already a
key/value file based data store)
> Thanks for your help!
>
> Olivier
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3256717.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: [Aggregator] Persistence ?
Posted by "Olivier.Roger" <ol...@bsb.com>.
Hi Claus,
That seems to be what I needed.
I still have a few questions:
1 - In what type of database field should I persist the Exchange objects?
I was thinking to save it as a Blob, would that be a good fit ?
2 - About the Service lifecycle, I am not sure what the aggregator
repository would do on start and stop.
Do you see a reason why I should use it ?
Maybe verify the required table are created, if not create them ?
Thanks for your help!
Olivier
--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3256717.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [Aggregator] Persistence ?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Oliver
On Mon, Nov 8, 2010 at 4:09 PM, Olivier.Roger <ol...@bsb.com> wrote:
>
> Hello Camel!
>
> I was wondering if someone had already implemented the
> org.apache.camel.spi.AggregationRepository for JDBC database ?
> If not I would like to give it a try, Is there something I need to know
> before I begin?
>
We love contributions. So cool if you want to try out a JDBC based version.
> I read Camel In Action, chap. 8 sec.2 and analyzed the HawtDB implementation
> and here is was I understand. Please correct me if I am wrong.
>
> 1. I need to implement the RecoverableAggregationRepository for my route to
> be transactional
> 2. Two index (would be table with jdbc) are used to store exchanges, a
> normal and a complete one.
> 2a. The normal index use the correlation id as key
> 2b. The complete index uses the exchanged ID as key
> 3. The remove operation move an exchange from the normal to the completed
> index.
> 4. the scan method is run at regular interval to reprocess failed outgoing
> exchanges (every 5 sec by default).
Yeah thats correct.
>
> I have one question concerning the HawtDB implementation.
> I see the doStart() method uses the ServiceHelper. I do not see any
> documentation about that on the website, not in the Book either. Would I
> need it in my implementation ?
>
If your implementation is a Service then you got lifecycle callbacks from Camel
http://camel.apache.org/lifecycle.html
This makes it easy to do custom logic on start/stop etc.
In the book it ought to be in chapter 11 about developing components.
I will let Jonathan know so he can add a bit details about it.
> Thanks in advance,
>
> Olivier
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-Persistence-tp2800301p3255168.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/