You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by bterhune <bt...@tripwire.com> on 2018/03/14 15:42:04 UTC
Camel Routing places KahaDB in an unexpected place
We recently switched our activemq (5.15.2) broker to broker routing from
using "Network of Brokers" to "Camel Routing" (2.13.0) using Spring XML
configuration.
In our activemq.xml configuration we specify the location of kahaDB
persistence.
We notice after our product restarts, camel appears to be creating a KahaDB
instance under the "Current Working Directory". Based on our current
deployment requirements this is not desirable. The directory format is
<WorkingDir>activemq-data/<brokerName>/KahaDB
Is there a way to specify a location for this mystery KahaDB we think Camel
is laying down?
If this cannot be changed, what is being stored? Can we assume this is
routing information and will not grow if our routes are static?
Thanks, Bob
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel Routing places KahaDB in an unexpected place
Posted by Tim Bain <tb...@alumni.duke.edu>.
Thank you for all of those explanations. The first solution (increasing the
time between retries) didn't make sense to me, but your final explanation
fully explains the behavior. Thanks for sharing the information with the
list, to help anyone who hits this in the future.
Tim
On Wed, Feb 6, 2019, 3:14 PM bterhune <bterhune@tripwire.com wrote:
> One last update.. The more I learn about what is going on, I realize the
> information I left out of my original post.
>
> The Camel configuration is connecting to the broker in question using the
> vm: protocol AND connecting to a remote broker. The VM connection was the
> one causing reconnect to spin up an embedded instance of the broker using
> default configuration.
>
> The change I made was to not create a broker if is not running.
>
> <property name="brokerURL" value="failover:(vm://my-broker?create=false)"
> />
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Re: Camel Routing places KahaDB in an unexpected place
Posted by bterhune <bt...@tripwire.com>.
One last update.. The more I learn about what is going on, I realize the
information I left out of my original post.
The Camel configuration is connecting to the broker in question using the
vm: protocol AND connecting to a remote broker. The VM connection was the
one causing reconnect to spin up an embedded instance of the broker using
default configuration.
The change I made was to not create a broker if is not running.
<property name="brokerURL" value="failover:(vm://my-broker?create=false)" />
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel Routing places KahaDB in an unexpected place
Posted by bterhune <bt...@tripwire.com>.
As a workaround I found setting "initialRecconectDelay=1500" on the failover
connection string caused enough delay to avoid the KahaDB from being
creating in the default location.
http://activemq.apache.org/failover-transport-reference.html
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel Routing places KahaDB in an unexpected place
Posted by bterhune <bt...@tripwire.com>.
Using ActiveMQ 5.15.8 now, I took your advice and put breakpoints around.
Here is what I found..
1. On startup I hit MessageDatabase.setDirectory() with our customer folder
"/data/activemq/kahadb"
2. KahaDB directories are correct, let some time pass for messages to pass
across broker
3. Shutdown.. (We launcher a broker instance from our Java application)
I hit a breakpoint in BrokerService.createPersistenceAdapter():2503
<https://gitbox.apache.org/repos/asf?p=activemq.git;a=blob;f=activemq-broker/src/main/java/org/apache/activemq/broker/BrokerService.java;h=f9cd50fe06920cd642c3d473a5d6713a4e80c866;hb=refs/heads/activemq-5.15.x#l2503>
Call Stack of breakpoint
getDataDirectoryFile:1096, BrokerService (org.apache.activemq.broker)
getBrokerDataDirectory:1101, BrokerService (org.apache.activemq.broker)
createPersistenceAdapter:2503, BrokerService (org.apache.activemq.broker)
getPersistenceAdapter:1267, BrokerService (org.apache.activemq.broker)
getSystemUsage:1179, BrokerService (org.apache.activemq.broker)
checkMemorySystemUsageLimits:2178, BrokerService
(org.apache.activemq.broker)
start:611, BrokerService (org.apache.activemq.broker)
doCompositeConnect:127, VMTransportFactory
(org.apache.activemq.transport.vm)
compositeConnect:91, TransportFactory (org.apache.activemq.transport)
* doReconnect:1011, FailoverTransport
(org.apache.activemq.transport.failover)*
iterate:148, FailoverTransport$2 (org.apache.activemq.transport.failover)
runTask:133, PooledTaskRunner (org.apache.activemq.thread)
run:48, PooledTaskRunner$1 (org.apache.activemq.thread)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)
If I had to guess, perhaps our camel connection to another broker is
attempting to re-establish the connection on broker shutdown and this is
causing an empty KahaDB to be created?
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel Routing places KahaDB in an unexpected place
Posted by Tim Bain <tb...@alumni.duke.edu>.
I'm still struggling to understand what exactly you're describing. To the
best of my knowledge, Camel doesn't use KahaDB, so that's making me unsure
of how to interpret what you've written. (Or if it does, please provide a
reference to documentation indicating how it does so, so I can learn
something I didn't already know.)
If I assume that Camel doesn't know anything at all about KahaDB and isn't
creating KahaDB files and that all such files are created by ActiveMQ, then
http://activemq.apache.org/kahadb.html describes the configuration options
needed to specify where the KahaDB database is written. You may have found
that page because you reference the <persistenceAdapter> configuration. Can
you please post that snippet of your configuration? Are you saying that
you're seeing two KahaDB instances, one in whatever directory you've
configured it and a second one in the directory you provided in your first
message? Or are you saying that there is only ever one set of KahaDB files,
but sometimes it's in the directory you chose and sometimes it's not?
To the best of my knowledge, ActiveMQ does not create KahaDB files on
shutdown, but rather on demand when messages need to be written to disk, so
I'm not what to make of the statement that the files are written only when
the broker is shut down. But it seems as though the path you described
would be created when calling
PersistenceAdapter.createPersistenceAdapter(), so you could attach a
debugger to the broker with a breakpoint on that method, and see under what
circumstances it's being called.
Tim
On Mon, Mar 19, 2018 at 11:56 AM, bterhune <bt...@tripwire.com> wrote:
> Thanks for looking at my message..
>
> We have two activeMQ brokers that send messages back and forth. We are
> now
> using camel and spring configuration to define those routes
>
> For example:
>
> <route id="SupportBundleResponse">
> <description>Support Bundle Response</description>
> <from id="F-A2G.SupportBundleResponse"
> uri="activeMqVM:topic:support.messages.
> SupportBundleResponse"/>
> <to id="T-A2G.SupportBundleResponse"
> uri="activeMqGW:topic:support.messages.SupportBundleResponse"/>
> </route>
>
> We are using persistence on topics but not durable subscribers.
>
> *Sometimes* when we shutdown, a KahaDB database is written to disk with the
> default of 32MB. Using the unix "strings" command to look at the db.data
> file the file contains all of our "producing" routes. I say sometimes
> because it only occurs about 50% of the time.
>
> As I mentioned we use <persistenceAdapter> configuration to place the
> KahaDB
> in a different location and limit the size to 16MB. So, what I am
> calling
> the "camel" kahaDB is not using our configuration location.
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Re: Camel Routing places KahaDB in an unexpected place
Posted by bterhune <bt...@tripwire.com>.
Thanks for looking at my message..
We have two activeMQ brokers that send messages back and forth. We are now
using camel and spring configuration to define those routes
For example:
<route id="SupportBundleResponse">
<description>Support Bundle Response</description>
<from id="F-A2G.SupportBundleResponse"
uri="activeMqVM:topic:support.messages.SupportBundleResponse"/>
<to id="T-A2G.SupportBundleResponse"
uri="activeMqGW:topic:support.messages.SupportBundleResponse"/>
</route>
We are using persistence on topics but not durable subscribers.
*Sometimes* when we shutdown, a KahaDB database is written to disk with the
default of 32MB. Using the unix "strings" command to look at the db.data
file the file contains all of our "producing" routes. I say sometimes
because it only occurs about 50% of the time.
As I mentioned we use <persistenceAdapter> configuration to place the KahaDB
in a different location and limit the size to 16MB. So, what I am calling
the "camel" kahaDB is not using our configuration location.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel Routing places KahaDB in an unexpected place
Posted by Tim Bain <tb...@alumni.duke.edu>.
Can you please provide a reference for what "Camel Routing" is in the
context of ActiveMQ, and how it's a replacement for a network of brokers,
or how Camel would cause the creation of KahaDB content? I've never heard
of either such thing and Google didn't shed any additional light, so as it
stands I have no idea what you're doing and therefore no idea how to help
you.
Also, if we ignore the assertion that Camel is in some way relevant to the
question, you say that "In our activemq.xml configuration we specify the
location of kahaDB
persistence." Can you please provide that portion of your activemq.xml?
Tim
On Wed, Mar 14, 2018, 10:45 AM bterhune <bt...@tripwire.com> wrote:
>
> We recently switched our activemq (5.15.2) broker to broker routing from
> using "Network of Brokers" to "Camel Routing" (2.13.0) using Spring XML
> configuration.
>
> In our activemq.xml configuration we specify the location of kahaDB
> persistence.
>
> We notice after our product restarts, camel appears to be creating a KahaDB
> instance under the "Current Working Directory". Based on our current
> deployment requirements this is not desirable. The directory format is
> <WorkingDir>activemq-data/<brokerName>/KahaDB
>
> Is there a way to specify a location for this mystery KahaDB we think Camel
> is laying down?
>
> If this cannot be changed, what is being stored? Can we assume this is
> routing information and will not grow if our routes are static?
>
> Thanks, Bob
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>