You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by Alan Burlison <Al...@sun.com> on 2009/07/06 18:59:20 UTC
Derby replication seems to be mostly useless
I'm having an incredible struggle getting Derby replication to work - in
fact, I can't get it to work at all. The problem seems to be centred
around the use of user authentication in conjunction with replication,
and when trying to start the slave across the network rather than locally.
For startSlave the documentation on says:
"If authentication or authorization is turned on, you must also specify
this attribute in conjunction with the user=userName and
password=userPassword attributes. If authorization is turned on, the
user must be the database owner."
The example given is:
-- start slave using database name in subprotocol, default slave host
-- and port, authorization
jdbc:derby:myDB;startslave=true;user=mary;password=little88lamb
So the parameters I'm using for the connect() call are:
DB URL: jdbc:derby://localhost:1528/opensolaris
Properties: {user=admin, password=admin, securityMechanism=8,
startSlave=true, slaveHost=localhost}
and then I get the following exception:
DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC: Cannot start
replication slave mode for database 'opensolaris'. The database has
already been booted. [XRE09]
And the database absolutely *isn't* already booted - I'm starting the
database up just before I run the above connect(), and all I see in the
log file is the normal network server startup message:
Apache Derby Network Server - 10.5.1.1 - (764942) started and ready to
accept connections on port 1528
in derby.log, and if I single step through the debugger I seen nothing
else in the log until I step over the connect() call, at which point I
see the exception above appear in the log file.
Does replication actually work at all in my scenario?
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Dag H. Wanvik wrote:
> No, that should not matter unless there is a bug somewhere. Maybe we
> can best untangle these issues if you can provide a repro?
I'm collapsing the 2 users the code currently uses down into one, if
that doesn't fix it I'll provide a pointer to the code, which is on
opensolaris.org
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Knut Anders Hatlen wrote:
> I think that's true. We made the network server shut down all open
> databases cleanly in DERBY-1274, but only if it had been started from
> the command line. The rationale was that a network server that had been
> started programmatically was not necessarily the only user of the open
> databases, as they could also be accessed via the embedded driver by
> other threads within the same process.
But in the testing we've done it was possible to replicate the problem
just using ij & the command-line. How is that possible if the databases
are shut down?
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Dag.Wanvik@Sun.COM (Dag H. Wanvik) writes:
> Alan Burlison <Al...@Sun.COM> writes:
>
>> Alan Burlison wrote:
>>
>>> I didn't, and I just have - and it seems to work, or at least the
>>> startSlave connect hangs as I would expect. I'll go through my code
>>> and modify it to do the same before claiming victory though.
>>
>> And if I try exactly the same thing programatically I get the
>> following exception:
>>
>> Connection authentication failure occurred. Reason: security
>> mechanism not supported. [08004]
>>
>> I have no idea why it works from the command-line with ij but not from
>> my app.
>
> I have filed DERBY-4299 for this issue (most likely a variant of DERBY-3896).
>
> I suspect that when you use the programmatic approach, the databases
> are not shut down cleanly. If the servers are started from the command
> line, I think the booted databases are shut down cleanly.
I think that's true. We made the network server shut down all open
databases cleanly in DERBY-1274, but only if it had been started from
the command line. The rationale was that a network server that had been
started programmatically was not necessarily the only user of the open
databases, as they could also be accessed via the embedded driver by
other threads within the same process.
--
Knut Anders
Re: Derby replication seems to be mostly useless
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> Dag H. Wanvik wrote:
>
>> I suspect that when you use the programmatic approach, the databases
>> are not shut down cleanly. If the servers are started from the command
>> line, I think the booted databases are shut down cleanly.
>>
>> Cf these lines in NotworkServerControlImpl.java:
>> :
>> // if the server is started from the command line, it should shutdown the
>> // databases it has booted.
>> private boolean shutdownDatabasesOnShutdown = false;
>>
>> This means that when the server is started via the API, each database
>> needs to be shut down individually to get a clean shutdown (with
>> checkpoint).
>
> Unfortunately I don't think that's workable in practice. Because
> Derby auto-starts databases on connection, there is no way of stopping
> clients opening the database again whilst the copy and child start is
> happening. There needs to be some way to ensure the database is locked
> and then to flush the log entries. Would a
> SYSCS_UTIL.SYSCS_FREEZE_DATABASE followed by a
> SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE do the trick?
No, I'm afraid not. SYSCS_FREEZE_DATABASE already performs a
checkpoint. The problem is that the checkpoint is not allowed to remove
the log generated by transactions that are still active.
One solution could be to introduce a variant of freeze that fails if
there are active transactions which have generated log records. Then
you'd just have to retry until you can successfully freeze the database,
but that could go on forever if new transactions keep coming
in. Variants that block until the database is frozen (possibly also
preventing new transactions from starting) or that abort other active
transactions could perhaps also be used.
Could the following be a practical workaround?
1. Add a replication user that's only used to start/stop replication
2. Temporarily disable all other users by unsetting derby.user.XXX
3. Perform clean shutdown of master database
4. Copy database and start slave
5. Start master
6. Re-enable the non-replication users
This way I *think* you'd prevent others from booting the database while
you're setting up replication.
--
Knut Anders
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Dag H. Wanvik wrote:
> I suspect that when you use the programmatic approach, the databases
> are not shut down cleanly. If the servers are started from the command
> line, I think the booted databases are shut down cleanly.
>
> Cf these lines in NotworkServerControlImpl.java:
> :
> // if the server is started from the command line, it should shutdown the
> // databases it has booted.
> private boolean shutdownDatabasesOnShutdown = false;
>
> This means that when the server is started via the API, each database
> needs to be shut down individually to get a clean shutdown (with
> checkpoint).
Unfortunately I don't think that's workable in practice. Because Derby
auto-starts databases on connection, there is no way of stopping clients
opening the database again whilst the copy and child start is happening.
There needs to be some way to ensure the database is locked and then
to flush the log entries. Would a SYSCS_UTIL.SYSCS_FREEZE_DATABASE
followed by a SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE do the trick?
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> Alan Burlison wrote:
>
>> I didn't, and I just have - and it seems to work, or at least the
>> startSlave connect hangs as I would expect. I'll go through my code
>> and modify it to do the same before claiming victory though.
>
> And if I try exactly the same thing programatically I get the
> following exception:
>
> Connection authentication failure occurred. Reason: security
> mechanism not supported. [08004]
>
> I have no idea why it works from the command-line with ij but not from
> my app.
I have filed DERBY-4299 for this issue (most likely a variant of DERBY-3896).
I suspect that when you use the programmatic approach, the databases
are not shut down cleanly. If the servers are started from the command
line, I think the booted databases are shut down cleanly.
Cf these lines in NotworkServerControlImpl.java:
:
// if the server is started from the command line, it should shutdown the
// databases it has booted.
private boolean shutdownDatabasesOnShutdown = false;
This means that when the server is started via the API, each database
needs to be shut down individually to get a clean shutdown (with
checkpoint).
Dag
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Alan Burlison wrote:
> I didn't, and I just have - and it seems to work, or at least the
> startSlave connect hangs as I would expect. I'll go through my code and
> modify it to do the same before claiming victory though.
And if I try exactly the same thing programatically I get the following
exception:
Connection authentication failure occurred. Reason: security mechanism
not supported. [08004]
I have no idea why it works from the command-line with ij but not from
my app.
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Dag H. Wanvik wrote:
> Before we do that though, I notice that you are specifying the
> securityMechanism=8 on the client connect here. Just a long shot, did
> you try without that setting?
I didn't, and I just have - and it seems to work, or at least the
startSlave connect hangs as I would expect. I'll go through my code and
modify it to do the same before claiming victory though.
I thought that the securityMechanism was mandatory on client connections
if the database had been created with it enabled?
And what is also puzzling is Knut's example uses securitymechanism on
the connect, and that works...
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> Dag H. Wanvik wrote:
>
>> No, that should not matter unless there is a bug somewhere. Maybe we
>> can best untangle these issues if you can provide a repro?
>
> OK, I've replicated this from the command-line. Here are the steps I used:
Great that you can reproduce it, Alan. I'm willing to have a look if
you can send me (a link to) the database (I can access SWAN).
Before we do that though, I notice that you are specifying the
securityMechanism=8 on the client connect here. Just a long shot, did
you try without that setting?
Thanks,
Dag
>
> 1. Copy the database from the master to the slave, with the master
> shut down.
>
> 2. Start the database server on the slave:
>
> $ export CLASSPATH=/opt/db-derby/lib/derbyrun.jar
> $ cd /tmp/auth
> $ java org.apache.derby.drda.NetworkServerControl start -h bleaklow -p
> 1527 -noSecurityManager
> 2009-07-08 13:31:46.286 GMT : Apache Derby Network Server - 10.5.1.1 -
> (764942) started and ready to accept connections on port 1527
>
> 3. Start ij on the master and try to start the slave database in
> startSlave mode:
>
> ij> connect
> jdbc:derby://bleaklow/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
> ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC:
> Cannot start replication slave mode for database 'opensolaris'. The
> database has already been booted.
>
> I thought that perhaps the problem was that I was trying to put the
> slave DB into startSlave mode from a different machine to the one that
> the slave server was running on, so I tried opening up the closed DB
> on the master in startSlave mode and got an identical result:
>
> $ java org.apache.derby.drda.NetworkServerControl start -h fire -p
> 1528 -noSecurityManager
> 2009-07-08 13:53:04.557 GMT : Apache Derby Network Server - 10.5.1.1 -
> (764942) started and ready to accept connections on port 1528
>
> ij> connect
> jdbc:derby://fire/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
> ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC:
> Cannot start replication slave mode for database 'opensolaris'. The
> database has already been booted.
>
> I also tried switching everything to localhost, as in the example script:
>
>> ij> connect 'jdbc:derby://localhost/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
>> ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC: Cannot start replication slave mode for database 'opensolaris'. The database has already been booted
>
> So whatever the problem is, it seems to be to so with the contents of
> the particular database that I'm trying to replicate. I can provide a
> copy to anyone that wants to dig - it is a small (3.6Mb) test datbase
> with just dummy data in it.
>
> --
> Alan Burlison
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Dag H. Wanvik wrote:
> No, that should not matter unless there is a bug somewhere. Maybe we
> can best untangle these issues if you can provide a repro?
OK, I've replicated this from the command-line. Here are the steps I used:
1. Copy the database from the master to the slave, with the master shut
down.
2. Start the database server on the slave:
$ export CLASSPATH=/opt/db-derby/lib/derbyrun.jar
$ cd /tmp/auth
$ java org.apache.derby.drda.NetworkServerControl start -h bleaklow -p
1527 -noSecurityManager
2009-07-08 13:31:46.286 GMT : Apache Derby Network Server - 10.5.1.1 -
(764942) started and ready to accept connections on port 1527
3. Start ij on the master and try to start the slave database in
startSlave mode:
ij> connect
'jdbc:derby://bleaklow/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC:
Cannot start replication slave mode for database 'opensolaris'. The
database has already been booted.
I thought that perhaps the problem was that I was trying to put the
slave DB into startSlave mode from a different machine to the one that
the slave server was running on, so I tried opening up the closed DB on
the master in startSlave mode and got an identical result:
$ java org.apache.derby.drda.NetworkServerControl start -h fire -p 1528
-noSecurityManager
2009-07-08 13:53:04.557 GMT : Apache Derby Network Server - 10.5.1.1 -
(764942) started and ready to accept connections on port 1528
ij> connect
'jdbc:derby://fire/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC:
Cannot start replication slave mode for database 'opensolaris'. The
database has already been booted.
I also tried switching everything to localhost, as in the example script:
> ij> connect 'jdbc:derby://localhost/opensolaris;user=auth;password=CHANGEME;securityMechanism=8;startSlave=true';
> ERROR XRE09: DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC: Cannot start replication slave mode for database 'opensolaris'. The database has already been booted
So whatever the problem is, it seems to be to so with the contents of
the particular database that I'm trying to replicate. I can provide a
copy to anyone that wants to dig - it is a small (3.6Mb) test datbase
with just dummy data in it.
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by "Dag H. Wanvik" <Da...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> I'm wondering if this chaos is being caused by the fact that the derby
> database owner and the owner of the only schema in the database that
> is being replicated are different users? The database is created by
> user "admin", then the following DB configuration is done:
>
> derby.database.propertiesOnly = true
> derby.authentication.provider = BUILTIN
> derby.database.defaultConnectionMode = noAccess
> derby.database.requireAuthentication = true
> derby.user.admin = admin
> derby.user.auth = CHANGEME
> derby.database.fullAccessUsers = admin,auth
> create schema auth authorization auth
>
> The application then creates its tables in the auth schema. Could
> that be the issue?
No, that should not matter unless there is a bug somewhere. Maybe we
can best untangle these issues if you can provide a repro?
Dag
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Knut Anders Hatlen wrote:
> That is indeed strange. The derby.system.home property should only be
> used by the embedded driver (and by the network server since it uses the
> embedded driver). Setting it on the client side should not have any
> effect.
I've removed the setting of the derby.system.home property entirely but
the problem still persists. It seems to be that within a single JVM, if
the master is opened before the slave the slave open fails. That looks a
bit like there is some left-over state from the master open that is
affecting the slave open, but I have no idea a what.
There's another bizarre issue, I'm connecting to the master database
using the username & password of the database owner (the app uses a
different username & schema) and although the application gets back a
valid connection (the subsequent syscs_util.syscs_freeze_database works)
I see the following exception in derby.log:
ERROR 08004: Database connection refused.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at org.apache.derby.impl.sql.conn.GenericAuthorizer.refresh(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericAuthorizer.<init>(Unknown Source)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown
Source)
at org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at org.apache.derby.impl.drda.Database.makeDummyConnection(Unknown Source)
at
org.apache.derby.impl.drda.DRDAConnThread.validateSecMecUSRSSBPWD(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseACCSEC(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
How on earth can it be throwing that error yet still returning a
successful connection to the client application? I'm totally confused...
>> DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE04, SQLERRMC:
>> opensolaris.bleaklow.4851.XRE04.U.1 [XRE04]
>>
>> So there's obviously something else amiis as well.
>
> This error message is supposed to say
>
> Could not establish a connection to the peer of the replicated database
> 'opensolaris' on address 'bleaklow:4851'.
>
> (The unreadable message was fixed in DERBY-3417, but wasn't back-ported
> to 10.5.)
Ah OK, thanks - that explains that, at least.
> I'm wondering if this could be because the network server's default
> security policy doesn't allow connections to port 4851. If the root
> cause (found in derby.log on the server) is an AccessControlException
> that's probably the case. Either start the network server with
> -noSecurityManager or install a custom security policy which opens that
> port.
>
> http://db.apache.org/derby/docs/10.5/adminguide/cadminreplicsecurity.html
I don't think so, as I said the server is being started from within a
Java app rather than with the command-line utility, and security
policies aren't enabled.
What I see in the slave's derby.log just after the startMaster is:
Database Class Loader started - derby.database.classpath=''
2009-07-07 14:12:18.932 GMT Thread[DRDAConnThread_4,5,derby.daemons]
(XID = 529), (SESSIONID = 0), (DATABASE
= opensolaris), (DRDAID = null), Cleanup action starting
ERROR 08004: Database connection refused.
at
org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at
org.apache.derby.impl.sql.conn.GenericAuthorizer.refresh(Unknown Source)
at
org.apache.derby.impl.sql.conn.GenericAuthorizer.<init>(Unknown Source)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown
Source)
at
org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown
Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown
Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at
org.apache.derby.impl.drda.Database.makeDummyConnection(Unknown Source)
at
org.apache.derby.impl.drda.DRDAConnThread.validateSecMecUSRSSBPWD(Unknown
Source)
at
org.apache.derby.impl.drda.DRDAConnThread.parseACCSEC(Unknown Source)
at
org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
followed by a load more similar stack traces, then finally:
ERROR XRE09: Cannot start replication slave mode for database
'opensolaris'. The database has already been booted.
at
org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown
Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown
Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at org.apache.derby.impl.drda.Database.makeConnection(Unknown
Source)
at
org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown
Source)
at
org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown
Source)
at
org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown Source)
at
org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown
Source)
at
org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
which is complete rubbish as the database clearly hasn't been booted -
the startSlave causes the entire sequence of stack traces, and before
that call is made all that is in the log is the network server startup
message.
I'm wondering if this chaos is being caused by the fact that the derby
database owner and the owner of the only schema in the database that is
being replicated are different users? The database is created by user
"admin", then the following DB configuration is done:
derby.database.propertiesOnly = true
derby.authentication.provider = BUILTIN
derby.database.defaultConnectionMode = noAccess
derby.database.requireAuthentication = true
derby.user.admin = admin
derby.user.auth = CHANGEME
derby.database.fullAccessUsers = admin,auth
create schema auth authorization auth
The application then creates its tables in the auth schema. Could that
be the issue?
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> Knut Anders Hatlen wrote:
>
>> I think it should work, at least it did when I tested it in what I think
>> is a similar scenario. See the attached script which creates a database
>> with authentication and authorization enabled, starts slave and master
>> servers, and performs failover.
>>
>> Are you running the network server in a separate process, or is it
>> embedded inside another Java process? If the latter, it may be that the
>> database has already been booted when the server is started.
>
> The network server is a separate process, started from the command-line.
>
> I *think* I've partially figured out what the problem is. I'm running
> with two databases on the same machine, set up under two different
> Derby system directories, e.g.
>
> Master in /tmp/auth, containing database 'opensolaris', data files in
> /tmp/auth/opensolaris
>
> Slave in /tmp/auth1, containing database 'opensolaris' data files in
> /tmp/auth1/opensolaris
>
> I'm connecting to the master and the slave to start the replication
> from inside the same JVM, and although I'm setting derby.system.home
> before the connect calls, the derby docs say that derby.system.home is
> static so the first setting before I open a connection wins, and
> that's to the master - and I think that's what is causing the issues
> with the slave. In your test script you are doing the startSlave from
> a separate JVM, which is I guess why you aren't seeing the same
> problem.
>
> What is puzzling me is that I'm connecting to the slave using the
> network driver to databases running in a separate JVMs, so I would
> have expected the derby.system.home setting to be irrelevant, but it
> seems not.
That is indeed strange. The derby.system.home property should only be
used by the embedded driver (and by the network server since it uses the
embedded driver). Setting it on the client side should not have any
effect.
> To confirm that I've set up the slave on a separate machine, using the
> same derby.system.home as the master. If I do that the startSlave
> works just fine, which points to the issue being related to having the
> master and slave on the same machine but with different
> derby.system.home settings. As I said, I'm puzzled as to why
> derby.system.home should have any effect when I'm connecting with the
> network driver, but the evidence does seem to point in that direction.
>
> However even if I run with the slave on a different machine and get
> past the startSlave step, the subsequent startMaster fails with a
> bizarre-looking error message full of control characters:
>
> DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE04, SQLERRMC:
> opensolaris.bleaklow.4851.XRE04.U.1 [XRE04]
>
> So there's obviously something else amiis as well.
This error message is supposed to say
Could not establish a connection to the peer of the replicated database
'opensolaris' on address 'bleaklow:4851'.
(The unreadable message was fixed in DERBY-3417, but wasn't back-ported
to 10.5.)
I'm wondering if this could be because the network server's default
security policy doesn't allow connections to port 4851. If the root
cause (found in derby.log on the server) is an AccessControlException
that's probably the case. Either start the network server with
-noSecurityManager or install a custom security policy which opens that
port.
http://db.apache.org/derby/docs/10.5/adminguide/cadminreplicsecurity.html
--
Knut Anders
Re: Derby replication seems to be mostly useless
Posted by Alan Burlison <Al...@sun.com>.
Knut Anders Hatlen wrote:
> I think it should work, at least it did when I tested it in what I think
> is a similar scenario. See the attached script which creates a database
> with authentication and authorization enabled, starts slave and master
> servers, and performs failover.
>
> Are you running the network server in a separate process, or is it
> embedded inside another Java process? If the latter, it may be that the
> database has already been booted when the server is started.
The network server is a separate process, started from the command-line.
I *think* I've partially figured out what the problem is. I'm running
with two databases on the same machine, set up under two different Derby
system directories, e.g.
Master in /tmp/auth, containing database 'opensolaris', data files in
/tmp/auth/opensolaris
Slave in /tmp/auth1, containing database 'opensolaris' data files in
/tmp/auth1/opensolaris
I'm connecting to the master and the slave to start the replication from
inside the same JVM, and although I'm setting derby.system.home before
the connect calls, the derby docs say that derby.system.home is static
so the first setting before I open a connection wins, and that's to the
master - and I think that's what is causing the issues with the slave.
In your test script you are doing the startSlave from a separate JVM,
which is I guess why you aren't seeing the same problem.
What is puzzling me is that I'm connecting to the slave using the
network driver to databases running in a separate JVMs, so I would have
expected the derby.system.home setting to be irrelevant, but it seems not.
To confirm that I've set up the slave on a separate machine, using the
same derby.system.home as the master. If I do that the startSlave works
just fine, which points to the issue being related to having the master
and slave on the same machine but with different derby.system.home
settings. As I said, I'm puzzled as to why derby.system.home should
have any effect when I'm connecting with the network driver, but the
evidence does seem to point in that direction.
However even if I run with the slave on a different machine and get past
the startSlave step, the subsequent startMaster fails with a
bizarre-looking error message full of control characters:
DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE04, SQLERRMC:
opensolarisbleaklow4851XRE04.U.1 [XRE04]
So there's obviously something else amiis as well.
--
Alan Burlison
--
Re: Derby replication seems to be mostly useless
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Alan Burlison <Al...@Sun.COM> writes:
> I'm having an incredible struggle getting Derby replication to work -
> in fact, I can't get it to work at all. The problem seems to be
> centred around the use of user authentication in conjunction with
> replication, and when trying to start the slave across the network
> rather than locally.
>
> For startSlave the documentation on says:
>
> "If authentication or authorization is turned on, you must also
> specify this attribute in conjunction with the user=userName and
> password=userPassword attributes. If authorization is turned on, the
> user must be the database owner."
>
> The example given is:
>
> -- start slave using database name in subprotocol, default slave host
> -- and port, authorization
> jdbc:derby:myDB;startslave=true;user=mary;password=little88lamb
>
> So the parameters I'm using for the connect() call are:
>
> DB URL: jdbc:derby://localhost:1528/opensolaris
> Properties: {user=admin, password=admin, securityMechanism=8,
> startSlave=true, slaveHost=localhost}
>
> and then I get the following exception:
>
> DERBY SQL error: SQLCODE: -1, SQLSTATE: XRE09, SQLERRMC: Cannot start
> replication slave mode for database 'opensolaris'. The database has
> already been booted. [XRE09]
>
> And the database absolutely *isn't* already booted - I'm starting the
> database up just before I run the above connect(), and all I see in
> the log file is the normal network server startup message:
>
> Apache Derby Network Server - 10.5.1.1 - (764942) started and ready to
> accept connections on port 1528
>
> in derby.log, and if I single step through the debugger I seen nothing
> else in the log until I step over the connect() call, at which point I
> see the exception above appear in the log file.
>
> Does replication actually work at all in my scenario?
Hi Alan,
I think it should work, at least it did when I tested it in what I think
is a similar scenario. See the attached script which creates a database
with authentication and authorization enabled, starts slave and master
servers, and performs failover.
Are you running the network server in a separate process, or is it
embedded inside another Java process? If the latter, it may be that the
database has already been booted when the server is started.
--
Knut Anders