You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Jørgen Løland <Jo...@Sun.COM> on 2008/02/13 10:52:39 UTC
Opinions please - some questions regarding replication
I have a few questions regarding replication that would be great
to hear the community's opinion on before the 10.4 branching. The
reason for bringing this up before code freeze is that these will
have impact on existing applications if changed later.
1. The master of a replicated database ships log records and
other information to the slave through a network socket. 8001
is currently default port for this communication. From
http://www.iana.org/assignments/port-numbers it seems that
this port is reserved for
vcom-tunnel 8001/tcp VCOM Tunnel
vcom-tunnel 8001/udp VCOM Tunnel
Should we change port, and maybe even register a port in IANA,
or is it fine as it is? I have no idea how long it will take
to register a port, but chances are it will be longer than the
time we have for 10.4 with the current plan. Another
alternative is to use a port from the range called "dynamic
and/or private ports" (49152 through 65535) BTW: I noticed
that 1527 is registered by Oracle, which may also be a
problem?
tlisrv 1527/tcp oracle
tlisrv 1527/udp oracle
Note that this is only a question of default port. The port
number may be set explicitly by the client starting
replication.
2. About a week ago, Dibyendu Majumdar raised a question about
why the replication modules are placed under services and not
store. Should replication be moved from
org.apache.derby.{iapi|impl}.services.replication to e.g.
org.apache.derby.{iapi|impl}.store.replication, or is it OK to keep
the current placement?
3. Should replication be shipped in a separate JAR file? Pros and
cons as I see it:
+ Remove 65KB (unjared) of class files from derby.jar
+ Possible to have a different security policy file for replication?
- Yet another JAR file
--
Jørgen Løland
Re: Opinions please - some questions regarding replication
Posted by Narayanan <V....@Sun.COM>.
Jørgen Løland wrote:
>
> 3. It is possible to make a separate jar for replication if we solve
> some sealing issues (Dan, Narayanan), but with only 65K of
> uncompressed class files it may not be worth it (Rick, Jørgen). *Not
> concluded yet.*
There are no sealing issues. I saw it wrongly. I guess the thread
concluded by saying sealing is not an issue.
But if this is not a generic replication implementation that would not
be usable by someone who say might
be working on a pluggable storage architecture for Derby I do not see
why it should not be tied to the store
and remain in derby.jar. Also I agree that it is not excessive bloat.
>
>
Re: Opinions please - some questions regarding replication
Posted by Jørgen Løland <Jo...@Sun.COM>.
A short summary of the thread so far:
1. Rick and Jørgen thinks it would be good to register a port. We will
use 8001 for now unless someone has another suggestion, but will write
clearly in the documentation that the default port is likely to change.
I will create a jira for this and then contact IANA to get a port.
2. Dan, Dibyendu and Jørgen thinks it would be good to move the
replication code to o.a.d.i.store.replication (since replication is
integrated with the store). I'll create a jira for this.
3. It is possible to make a separate jar for replication if we solve
some sealing issues (Dan, Narayanan), but with only 65K of uncompressed
class files it may not be worth it (Rick, Jørgen). *Not concluded yet.*
--
Jørgen Løland
Re: Opinions please - some questions regarding replication
Posted by Rick Hillegas <Ri...@Sun.COM>.
Hi Jørgen,
Some unscientific ramblings of a layman follow...
Jørgen Løland wrote:
> I have a few questions regarding replication that would be great
> to hear the community's opinion on before the 10.4 branching. The
> reason for bringing this up before code freeze is that these will
> have impact on existing applications if changed later.
>
> 1. The master of a replicated database ships log records and
> other information to the slave through a network socket. 8001
> is currently default port for this communication. From
> http://www.iana.org/assignments/port-numbers it seems that
> this port is reserved for
>
> vcom-tunnel 8001/tcp VCOM Tunnel
> vcom-tunnel 8001/udp VCOM Tunnel
>
> Should we change port, and maybe even register a port in IANA,
> or is it fine as it is? I have no idea how long it will take
> to register a port, but chances are it will be longer than the
> time we have for 10.4 with the current plan. Another
> alternative is to use a port from the range called "dynamic
> and/or private ports" (49152 through 65535) BTW: I noticed
> that 1527 is registered by Oracle, which may also be a
> problem?
At first blush, registering a port seems like a good idea. It ought to
reduce the likelihood of ugly surprises. I got the impression that the
top range ( dynamic and/or private ports) was for short-lived, transient
conversations. It's probably not the right range if you want to conduct
a long-running conversation on your port.
A little mousing around turned up these resources: The process for
registering a port is defined in section 19.9 of
http://www.rfc-editor.org/rfc/rfc4340.txt and the corresponding form is
at http://www.iana.org/cgi-bin/usr-port-number.pl It appears that IANA
wants to give you a port number rather than vice-versa. So if you want
to register a port, then our user documentation should say that the port
used in 10.4 is likely to change in a future release.
>
> tlisrv 1527/tcp oracle
> tlisrv 1527/udp oracle
>
> Note that this is only a question of default port. The port
> number may be set explicitly by the client starting
> replication.
Oops. Changing this would be a backward compatibility issue and I doubt
you'll find much passion for this in the community.
>
> 2. About a week ago, Dibyendu Majumdar raised a question about
> why the replication modules are placed under services and not
> store. Should replication be moved from
> org.apache.derby.{iapi|impl}.services.replication to e.g.
> org.apache.derby.{iapi|impl}.store.replication, or is it OK to keep
> the current placement?
>
> 3. Should replication be shipped in a separate JAR file? Pros and
> cons as I see it:
>
> + Remove 65KB (unjared) of class files from derby.jar
> + Possible to have a different security policy file for replication?
> - Yet another JAR file
>
This doesn't seem like excessive bloat to me.
Thanks,
-Rick
Re: Opinions please - some questions regarding replication
Posted by Daniel John Debrunner <dj...@apache.org>.
Narayanan wrote:
> Daniel John Debrunner wrote:
>> Knut Anders Hatlen wrote:
>>> Narayanan <V....@Sun.COM> writes:
>>
>>>> "If you try to load a class in a sealed package from another source
>>>> other than the JAR file in which the sealed package lives, the JVM
>>>> will throw a |SecurityException|."
>>>> here http://www.ibm.com/developerworks/library/j-jar/#N101D6 as the
>>>> last line in the Package sealing sub-header.
>>>
>>> By "load a class ... from another source" I think he means that the
>>> loaded class is located in another source, not that the code that causes
>>> the class to be loaded is in another source.
>>
>> Right, derbynet.jar uses sealed classes from derby.jar, so sealing is
>> not an issue here.
>>
>> Dan.
>>
> Well I guess I misinterpreted this. But just thinking aloud why isn't
> derby.jar sealed in
> entirety if sealed classes can anyway be used from outside?
>
> Well maybe because the presence of the class was required in other jars
> as well and
> we wanted to allow the derby build system to include these classes in
> the other jars
> without causing a sealing violation similar to Derby-2701 (keeping with
> the explanation
> given by knut).
Right, in some cases we need the same class in multiple jars (e.g.
sysinfo related classes), in other cases we put classes from different
jars in the same package (o.a.d.jdbc has embedded drivers from derby.jar
and client drivers from derbyclient.jar).
Dan.
Re: Opinions please - some questions regarding replication
Posted by Narayanan <V....@Sun.COM>.
Daniel John Debrunner wrote:
> Knut Anders Hatlen wrote:
>> Narayanan <V....@Sun.COM> writes:
>
>>> "If you try to load a class in a sealed package from another source
>>> other than the JAR file in which the sealed package lives, the JVM
>>> will throw a |SecurityException|."
>>> here http://www.ibm.com/developerworks/library/j-jar/#N101D6 as the
>>> last line in the Package sealing sub-header.
>>
>> By "load a class ... from another source" I think he means that the
>> loaded class is located in another source, not that the code that causes
>> the class to be loaded is in another source.
>
> Right, derbynet.jar uses sealed classes from derby.jar, so sealing is
> not an issue here.
>
> Dan.
>
Well I guess I misinterpreted this. But just thinking aloud why isn't
derby.jar sealed in
entirety if sealed classes can anyway be used from outside?
Well maybe because the presence of the class was required in other jars
as well and
we wanted to allow the derby build system to include these classes in
the other jars
without causing a sealing violation similar to Derby-2701 (keeping with
the explanation
given by knut).
Well I must admit I am not entirely clear about the sealing in the jar
building system of
derby. I will dig in more here.
Narayanan
Re: Opinions please - some questions regarding replication
Posted by Daniel John Debrunner <dj...@apache.org>.
Knut Anders Hatlen wrote:
> Narayanan <V....@Sun.COM> writes:
>> "If you try to load a class in a sealed package from another source
>> other than the JAR file in which the sealed package lives, the JVM
>> will throw a |SecurityException|."
>> here http://www.ibm.com/developerworks/library/j-jar/#N101D6 as the
>> last line in the Package sealing sub-header.
>
> By "load a class ... from another source" I think he means that the
> loaded class is located in another source, not that the code that causes
> the class to be loaded is in another source.
Right, derbynet.jar uses sealed classes from derby.jar, so sealing is
not an issue here.
Dan.
Re: Opinions please - some questions regarding replication
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Narayanan <V....@Sun.COM> writes:
> Daniel John Debrunner wrote:
>>
>> Sealing does not hide packages from other code, does it? I thought
>> it only prevented other code from inserting classes into the same
>> package.
>>
>> Dan.
> From Derby-2071 I can see that accessing classes from
> "org.apache.derby.iapi.services.io" in another jar can cause you a
> sealing violation.
I think that's because accessing classes from another jar will cause
Derby's build system to include those classes in that other jar as well,
unless it's explicitly told not to do so. See the findDependencies()
method in org.apache.derbyBuild.classlister.
> Also I found
>
> "If you try to load a class in a sealed package from another source
> other than the JAR file in which the sealed package lives, the JVM
> will throw a |SecurityException|."
> here http://www.ibm.com/developerworks/library/j-jar/#N101D6 as the
> last line in the Package sealing sub-header.
By "load a class ... from another source" I think he means that the
loaded class is located in another source, not that the code that causes
the class to be loaded is in another source.
--
Knut Anders
Re: Opinions please - some questions regarding replication
Posted by Narayanan <V....@Sun.COM>.
Daniel John Debrunner wrote:
> Narayanan wrote:
>> Jørgen Løland wrote:
>>> I have a few questions regarding replication that would be great
>>> to hear the community's opinion on before the 10.4 branching. The
>>> reason for bringing this up before code freeze is that these will
>>> have impact on existing applications if changed later.
>>>
>>>
>>> 3. Should replication be shipped in a separate JAR file? Pros and
>>> cons as I see it:
>>>
>>> + Remove 65KB (unjared) of class files from derby.jar
>>> + Possible to have a different security policy file for replication?
>>> - Yet another JAR file
>>>
>> I found the following imports in the SlaveController class.
>> import org.apache.derby.iapi.error.StandardException;
>>
>> import org.apache.derby.iapi.store.raw.RawStoreFactory;
>> import org.apache.derby.iapi.store.raw.log.LogFactory;
>> import org.apache.derby.impl.store.raw.log.LogToFile;
>>
>> If SlaveController were moved to a separate replication jar wouldn't
>> referencing the above classes
>> from derby.jar result in sealing violations? This would require
>> giving special
>> permissions for these packages in the manifest creation in build.xml
>> like this for example
>
> Sealing does not hide packages from other code, does it? I thought it
> only prevented other code from inserting classes into the same package.
>
> Dan.
From Derby-2071 I can see that accessing classes from
"org.apache.derby.iapi.services.io" in another jar can cause you a
sealing violation.
Also I found
"If you try to load a class in a sealed package from another source
other than the JAR file in which the sealed package lives, the JVM will
throw a |SecurityException|."
here http://www.ibm.com/developerworks/library/j-jar/#N101D6 as the last
line in the Package sealing sub-header.
Narayanan
Re: Opinions please - some questions regarding replication
Posted by Daniel John Debrunner <dj...@apache.org>.
Narayanan wrote:
> Jørgen Løland wrote:
>> I have a few questions regarding replication that would be great
>> to hear the community's opinion on before the 10.4 branching. The
>> reason for bringing this up before code freeze is that these will
>> have impact on existing applications if changed later.
>>
>>
>> 3. Should replication be shipped in a separate JAR file? Pros and
>> cons as I see it:
>>
>> + Remove 65KB (unjared) of class files from derby.jar
>> + Possible to have a different security policy file for replication?
>> - Yet another JAR file
>>
> I found the following imports in the SlaveController class.
> import org.apache.derby.iapi.error.StandardException;
>
> import org.apache.derby.iapi.store.raw.RawStoreFactory;
> import org.apache.derby.iapi.store.raw.log.LogFactory;
> import org.apache.derby.impl.store.raw.log.LogToFile;
>
> If SlaveController were moved to a separate replication jar wouldn't
> referencing the above classes
> from derby.jar result in sealing violations? This would require giving
> special
> permissions for these packages in the manifest creation in build.xml
> like this for example
Sealing does not hide packages from other code, does it? I thought it
only prevented other code from inserting classes into the same package.
Dan.
Re: Opinions please - some questions regarding replication
Posted by Narayanan <V....@Sun.COM>.
Jørgen Løland wrote:
> I have a few questions regarding replication that would be great
> to hear the community's opinion on before the 10.4 branching. The
> reason for bringing this up before code freeze is that these will
> have impact on existing applications if changed later.
>
>
> 3. Should replication be shipped in a separate JAR file? Pros and
> cons as I see it:
>
> + Remove 65KB (unjared) of class files from derby.jar
> + Possible to have a different security policy file for replication?
> - Yet another JAR file
>
I found the following imports in the SlaveController class.
import org.apache.derby.iapi.error.StandardException;
import org.apache.derby.iapi.store.raw.RawStoreFactory;
import org.apache.derby.iapi.store.raw.log.LogFactory;
import org.apache.derby.impl.store.raw.log.LogToFile;
If SlaveController were moved to a separate replication jar wouldn't
referencing the above classes
from derby.jar result in sealing violations? This would require giving
special
permissions for these packages in the manifest creation in build.xml
like this for example
<section name="org/apache/derby/impl/tools/sysinfo/">
<attribute name="Sealed" value="false"/>
</section>
<section name="org/apache/derby/iapi/services/context/">
<attribute name="Sealed" value="false"/>
</section>
<section name="org/apache/derby/iapi/services/info/">
<attribute name="Sealed" value="false"/>
</section>
<section name="org/apache/derby/jdbc/">
<attribute name="Sealed" value="false"/>
</section>
I am not sure it is good to unseal store level packages.
Narayanan
Re: Opinions please - some questions regarding replication
Posted by Daniel John Debrunner <dj...@apache.org>.
Jørgen Løland wrote:
> 2. About a week ago, Dibyendu Majumdar raised a question about
> why the replication modules are placed under services and not
> store. Should replication be moved from
> org.apache.derby.{iapi|impl}.services.replication to e.g.
> org.apache.derby.{iapi|impl}.store.replication, or is it OK to keep
> the current placement?
I don't see the replication code falling into services. They are meant
to be low-level, mostly self-contained modules. It seems that
replication has a need for store modules? Is the replication code a
layer on top of store, or integral with the store. If it's a separate
layer then I would suggest its own layer.
org.apache.derby.{iapi|impl}.replication
Dan.