Re: [jira] Commented: (DERBY-700) Derby does not prevent dual boot of database from different classloaders on Linux

[Mike Matrigali <mi...@sbcglobal.net>]

Overall I prefer your proposal to mine, assuming it works.  Have you
had a chance to prototype this on a non-windows system?  I couldn't
find a problem in the pseudo-code from reading it.

My concerns are:
1) Can we count on class finalizers?  What, if any situations are there
    that should have released the lock but we missed because the class
    finalizer was not run?  At least the result of this problem is a
    false positive rather than a corrupt db.
2) Is it ok to require the new system properties/permissions - see
    below.
3) What level JVM do we require for this solution, off hand I was 
wondering if we are counting a specific level for fine grained security
permision for the system property setting.

Other comments below


Suresh Thalamati (JIRA) wrote:
>     [ https://issues.apache.org/jira/browse/DERBY-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12484998 ] 
> 
> Suresh Thalamati commented on DERBY-700:
> ----------------------------------------
> 
> Thanks a lot for summarizing the problems and possible solutions 
> for this issue, Mike. I think the timer base solution you mentioned
> might work, but I am not comfortable with a timer based solution. As you
> mentioned, users might complain about the background writes, and also 
> I think  configuring N to the right value to differentiate false 
> negatives/positives boots going to be hard. It will depend on the load 
> and the machine configuration (no of cpus ) ..etc.
> 
> I was trying to find alternative solutions, without much success. Only
> solution I could come up with involves using a system property. I 
> understand, earlier we discussed using the system properties and it was 
> decided as not a such a good idea. But considering there are NO other 
> better solutions found for this problem, so far. I was think having one 
> property to maintain a JVMID may not be so bad, user just need to give 
> security permission to set one property, i.e  if what I 
> describe below actually works!
> 
> I would really appreciate  any  suggestions/feedback  for this solution . 
> 
> My understanding is a solution to this problem need to solve primarily 
> following three issues:
> 
> 1) Maintaining a state that a database is already booted, if the database
>    if booted successfully. 
> 2) Change the state to NOT_BOOTED, if it is not booted any more because of  a
>     a) Shutdown of the database
>     b) Class loader that booted the db is garbage collected.
>     c) JVM exited. 
>  
> 3) synchronization across class loaders. 
> 
> Pseudo code below that attempts to solve this problems by making the 
> following Assumptions :
> 
>  1) It is ok to use ONE  system property  "derby.storage.jvmid" to identify 
>      a jvm instance id. 
This seems reasonable to me.  It really is a system property that is 
meant to be shared across class loaders.  Unlike some of our system 
properties which would be nice to isolate to particular class loaders.
>  2) It is ok to use interned strings to synchronize across class loader. 
I don't see any problem with this, though it might be nice to have a 
test that verifies this works.  It seems sort of a tricky thing to count 
on, so would be nice to verify it works on whatever jvm we run tests on.
It is the only way so far I have thought of to synchronize safely across
class loader.
>  3) It is ok to call getCanonicalPath(), i think this may require permission 
>     for "user.dir" property if it is not already required. Other solution
>     may be to assign an ID string on create of the DB and user that for 
>     DB level synchronization. 
I don't know much about this issue.  Any thoughts from people looking
at increased security in 10.3?

>  4) It is ok to rely on the class finalizer to cleanup db lock state, 
>     when the database  is NOT any more because the loader that booted 
>     the database is garbage  collected. 
I can't think of any other way, other than the crude timer approach.  As
usual be careful not to code a possible deadlock into the finalizer 
routine.
> 
> 
> /*
> Pseudo code to lock the DB to prevent multiple instance of a database running 
> concurrently through class loaders in a single instance of jvm or
> multiple instance of jvm.   
> 
> Note: Following code class is in a separate class just to understand it 
> as separate issue , this code should probably go into the 
> dataFactory class where current db-locking is done. 
> */
> Class DbLock {
> 
>     private static final String DERYB_JVM_ID  = "derby.storage.jvmid";
>     private String dbCannonicalPath;   // canonical of the db being booted.
>     private FileLock fileLock  = null;
>     private boolean dbLocked = false;
> 
>     DbLock (String dbCannonicalPath) {
>         this.dbCannonicalPath = dbCannonicalPath;
>     }
> 
>     /* 
>      * get a unique JVM ID 
>      */
>     private getJvmId () {
>         // synchronize across class loaders.
>         synchronize(DERYB_JVM_ID) {
> 
>             jvmid = System.getProperty(DERYB_JVM_ID);
>             // if jvm id is not already exist, generate one 
>             // and save it into the "derby.storage.jvmid" system
>             // property.
>             if (jvmid == null) {
>                 //generate a new UUID based on the time and IP ..etc. 
>                 jvmid = generateJvmId() 
>                     System.setProperty("derby.storage.jvmid");
>             }
>         }
>     }
> 
>     /*
>      *  Lock the db,  so that other class loader or
>      *  another jvm won't be able to boot the same database.
>      */
>     public lock_db_onboot(String dbCannonicalPath)  {
> 
>          // Get a file Lock on boot() ; // this already works 
>          fileLock = getFileLock("dbex.lck");
>          if (lock == null) {
>              // if we don't get lock means , some other jvm already 
>              // booted it throws  ALREADY_BOOTED error.
>              throw ALREADY_BOOTED;
>          } else {
> 
>              // file lock can be acquired even if the database is already 
>              // booted by a different class loader. Check if another class 
>              // loader has booted  the DB.  This is done by checking the 
>              // JVMID written in the dbex.lck  file. If the JVMID is same 
>              // as what is stored in the system property,
>              // then database is already booted , throw the error. 
>              currentJvmId =  getJvmId();
>              synchronize(dbCannonicalPath) {
>                  onDisk_JVM_ID = readIdFromDisk() ; // read ID from the dbex.lck file.
>                  if (OnDisk_JVM_ID == current_jvm_id ) 
>                      throw ("DATABASE IS ALREADY BOOTED");  
>                  else{
>                      dbLocked = true;
>                      writeId(currentJvmId);  //update the dbex.lck file) . 
>                  }
>              }
>          }
>     }
> 
> 
> 
>     /*
>      * Called on shutdown/garbage collection. 
>      */ 
>     unlock_db() {
>         if (dbLocked) {
>             Strinng Ondisk_jvm_id =  "-1";  //jvm id should never have been a -1.
>             synchronize(dbCannonicalPath) {
>                 writeIdToDisk(Ondisk_jvm_id);  //update the dbex.lck file) . 
>             }
>             releaseFileLock(fileLock);
>             dbLocked =  false;
>         }
>     }
> 
> 
>     /*
>      * if the db is not shutdown, this method should release 
>      * the  db lock related resources during this class finalization.
>      */
>     protected void finalize() throws Throwable 
>     {
>         unlock_db();
>     }
> }
> 
> 
> 
> 
>>Derby does not prevent dual boot of database from different classloaders on Linux
>>---------------------------------------------------------------------------------
>>
>>                Key: DERBY-700
>>                URL: https://issues.apache.org/jira/browse/DERBY-700
>>            Project: Derby
>>         Issue Type: Bug
>>         Components: Store
>>   Affects Versions: 10.1.2.1
>>        Environment: ava -version
>>java version "1.4.2_08"
>>Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_08-b03)
>>Java HotSpot(TM) Client VM (build 1.4.2_08-b03, mixed mode)
>>           Reporter: Kathey Marsden
>>           Priority: Critical
>>        Attachments: DERBY-700.diff, DERBY-700.stat, DERBY-700_v1_use_to_run_DualBootrepro_multithreaded.diff, DERBY-700_v1_use_to_run_DualBootrepro_multithreaded.stat, DualBootRepro.java, DualBootRepro2.zip, DualBootRepro_mutltithreaded.tar.bz2
>>
>>
>>Derby does not prevent dual boot from two different classloaders on Linux.
>>To reproduce run the  program DualBootRepro with no derby jars in your classpath. The program assumes derby.jar is in 10.1.2.1/derby.jar, you can change the location by changing the DERBY_LIB_DIR variable.
>>On Linux the output is:
>>$java -cp . DualBootRepro
>>Loading derby from file:10.1.2.1/derby.jar
>>10.1.2.1/derby.jar
>>Booted database in loader java.net.URLClassLoader@8ed465
>>FAIL: Booted database in 2nd loader java.net.URLClassLoader@dc6a77
>>On Windows I get the expected output.
>>$ java -cp . DualBootRepro
>>Loading derby from file:10.1.2.1/derby.jar
>>10.1.2.1/derby.jar
>>Booted database in loader java.net.URLClassLoader@1ac04e8
>>PASS: Expected exception for dualboot:Another instance of Derby may have already booted the database D:\marsden\repro\dualboot\mydb.
> 
>