TRAFODION-1578 Proposal for SPJ management

[xu...@126.com]

Hi all,

 

As recommended, it should be better for Apache Trafodion. Hans/Venkat,
thanks for your comments.

 

1.       Security: it may use Java SecurityManager and POLICY. It allows to
set a particular directory access permission for a JVM.

2.       1) If the JARs don't exist at the first time, the JARs will be
copied to local, as well as when it has new updates. Check JARs every time
once starting JVM.

2) Only update the JARs for the node needed. If node is down, JARs will be
copied from HDFS again if needed(check MD5 if the file exists)

3.  Let's create a procedure or sth for downloading and packing. Should be
easy to pack the JARs.

4. As I said, there is an idle time for the JVM and DCSMaster will assign
the same existing JVM for a CALL. Assume there is a JAR with size 10Mb, that
it can be split into 10 parts(1Mb/part). Add a head for every part(1/10,
2/10 .). Validate MD5 while all parts are done.

5. JVM is isolated with its own process id. The PID is stored into
Zookeeper, so list all JVM is trying to list all the JVMs that the current
user owned. It might be implemented on DCS-side that it'll be available for
all client drivers.

6. Yes, check the size and the head of package as well as total size. It's
opensourced that some bad guys also can do that.

 

 

Best Regards,

Kevin Xu