答复: TRAFODION-1578 Proposal for SPJ management

["Xu, Kai-Hua (Kevin)" <ka...@esgyn.cn>]

Add Hans and Venkat. I’m not sure whether you have seen it or not.

Best Regards,
Kevin Xu

发件人: Xu, Kai-Hua (Kevin)
发送时间: 2015年11月4日 18:44
收件人: 'dev@trafodion.incubator.apache.org' <de...@trafodion.incubator.apache.org>
主题: TRAFODION-1578 Proposal for SPJ management

Hi all,

As recommended, it should be better for Apache Trafodion. Hans/Venkat, thanks for your comments.


1.       Security: it may use Java SecurityManager and POLICY. It allows to set a particular directory access permission for a JVM.

2.       1) If the JARs don’t exist at the first time, the JARs will be copied to local, as well as when it has new updates. Check JARs every time once starting JVM.

2) Only update the JARs for the node needed. If node is down, JARs will be copied from HDFS again if needed(check MD5 if the file exists)
3.  Let’s create a procedure or sth for downloading and packing. Should be easy to pack the JARs.
4. As I said, there is an idle time for the JVM and DCSMaster will assign the same existing JVM for a CALL. Assume there is a JAR with size 10Mb, that it can be split into 10 parts(1Mb/part). Add a head for every part(1/10, 2/10 …). Validate MD5 while all parts are done.
5. JVM is isolated with its own process id. The PID is stored into Zookeeper, so list all JVM is trying to list all the JVMs that the current user owned. It might be implemented on DCS-side that it’ll be available for all client drivers.
6. Yes, check the size and the head of package as well as total size. It’s opensourced that some bad guys also can do that.


Best Regards,
Kevin Xu