[jira] [Comment Edited] (TRAFODION-1578) Proposal for SPJ management

["Kevin Xu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995948#comment-14995948 ] 

Kevin Xu edited comment on TRAFODION-1578 at 11/9/15 5:04 AM:
--------------------------------------------------------------

For muti-thread DCS, it's DCSMaster->DCSServer->jdbcT2. For SPJ, it's DCSMaster->DCSServer->SQ Engine -> UDR->jdbcT2(JAVA-> C++ -> JAVA, why not JAVA -> JAVA). it's not necessary to create a new JVM for each connection. But for single JVM, it should load the JARs dynamicly as what UDR is doing and has security issue. So the proposal is to start a new JVM and manage its security by Java SecurityMangement. Contains SQL is trying to reuse JVM, right? As i mentioned, there is a customized idle timeout( contains non-stop) for standing by. 
yes, the users can modify JVM options directly, but it will have limitation on customized options like max heap size of JVM, the number of connections for one user. And also, allowed to start/restart. i'd like to do an experiment on a new command 'agent'.


was (Author: kevinxu021):
For muti-thread DCS, it's DCSMaster->DCSServer->jdbcT2. For SPJ, it's DCSMaster->DCSServer->UDR->jdbcT2(JAVA-> C++ -> JAVA, why not JAVA -> JAVA). it's not necessary to create a new JVM for each connection. But for single JVM, it should load the JARs dynamicly as what UDR is doing. Contains SQL is trying to reuse JVM, right? As i mentioned, there is a customized idle timeout( contains non-stop) for standing by. 
yes, the users can modify JVM options directly, but it will have limitation on customized options like max heap size of JVM, the number of connections for one user. And also, allowed to start/restart. i'd like to do an experiment on a new command 'agent'.

> Proposal for SPJ management
> ---------------------------
>
>                 Key: TRAFODION-1578
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-1578
>             Project: Apache Trafodion
>          Issue Type: Improvement
>          Components: connectivity-dcs
>            Reporter: Kevin Xu
>
> JAR upload process:
> 1. Initialize JAR upload procedure by default
> 2. JAR upload by Trafci(add library LIB_NAME JAR_LOCAL_PATH). Upload and create library will be done here. And also, you can only upload the JARs by UPLOAD command on Trafci that it will not create a lib.
>    Tips: Before put the JAR into HDFS check MD5 first, if the file exists, only add a record in metadata table in case users upload the same JAR many times on platform.
> 3. On server-side, the JAR will store in HDFS. At the same time JAR metadata(path in HDFS, MD5 of the file, and others) stores in store procedure metadata table.
> 4. create procedure is the same as now.
> JAR perform process:
> 1. Send a CALL by Trafci/JDBC/ODBC/ADO.NET.
> 2. DCSMaster assign a DCSServer for the CALL.
> 3. DCSServer start a JVM for the user. User can modify JVM options, program properties and JAVA classpath. At the same time, a monitor class will be starting in the JVM witch will register a node on Zookeeper for this JVM as well as metadata info( process id, server info and so on) and the node will be removed while JVM exiting. It allows customer to specify JVM idle time in case of some realtime senarior like Kafka consumer. 
> 4. Useful commands on Trafci: list all JVMs in user; kill one of them that no long in use; Restart JVMs with latest JARs and so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)