[GitHub] zeppelin issue #3092: [ZEPPELIN-3653] - New Java interpreter

[selvinsource <gi...@git.apache.org>]

Github user selvinsource commented on the issue:

    https://github.com/apache/zeppelin/pull/3092
  
    @zjffdu thanks for reviewing the PR.
    
    Let me give you a bit of background before I answer your question on using another java interpreter.
    
    First of all, I found lately Zeppelin an amazing tool to work with and in particular using the Spark/scala interpreter.
    However I also started evaluating / using Hazelcast Jet, which is a similar streaming/batch engine, entirely written in Java, and I immediately had a need to have that in Zeppelin to do some visualization of the results.
    
    I therefore started to look into how to create an interpreter for Jet and went through all the existing interpreter to get some ideas. I then realized that the Beam interpreter is just a java interpreter apart from the many dependencies on Beam/Google libraries.
    A solution could have been to clone the Beam interpreter to create a Jet one with dependencies only on Hazelcast Jet but that didn't seem the best solution (when you start cloning code).
    
    Hence my idea to create a Java interpreter (using what has been done and tested already with Beam) and have Beam (and eventually Hazelcast Jet) interpreters to simply extends the Java one with engine specific dependencies.
    
    So, this PR is a start, to have a common Java interpreter that other Java API/engine interpreters can extend.
    
    I agree the current implementation is not ideal as a whole class with a main method is needed to run some simple java code, the better approach is to use a Java REPL API.
    
    The obvious choice is Java 9 JShell as it is the only one that is going to be supported in the long run and no need of other dependencies as it is shipped as part of the JDK. I don't think it's wise to use any other Java REPL as they won't compete with JShell which will be the standard that every IDE will integrate with.
    
    However, the question back to you, is Zeppelin ready to be compiled with Java 9 and requiring Java 9 run time? Is it wise at this stage?
    Talking from experience, in production environments I have seen rarely Java 9, the Java community is waiting for the next LTS release which is Java 11 (due in Sep 2018).
    
    I would say to start with a Java interpreter like the one I proposed (re-using what's good was done for Beam), have Beam and eventually Jet extends the Java interpreter. Once Zeppelin will run on Java 11, the Java interpreter can be easily upgraded to use the JShell API therefore all other java based interpreters will have the same benefits (Beam, Jet, etc...).
    
    Definitely the Java interpreter will be much more usable and immediate with a REPL, it just need to wait for the JShell to be available.
    
    If you want I can already create a JIRA ticket to eventually update the Java interpreter to JShell as soon as this is available and I am happy to contribute to it: having seen the JShell API is going to be even simpler than the current REPL implementation that has been used in the Beam interpreter (but let's not forget that it works just fine!).


---