[jira] [Commented] (TWILL-261) Add supports to run TwillApplications against Kubernetes cluster

["Albert Shau (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TWILL-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866070#comment-16866070 ] 

Albert Shau commented on TWILL-261:
-----------------------------------

There is the beginnings of an implementation at [https://github.com/cdapio/cdap/tree/develop/cdap-kubernetes/src/main/java/io/cdap/cdap/k8s/runtime], though the implementations there are pretty specific to the needs of the CDAP project.

 

At a high level, there are Kubernetes implementations of the TwillRunner, TwillPreparer, TwillController, and other relevant classes. A Twill application is run as a Kubernetes Deployment, files to localize are set in a Kubernetes ConfigMap. However, we probably need to re-think some of the APIs that currently exist in Twill, as some of them are pretty specific to YARN and have no analogous functionality in Kubernetes. Some examples of these are setting log levels, classloader, classpath, etc. On the flip side, there are Kubernetes specific things as well, such as the image to run.

 

One way to do this may be to change the interface to only support those that are common to the two implementations, with the YARN specific and Kubernetes specific preparers having additional platform-specific setters. 

> Add supports to run TwillApplications against Kubernetes cluster
> ----------------------------------------------------------------
>
>                 Key: TWILL-261
>                 URL: https://issues.apache.org/jira/browse/TWILL-261
>             Project: Apache Twill
>          Issue Type: Story
>            Reporter: Terence Yim
>            Assignee: Terence Yim
>            Priority: Major
>
> Top level story to brainstorm and gather tasks needed in order to bring TwillApplication to Kubernetes cluster



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)