You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/08/03 07:28:22 UTC
[GitHub] [incubator-kyuubi] ulysses-you opened a new pull request, #3178: Add application operation docs
ulysses-you opened a new pull request, #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add docs for application operation plugin
### _How was this patch tested?_
<img width="822" alt="image" src="https://user-images.githubusercontent.com/12025282/182549580-efd7e56d-4b3a-4996-8c72-73aa22a4423a.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#issuecomment-1203587729
cc @yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936347073
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+For now, Kyuubi has already supported three built-in Application Operations: ``JpsApplicationOperation``, ``YarnApplicationOperation`` and ``KubernetesApplicationOperation``.
Review Comment:
application operations
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
Review Comment:
so you are missing the step of creating a service manifest?
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
Review Comment:
adding a ..
or
custom application operation`s`
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
Review Comment:
let's keep this caution
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
Review Comment:
configuring , extra cluster managerincluding, getting, information, killing
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
Review Comment:
keep a consistent style with authentication?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936598925
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,117 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``org.apache.kyuubi.engine.ApplicationOperation`` for extra cluster manager which provides an ability to control application, including getting information, killing.
+
+ .. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+ /**
+ * (killed or not, hint message)
+ */
+ type KillResponse = (Boolean, String)
+
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936383062
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
Review Comment:
added this step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936413494
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,97 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Build A Custom Application Operation
+---------------------
+
+Kyuubi supports configuring custom ``ApplicationOperation`` for certain extra cluster manager which provides an ability to control application, including getting information, killing.
+
+The steps of adding a custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+3. create a directory META-INF.services and a file with the fully-qualified name of the interface ``ApplicationOperation``:
+
+ .. code-block:: java
+
+ META-INF.services/org.apache.kyuubi.engine.ApplicationOperation
+
+ then add your fully-qualified name of custom application operation into the file.
+
+4. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+ .. code-block:: scala
Review Comment:
we used to put this the part of feature desc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r937283139
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,137 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Inside Kyuubi, the Kyuubi server uses the ``ApplicationManager`` module to manage all applications launched by itself, including different kinds of Kyuubi engines and self-contained applications.
+
+The ``ApplicationManager`` leverages methods provided by application operation implementations derived from ``org.apache.kyuubi.engine.ApplicationOperation`` to monitor the status of those applications and kill abnormal applications in case they get orphaned and may introduce more methods in the future.
+
+An ``ApplicationOperation`` implementation is usually built upon clients or APIs provided by cluster managers, such as Hadoop YARN, Kubernetes, etc.
+
+For now, Kyuubi has already supported serval built-in application operations:
+
+- ``JpsApplicationOperation``: an operation that can manage apps with a local process, e.g. a local mode spark application
+- ``YarnApplicationOperation``: an operation that can manage apps with a Hadoop Yarn cluster, e.g. a spark on yarn application
+- ``KubernetesApplicationOperation``: an operation that can manage apps with a k8s cluster, e.g. a spark on k8s application
+
+Besides those built-in ones, Kyuubi also supports loading custom ``ApplicationOperation``s through the Java `ServiceLoader`(SPI) for extra cluster managers.
Review Comment:
the highlighting here is malformed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936412758
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,97 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Build A Custom Application Operation
+---------------------
+
+Kyuubi supports configuring custom ``ApplicationOperation`` for certain extra cluster manager which provides an ability to control application, including getting information, killing.
+
+The steps of adding a custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
Review Comment:
for instance, we use Unordered Lists before
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936536320
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,116 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``ApplicationOperation`` for certain extra cluster manager which provides an ability to control application, including getting information, killing.
+
+ .. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+ /**
+ * (killed or not, hint message)
+ */
+ type KillResponse = (Boolean, String)
+
+ object ApplicationState extends Enumeration {
+ type ApplicationState = Value
+ val PENDING, RUNNING, FINISHED, KILLED, FAILED, ZOMBIE, NOT_FOUND, UNKNOWN = Value
+ }
+
+ case class ApplicationInfo(
+ id: String,
+ name: String,
+ state: ApplicationState,
+ url: Option[String] = None,
+ error: Option[String] = None)
+
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
Review Comment:
It's ok to use FQCN here. and tt's also much better to use `org.apache.kyuubi.engine.ApplicationOperation` the first time when it shows in this page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you closed pull request #3178: Add application operation docs
URL: https://github.com/apache/incubator-kyuubi/pull/3178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#issuecomment-1204684235
thank you for review, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936383363
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
Review Comment:
changed to `Build A Custom Application Operation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936357496
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
Review Comment:
KillResponse needs to be elaborated
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
Review Comment:
ApplicationInfo needs to be elaborated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936536979
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
Review Comment:
so the note is unnecessary now, as it obviously shown by the step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936544423
##########
docs/extensions/server/applications.rst:
##########
@@ -113,4 +82,54 @@ The steps of adding a custom Application Operation
url: Option[String] = None,
error: Option[String] = None)
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+Enable Custom Application Operation
+-----------------------------------
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+1. create a directory META-INF.services and a file with the fully-qualified name of the interface ``ApplicationOperation``:
Review Comment:
got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#issuecomment-1203692207
@yaooqinn addressed commnets and updated the snapshot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936455413
##########
docs/extensions/server/applications.rst:
##########
@@ -113,4 +82,54 @@ The steps of adding a custom Application Operation
url: Option[String] = None,
error: Option[String] = None)
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+Enable Custom Application Operation
+-----------------------------------
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+1. create a directory META-INF.services and a file with the fully-qualified name of the interface ``ApplicationOperation``:
Review Comment:
do you mean add `mvn package` in this stpes ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#issuecomment-1203643108
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3178](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7d9aa0a) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/024fa2db70cec76c40756930af2c09f363451d5b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (024fa2d) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #3178 +/- ##
============================================
- Coverage 51.40% 51.40% -0.01%
Complexity 6 6
============================================
Files 456 456
Lines 25422 25422
Branches 3540 3540
============================================
- Hits 13069 13067 -2
- Misses 11095 11098 +3
+ Partials 1258 1257 -1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...pache/kyuubi/engine/YarnApplicationOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvWWFybkFwcGxpY2F0aW9uT3BlcmF0aW9uLnNjYWxh) | `64.81% <0.00%> (-1.86%)` | :arrow_down: |
| [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `75.23% <0.00%> (-0.96%)` | :arrow_down: |
| [...yuubi/server/metadata/jdbc/JDBCMetadataStore.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvbWV0YWRhdGEvamRiYy9KREJDTWV0YWRhdGFTdG9yZS5zY2FsYQ==) | `89.27% <0.00%> (-0.70%)` | :arrow_down: |
| [...g/apache/kyuubi/operation/BatchJobSubmission.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQmF0Y2hKb2JTdWJtaXNzaW9uLnNjYWxh) | `74.40% <0.00%> (ø)` | |
| [...che/kyuubi/server/KyuubiTHttpFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpVEh0dHBGcm9udGVuZFNlcnZpY2Uuc2NhbGE=) | `64.96% <0.00%> (+0.72%)` | :arrow_up: |
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/3178/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `78.40% <0.00%> (+1.13%)` | :arrow_up: |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936539012
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,116 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``ApplicationOperation`` for certain extra cluster manager which provides an ability to control application, including getting information, killing.
Review Comment:
why do we have to use `certain` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936584267
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,117 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``org.apache.kyuubi.engine.ApplicationOperation`` for extra cluster manager which provides an ability to control application, including getting information, killing.
+
+ .. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+ /**
+ * (killed or not, hint message)
+ */
+ type KillResponse = (Boolean, String)
+
Review Comment:
.. code-block:: scala
and describe it a bit, see an example I give it to you https://github.com/apache/incubator-kyuubi/pull/3178/files#r936582755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936583558
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,117 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``org.apache.kyuubi.engine.ApplicationOperation`` for extra cluster manager which provides an ability to control application, including getting information, killing.
+
+ .. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
Review Comment:
.. code-block:: scala
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936428585
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
Review Comment:
oh, I got it now. updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936383787
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
+---------------------
-.. warning::
- This page is still in-progress.
+Kyuubi supports configure custom ``ApplicationOperation`` for certain cluster manager which provides an ability to control application, includes get application info and kill application.
+
+The steps of adding custom Application Operation
+-------------------------------------------------------
+
+1. reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+2. create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+3. compile and put the jar into ``$KYUUBI_HOME/jars``
+
+.. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
Review Comment:
added the related code block
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936539359
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,116 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``ApplicationOperation`` for certain extra cluster manager which provides an ability to control application, including getting information, killing.
Review Comment:
`which` here is for `cluster manager`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#issuecomment-1203676153
you also missed how to impl the ApplicationState converter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936432845
##########
docs/extensions/server/applications.rst:
##########
@@ -113,4 +82,54 @@ The steps of adding a custom Application Operation
url: Option[String] = None,
error: Option[String] = None)
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+Enable Custom Application Operation
+-----------------------------------
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+1. create a directory META-INF.services and a file with the fully-qualified name of the interface ``ApplicationOperation``:
Review Comment:
this is better to be added in the steps of building tar ball?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936533413
##########
docs/extensions/server/applications.rst:
##########
@@ -113,4 +82,54 @@ The steps of adding a custom Application Operation
url: Option[String] = None,
error: Option[String] = None)
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+Enable Custom Application Operation
+-----------------------------------
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+1. create a directory META-INF.services and a file with the fully-qualified name of the interface ``ApplicationOperation``:
Review Comment:
no, I mean this belongs to the above section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936582755
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,117 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Kyuubi supports configuring custom ``org.apache.kyuubi.engine.ApplicationOperation`` for extra cluster manager which provides an ability to control application, including getting information, killing.
Review Comment:
```suggestion
Inside Kyuubi, the Kyuubi server uses the ``ApplicationManager`` module to manage all applications launched by itself, including different kinds of Kyuubi engines and self-contained applications.
The ``ApplicationManager`` leverages methods provided by application operation implementations derived from ``org.apache.kyuubi.engine.ApplicationOperation`` to monitor the status of those applications and kill abnormal applications in case they get orphaned and may introduce more methods in the future.
An ``ApplicationOperation`` implementation is usually built upon clients or APIs provided by cluster managers, such as Hadoop YARN, Kubernetes, etc.
For now, Kyuubi has already supported serval built-in application operations:
- ``JpsApplicationOperation``: an operation that can manage apps with a local process, e.g. a local mode spark application
- ``YarnApplicationOperation``: an operation that can manage apps with a Hadoop Yarn cluster, e.g. a spark on yarn application
- ``KubernetesApplicationOperation``: an operation that can manage apps with a k8s cluster, e.g. a spark on k8s application
Besides those built-in ones, Kyuubi also supports loading custom ``ApplicationOperation``s through the Java `ServiceLoader`(SPI) for extra cluster managers.
The rest of this article will show you the specifications and steps to build and enable a custom operation.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r937292030
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,137 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Inside Kyuubi, the Kyuubi server uses the ``ApplicationManager`` module to manage all applications launched by itself, including different kinds of Kyuubi engines and self-contained applications.
+
+The ``ApplicationManager`` leverages methods provided by application operation implementations derived from ``org.apache.kyuubi.engine.ApplicationOperation`` to monitor the status of those applications and kill abnormal applications in case they get orphaned and may introduce more methods in the future.
+
+An ``ApplicationOperation`` implementation is usually built upon clients or APIs provided by cluster managers, such as Hadoop YARN, Kubernetes, etc.
+
+For now, Kyuubi has already supported serval built-in application operations:
+
+- ``JpsApplicationOperation``: an operation that can manage apps with a local process, e.g. a local mode spark application
+- ``YarnApplicationOperation``: an operation that can manage apps with a Hadoop Yarn cluster, e.g. a spark on yarn application
+- ``KubernetesApplicationOperation``: an operation that can manage apps with a k8s cluster, e.g. a spark on k8s application
+
+Besides those built-in ones, Kyuubi also supports loading custom ``ApplicationOperation``s through the Java `ServiceLoader`(SPI) for extra cluster managers.
Review Comment:
it seems it can not accept other word after ` without a blank, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r936411715
##########
docs/extensions/server/applications.rst:
##########
@@ -18,7 +18,71 @@ Manage Applications against Extra Cluster Managers
.. versionadded:: 1.6.0
-.. caution:: unstable
+Application Operation
Review Comment:
I didn't mean only this one but the full hierarchy of this page
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a diff in pull request #3178: Add application operation docs
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on code in PR #3178:
URL: https://github.com/apache/incubator-kyuubi/pull/3178#discussion_r937283867
##########
docs/extensions/server/applications.rst:
##########
@@ -20,5 +20,137 @@ Manage Applications against Extra Cluster Managers
.. caution:: unstable
-.. warning::
- This page is still in-progress.
+Inside Kyuubi, the Kyuubi server uses the ``ApplicationManager`` module to manage all applications launched by itself, including different kinds of Kyuubi engines and self-contained applications.
+
+The ``ApplicationManager`` leverages methods provided by application operation implementations derived from ``org.apache.kyuubi.engine.ApplicationOperation`` to monitor the status of those applications and kill abnormal applications in case they get orphaned and may introduce more methods in the future.
+
+An ``ApplicationOperation`` implementation is usually built upon clients or APIs provided by cluster managers, such as Hadoop YARN, Kubernetes, etc.
+
+For now, Kyuubi has already supported serval built-in application operations:
+
+- ``JpsApplicationOperation``: an operation that can manage apps with a local process, e.g. a local mode spark application
+- ``YarnApplicationOperation``: an operation that can manage apps with a Hadoop Yarn cluster, e.g. a spark on yarn application
+- ``KubernetesApplicationOperation``: an operation that can manage apps with a k8s cluster, e.g. a spark on k8s application
+
+Besides those built-in ones, Kyuubi also supports loading custom ``ApplicationOperation``s through the Java `ServiceLoader`(SPI) for extra cluster managers.
+
+The rest of this article will show you the specifications and steps to build and enable a custom operation.
+
+ .. code-block:: scala
+
+ trait ApplicationOperation {
+
+ /**
+ * Step for initializing the instance.
+ */
+ def initialize(conf: KyuubiConf): Unit
+
+ /**
+ * Step to clean up the instance
+ */
+ def stop(): Unit
+
+ /**
+ * Called before other method to do a quick skip
+ *
+ * @param clusterManager the underlying cluster manager or just local instance
+ */
+ def isSupported(clusterManager: Option[String]): Boolean
+
+ /**
+ * Kill the app/engine by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * For example,
+ * if the Hadoop Yarn is used, for spark applications,
+ * the tag will be preset via spark.yarn.tags
+ * @return a message contains response describing how the kill process.
+ *
+ * @note For implementations, please suppress exceptions and always return KillResponse
+ */
+ def killApplicationByTag(tag: String): KillResponse
+
+ /**
+ * Get the engine/application status by the unique application tag
+ *
+ * @param tag the unique application tag for engine instance.
+ * @return [[ApplicationInfo]]
+ */
+ def getApplicationInfoByTag(tag: String): ApplicationInfo
+ }
+
+ .. code-block:: scala
+
+ /**
+ * (killed or not, hint message)
+ */
+ type KillResponse = (Boolean, String)
+
+An ``ApplicationInfo`` is used to represented the application information, including application id, name, state, url address and error message.
+
+ .. code-block:: scala
+
+ object ApplicationState extends Enumeration {
+ type ApplicationState = Value
+ val PENDING, RUNNING, FINISHED, KILLED, FAILED, ZOMBIE, NOT_FOUND, UNKNOWN = Value
+ }
+
+ case class ApplicationInfo(
+ id: String,
+ name: String,
+ state: ApplicationState,
+ url: Option[String] = None,
+ error: Option[String] = None)
+
+For application state mapping, you can reference the implementation of yarn:
+
+ .. code-block:: scala
+
+ def toApplicationState(state: YarnApplicationState): ApplicationState = state match {
+ case YarnApplicationState.NEW => ApplicationState.PENDING
+ case YarnApplicationState.NEW_SAVING => ApplicationState.PENDING
+ case YarnApplicationState.SUBMITTED => ApplicationState.PENDING
+ case YarnApplicationState.ACCEPTED => ApplicationState.PENDING
+ case YarnApplicationState.RUNNING => ApplicationState.RUNNING
+ case YarnApplicationState.FINISHED => ApplicationState.FINISHED
+ case YarnApplicationState.FAILED => ApplicationState.FAILED
+ case YarnApplicationState.KILLED => ApplicationState.KILLED
+ case _ =>
+ warn(s"The yarn driver state: $state is not supported, " +
+ "mark the application state as UNKNOWN.")
+ ApplicationState.UNKNOWN
+ }
+
+Build A Custom Application Operation
+------------------------------------
+
+- reference kyuubi-server
+
+ .. code-block:: xml
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-server_2.12</artifactId>
+ <version>1.5.2-incubating</version>
+ <scope>provided</scope>
+ </dependency>
+
+- create a custom class which implements the ``org.apache.kyuubi.engine.ApplicationOperation``.
+
+- create a directory META-INF.services and a file with ``org.apache.kyuubi.engine.ApplicationOperation``:
+
+ .. code-block:: java
+
+ META-INF.services/org.apache.kyuubi.engine.ApplicationOperation
+
+ then add your fully-qualified name of custom application operation into the file.
+
+
+Enable Custom Application Operation
+-----------------------------------
+
+.. note:: Kyuubi uses Java SPI to load the custom Application Operation
+
+- compile and put the jar into ``$KYUUBI_HOME/jars``
+
+For now, Kyuubi has already supported three built-in application operations: ``JpsApplicationOperation``, ``YarnApplicationOperation`` and ``KubernetesApplicationOperation``.
Review Comment:
can be removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org