You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/10/03 19:00:46 UTC
[GitHub] [incubator-kyuubi] timothy65535 opened a new pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
timothy65535 opened a new pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
For more detail, please go to https://github.com/apache/incubator-kyuubi/issues/981
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a9b5084) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head a9b5084 differs from pull request most recent head 52daf13. Consider uploading reports for the commit 52daf13 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5975 +169
Branches 614 635 +21
=======================================
- Misses 5806 5975 +169
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/engine/ShareLevel.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvU2hhcmVMZXZlbC5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...52daf13](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r724717853
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -875,7 +875,7 @@ object KyuubiConf {
.toSequence()
.checkValue(_.toSet.subsetOf(Set("SPARK", "JSON", "JDBC", "CUSTOM")),
"Unsupported event loggers")
- .createWithDefault(Nil)
+ .createWithDefault(Seq("SPARK"))
Review comment:
OK. let's add some unit tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r725971225
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
you mean let `engine.event.loggers` to`Nil`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726726087
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
Review comment:
Shall we use Client IP here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726731801
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
Review comment:
Yeah.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r724675596
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -875,7 +875,7 @@ object KyuubiConf {
.toSequence()
.checkValue(_.toSet.subsetOf(Set("SPARK", "JSON", "JDBC", "CUSTOM")),
"Unsupported event loggers")
- .createWithDefault(Nil)
+ .createWithDefault(Seq("SPARK"))
Review comment:
unnecessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (dd729d7) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/1b1b8981237888884bc3f7a78f62eb137c9f4802?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1b1b898) will **not change** coverage.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1b1b898...dd729d7](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726729136
##########
File path: docs/deployment/settings.md
##########
@@ -178,7 +178,7 @@ kyuubi\.engine<br>\.deregister\.exception<br>\.messages|<div style='width: 65pt;
kyuubi\.engine<br>\.deregister\.exception<br>\.ttl|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT30M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Time to live(TTL) for exceptions pattern specified in kyuubi.engine.deregister.exception.classes and kyuubi.engine.deregister.exception.messages to deregister engines. Once the total error count hits the kyuubi.engine.deregister.job.max.failures within the TTL, an engine will deregister itself and wait for self-terminated. Otherwise, we suppose that the engine has recovered from temporary failures.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.2.0</div>
kyuubi\.engine<br>\.deregister\.job\.max<br>\.failures|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>4</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Number of failures of job before deregistering the engine.</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.2.0</div>
kyuubi\.engine\.event<br>\.json\.log\.path|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>file:/tmp/kyuubi/events</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The location of all the engine events go for the builtin JSON logger.<ul><li>Local Path: start with 'file:'</li><li>HDFS Path: start with 'hdfs:'</li></ul></div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.3.0</div>
-kyuubi\.engine\.event<br>\.loggers|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark history events</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul></div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.3.0</div>
+kyuubi\.engine\.event<br>\.loggers|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>SPARK</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark history events</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul></div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.3.0</div>
Review comment:
the description here need to be updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726729708
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
Review comment:
move my comment https://github.com/apache/incubator-kyuubi/pull/1187/files#r726726087 here is more suitable. sorry for the misleading
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (12e6fb1) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d1e7f0cc20ff2321d2bcd50849206cc50740fb07?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1e7f0c) will **decrease** coverage by `0.06%`.
> The diff coverage is `75.65%`.
> :exclamation: Current head 12e6fb1 differs from pull request most recent head 0e647bf. Consider uploading reports for the commit 0e647bf to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
============================================
- Coverage 79.06% 78.99% -0.07%
Complexity 93 93
============================================
Files 182 182
Lines 6681 6827 +146
Branches 783 800 +17
============================================
+ Hits 5282 5393 +111
- Misses 946 966 +20
- Partials 453 468 +15
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `77.77% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `80.95% <66.66%> (-2.39%)` | :arrow_down: |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `80.21% <75.71%> (-15.02%)` | :arrow_down: |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `65.33% <85.71%> (+1.44%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.33% <100.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d1e7f0c...0e647bf](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726722597
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
hi @yaooqinn, we need to push event to spark listenerbus first, then update the store.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r725964974
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
can we test the negative case that you mentioned https://github.com/apache/incubator-kyuubi/pull/1187/files#r724715328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r724939545
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Execute", true, None))
Review comment:
`Total Statements` seems better ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (dd729d7) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head dd729d7 differs from pull request most recent head 99f4bdd. Consider uploading reports for the commit 99f4bdd to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...99f4bdd](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-940625528
Final view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (dd729d7) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head dd729d7 differs from pull request most recent head 892a633. Consider uploading reports for the commit 892a633 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...892a633](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726731719
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
Review comment:
Got
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726726270
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
+ case "Session ID" => Ordering.by(_.sessionId)
+ case "Start Time" => Ordering by (_.startTime)
+ case "Finish Time" => Ordering.by(_.endTime)
+ case "Duration" => Ordering.by(_.totalTime)
Review comment:
shall we name totalTime to Duration to be consistent?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726734839
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-939864066
Thanks @yaooqinn @ulysses-you for the suggestion, patch is updated accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726725904
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
I see, you were right. so can we update the description of `SPARK` item, it's now not history only, but to spark listen bus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726704313
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
I see, but why do we have to use SPARK here for the in-memory store, SPARK is used to turn on a writer to spark history
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726732945
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
May just change the desc to `SPARK: the events will be written to the spark listener bus.` is enough. Because `history` or `store` are different listeners.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (12e6fb1) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/1b1b8981237888884bc3f7a78f62eb137c9f4802?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1b1b898) will **increase** coverage by `78.99%`.
> The diff coverage is `75.65%`.
> :exclamation: Current head 12e6fb1 differs from pull request most recent head dd729d7. Consider uploading reports for the commit dd729d7 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=============================================
+ Coverage 0.00% 78.99% +78.99%
- Complexity 0 93 +93
=============================================
Files 156 182 +26
Lines 5806 6827 +1021
Branches 614 800 +186
=============================================
+ Hits 0 5393 +5393
+ Misses 5806 966 -4840
- Partials 0 468 +468
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `77.77% <0.00%> (+77.77%)` | :arrow_up: |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `80.95% <66.66%> (+80.95%)` | :arrow_up: |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `80.21% <75.71%> (+80.21%)` | :arrow_up: |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `65.33% <85.71%> (+65.33%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.33% <100.00%> (+95.33%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/events/JsonEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvSnNvbkV2ZW50TG9nZ2VyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...main/java/org/apache/kyuubi/jdbc/KyuubiDriver.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhpdmUtamRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3l1dWJpL2pkYmMvS3l1dWJpRHJpdmVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/kyuubi/sql/KyuubiAnalysis.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGV2L2t5dXViaS1leHRlbnNpb24tc3BhcmstMy0xL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL3NxbC9LeXV1YmlBbmFseXNpcy5zY2FsYQ==) | `84.11% <0.00%> (ø)` | |
| [...pache/kyuubi/jdbc/hive/KyuubiDatabaseMetaData.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhpdmUtamRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3l1dWJpL2pkYmMvaGl2ZS9LeXV1YmlEYXRhYmFzZU1ldGFEYXRhLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../kyuubi/sql/watchdog/KyuubiWatchDogException.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGV2L2t5dXViaS1leHRlbnNpb24tc3BhcmstMy0xL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL3NxbC93YXRjaGRvZy9LeXV1YmlXYXRjaERvZ0V4Y2VwdGlvbi5zY2FsYQ==) | `100.00% <0.00%> (ø)` | |
| ... and [163 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1b1b898...dd729d7](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (52daf13) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5975 +169
Branches 614 635 +21
=======================================
- Misses 5806 5975 +169
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/engine/ShareLevel.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvU2hhcmVMZXZlbC5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...52daf13](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726005826
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
We can not let `engine.event.loggers` to `Nil`, because the kyuubiconf will check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (12e6fb1) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d1e7f0cc20ff2321d2bcd50849206cc50740fb07?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1e7f0c) will **decrease** coverage by `0.06%`.
> The diff coverage is `75.65%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
============================================
- Coverage 79.06% 78.99% -0.07%
Complexity 93 93
============================================
Files 182 182
Lines 6681 6827 +146
Branches 783 800 +17
============================================
+ Hits 5282 5393 +111
- Misses 946 966 +20
- Partials 453 468 +15
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `77.77% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `80.95% <66.66%> (-2.39%)` | :arrow_down: |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `80.21% <75.71%> (-15.02%)` | :arrow_down: |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `65.33% <85.71%> (+1.44%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.33% <100.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d1e7f0c...12e6fb1](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726729709
##########
File path: docs/deployment/settings.md
##########
@@ -178,7 +178,7 @@ kyuubi\.engine<br>\.deregister\.exception<br>\.messages|<div style='width: 65pt;
kyuubi\.engine<br>\.deregister\.exception<br>\.ttl|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT30M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Time to live(TTL) for exceptions pattern specified in kyuubi.engine.deregister.exception.classes and kyuubi.engine.deregister.exception.messages to deregister engines. Once the total error count hits the kyuubi.engine.deregister.job.max.failures within the TTL, an engine will deregister itself and wait for self-terminated. Otherwise, we suppose that the engine has recovered from temporary failures.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.2.0</div>
kyuubi\.engine<br>\.deregister\.job\.max<br>\.failures|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>4</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Number of failures of job before deregistering the engine.</div>|<div style='width: 30pt'>int</div>|<div style='width: 20pt'>1.2.0</div>
kyuubi\.engine\.event<br>\.json\.log\.path|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>file:/tmp/kyuubi/events</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The location of all the engine events go for the builtin JSON logger.<ul><li>Local Path: start with 'file:'</li><li>HDFS Path: start with 'hdfs:'</li></ul></div>|<div style='width: 30pt'>string</div>|<div style='width: 20pt'>1.3.0</div>
-kyuubi\.engine\.event<br>\.loggers|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark history events</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul></div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.3.0</div>
+kyuubi\.engine\.event<br>\.loggers|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>SPARK</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark history events</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul></div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.3.0</div>
Review comment:
Got
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (99f4bdd) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head 99f4bdd differs from pull request most recent head a9b5084. Consider uploading reports for the commit a9b5084 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...a9b5084](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r724715328
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -875,7 +875,7 @@ object KyuubiConf {
.toSequence()
.checkValue(_.toSet.subsetOf(Set("SPARK", "JSON", "JDBC", "CUSTOM")),
"Unsupported event loggers")
- .createWithDefault(Nil)
+ .createWithDefault(Seq("SPARK"))
Review comment:
if keep `Nil`, session stats section will show nothing on sever page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726728972
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
+ case "Session ID" => Ordering.by(_.sessionId)
+ case "Start Time" => Ordering by (_.startTime)
+ case "Finish Time" => Ordering.by(_.endTime)
+ case "Duration" => Ordering.by(_.totalTime)
Review comment:
make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r725979300
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
We can turn off this feature by set it to nil as you expected,right? it make it necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (99f4bdd) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head 99f4bdd differs from pull request most recent head 36676f2. Consider uploading reports for the commit 36676f2 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...36676f2](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726728276
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
Review comment:
This ip is client ip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726728797
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
Review comment:
I mean show `Client IP` on the web UI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r725971225
##########
File path: externals/kyuubi-spark-sql-engine/src/test/scala/org/apache/spark/kyuubi/ui/EngineTabSuite.scala
##########
@@ -64,5 +64,41 @@ class EngineTabSuite extends WithSparkSQLEngine with JDBCTestUtils {
}
}
+ test("session stats for engine tab") {
Review comment:
you mean let `engine.event.loggers` to`Nil`? It maybe seems not a bit necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#issuecomment-933117511
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1187](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (99f4bdd) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/fdff2b62408c9fbede4c81935422aaea1c5752cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdff2b6) will **not change** coverage.
> The diff coverage is `0.00%`.
> :exclamation: Current head 99f4bdd differs from pull request most recent head 52daf13. Consider uploading reports for the commit 52daf13 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1187 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 156 156
Lines 5806 5952 +146
Branches 614 631 +17
=======================================
- Misses 5806 5952 +146
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...n/scala/org/apache/spark/kyuubi/ui/EngineTab.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVRhYi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [fdff2b6...52daf13](https://codecov.io/gh/apache/incubator-kyuubi/pull/1187?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #1187: [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #1187:
URL: https://github.com/apache/incubator-kyuubi/pull/1187#discussion_r726728419
##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EnginePage.scala
##########
@@ -83,4 +92,244 @@ case class EnginePage(parent: EngineTab) extends WebUIPage("") {
Seq.empty
}
}
+
+ /** Generate stats of sessions for the engine */
+ private def generateSessionStatsTable(request: HttpServletRequest): Seq[Node] = {
+ val numSessions = store.getSessionList.size
+ val table = if (numSessions > 0) {
+
+ val sessionTableTag = "sessionstat"
+
+ val sessionTablePage =
+ Option(request.getParameter(s"$sessionTableTag.page")).map(_.toInt).getOrElse(1)
+
+ try {
+ Some(new SessionStatsPagedTable(
+ request,
+ parent,
+ store.getSessionList,
+ "kyuubi",
+ UIUtils.prependBaseUri(request, parent.basePath),
+ sessionTableTag
+ ).table(sessionTablePage))
+ } catch {
+ case e@(_: IllegalArgumentException | _: IndexOutOfBoundsException) =>
+ Some(<div class="alert alert-error">
+ <p>Error while rendering job table:</p>
+ <pre>
+ {Utils.stringifyException(e)}
+ </pre>
+ </div>)
+ }
+ } else {
+ None
+ }
+
+ val content =
+ <span id="sessionstat" class="collapse-aggregated-sessionstat collapse-table"
+ onClick="collapseTable('collapse-aggregated-sessionstat',
+ 'aggregated-sessionstat')">
+ <h4>
+ <span class="collapse-table-arrow arrow-open"></span>
+ <a>Session Statistics ({numSessions})</a>
+ </h4>
+ </span> ++
+ <div class="aggregated-sessionstat collapsible-table">
+ {table.getOrElse("No statistics have been generated yet.")}
+ </div>
+
+ content
+ }
+
+ private class SessionStatsPagedTable(
+ request: HttpServletRequest,
+ parent: EngineTab,
+ data: Seq[SessionEvent],
+ subPath: String,
+ basePath: String,
+ sessionStatsTableTag: String) extends PagedTable[SessionEvent] {
+
+ private val (sortColumn, desc, pageSize) =
+ getRequestTableParameters(request, sessionStatsTableTag, "Start Time")
+
+ private val encodedSortColumn = URLEncoder.encode(sortColumn, UTF_8.name())
+
+ private val parameterPath =
+ s"$basePath/$subPath/?${getRequestParameterOtherTable(request, sessionStatsTableTag)}"
+
+ override val dataSource = new SessionStatsTableDataSource(data, pageSize, sortColumn, desc)
+
+ override def tableId: String = sessionStatsTableTag
+
+ override def tableCssClass: String =
+ "table table-bordered table-sm table-striped table-head-clickable table-cell-width-limited"
+
+ override def pageLink(page: Int): String = {
+ parameterPath +
+ s"&$pageNumberFormField=$page" +
+ s"&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc" +
+ s"&$pageSizeFormField=$pageSize" +
+ s"#$sessionStatsTableTag"
+ }
+
+ override def pageSizeFormField: String = s"$sessionStatsTableTag.pageSize"
+
+ override def pageNumberFormField: String = s"$sessionStatsTableTag.page"
+
+ override def goButtonFormPath: String =
+ s"$parameterPath&$sessionStatsTableTag.sort=$encodedSortColumn" +
+ s"&$sessionStatsTableTag.desc=$desc#$sessionStatsTableTag"
+
+ override def headers: Seq[Node] = {
+ val sessionTableHeadersAndTooltips: Seq[(String, Boolean, Option[String])] =
+ Seq(
+ ("User", true, None),
+ ("IP", true, None),
+ ("Session ID", true, None),
+ ("Start Time", true, None),
+ ("Finish Time", true, None),
+ ("Duration", true, None),
+ ("Total Statements", true, None))
+
+ headerStatRow(sessionTableHeadersAndTooltips, desc, pageSize, sortColumn,
+ parameterPath, sessionStatsTableTag, sessionStatsTableTag)
+ }
+
+ override def row(session: SessionEvent): Seq[Node] = {
+ val sessionLink = "%s/%s/session/?id=%s".format(
+ UIUtils.prependBaseUri(request, parent.basePath), parent.prefix, session.sessionId)
+ <tr>
+ <td> {session.username} </td>
+ <td> {session.ip} </td>
+ <td> <a href={sessionLink}> {session.sessionId} </a> </td>
+ <td> {formatDate(session.startTime)} </td>
+ <td> {if (session.endTime > 0) formatDate(session.endTime)} </td>
+ <td> {formatDurationVerbose(session.totalTime)} </td>
+ <td> {session.totalOperations} </td>
+ </tr>
+ }
+ }
+
+ /**
+ * Returns parameter of this table.
+ */
+ def getRequestTableParameters(
+ request: HttpServletRequest,
+ tableTag: String,
+ defaultSortColumn: String): (String, Boolean, Int) = {
+ val parameterSortColumn = request.getParameter(s"$tableTag.sort")
+ val parameterSortDesc = request.getParameter(s"$tableTag.desc")
+ val parameterPageSize = request.getParameter(s"$tableTag.pageSize")
+ val sortColumn = Option(parameterSortColumn).map { sortColumn =>
+ UIUtils.decodeURLParameter(sortColumn)
+ }.getOrElse(defaultSortColumn)
+ val desc = Option(parameterSortDesc).map(_.toBoolean).getOrElse(
+ sortColumn == defaultSortColumn
+ )
+ val pageSize = Option(parameterPageSize).map(_.toInt).getOrElse(100)
+
+ (sortColumn, desc, pageSize)
+ }
+
+ /**
+ * Returns parameters of other tables in the page.
+ */
+ def getRequestParameterOtherTable(request: HttpServletRequest, tableTag: String): String = {
+ request.getParameterMap.asScala
+ .filterNot(_._1.startsWith(tableTag))
+ .map(parameter => parameter._1 + "=" + parameter._2(0))
+ .mkString("&")
+ }
+
+ def headerStatRow(
+ headerInfo: Seq[(String, Boolean, Option[String])],
+ desc: Boolean,
+ pageSize: Int,
+ sortColumn: String,
+ parameterPath: String,
+ tableTag: String,
+ headerId: String): Seq[Node] = {
+ val row: Seq[Node] = {
+ headerInfo.map { case (header, sortable, tooltip) =>
+ if (header == sortColumn) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.desc=${!desc}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+ val arrow = if (desc) "▾" else "▴" // UP or DOWN
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header} {Unparsed(arrow)}
+ </span>
+ </a>
+ </th>
+ } else {
+ if (sortable) {
+ val headerLink = Unparsed(
+ parameterPath +
+ s"&$tableTag.sort=${URLEncoder.encode(header, UTF_8.name())}" +
+ s"&$tableTag.pageSize=$pageSize" +
+ s"#$headerId")
+
+ <th>
+ <a href={headerLink}>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </a>
+ </th>
+ } else {
+ <th>
+ <span data-toggle="tooltip" data-placement="top" title={tooltip.getOrElse("")}>
+ {header}
+ </span>
+ </th>
+ }
+ }
+ }
+ }
+ <thead>
+ <tr>{row}</tr>
+ </thead>
+ }
+}
+
+private class SessionStatsTableDataSource(
+ info: Seq[SessionEvent],
+ pageSize: Int,
+ sortColumn: String,
+ desc: Boolean) extends PagedDataSource[SessionEvent](pageSize) {
+
+ // Sorting SessionEvent data
+ private val data = info.sorted(ordering(sortColumn, desc))
+
+ override def dataSize: Int = data.size
+
+ override def sliceData(from: Int, to: Int): Seq[SessionEvent] = data.slice(from, to)
+
+ /**
+ * Return Ordering according to sortColumn and desc.
+ */
+ private def ordering(sortColumn: String, desc: Boolean): Ordering[SessionEvent] = {
+ val ordering: Ordering[SessionEvent] = sortColumn match {
+ case "User" => Ordering.by(_.username)
+ case "IP" => Ordering.by(_.ip)
+ case "Session ID" => Ordering.by(_.sessionId)
+ case "Start Time" => Ordering by (_.startTime)
+ case "Finish Time" => Ordering.by(_.endTime)
+ case "Duration" => Ordering.by(_.totalTime)
Review comment:
ok, let's rename it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org