You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/11/15 07:05:58 UTC
[GitHub] [incubator-kyuubi] cfmcgrady opened a new issue, #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
cfmcgrady opened a new issue, #3805:
URL: https://github.com/apache/incubator-kyuubi/issues/3805
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### What kind of test improve?
- [ ] Fix flaky tests.
- [ ] Fix bug in tests.
- [ ] Increase test coverage.
- [X] Other kinds of test improve.
### Describe the issues of the existing tests or improvements for new tests
https://github.com/apache/incubator-kyuubi/actions/runs/3458458256/jobs/5772898116
```
- pyspark support *** FAILED ***
org.apache.kyuubi.jdbc.hive.KyuubiSQLException: SPARK_HOME not found!
at org.apache.kyuubi.jdbc.hive.Utils.verifySuccess(Utils.java:85)
at org.apache.kyuubi.jdbc.hive.Utils.verifySuccessWithInfo(Utils.java:71)
at org.apache.kyuubi.jdbc.hive.KyuubiStatement.runAsyncOnServer(KyuubiStatement.java:272)
at org.apache.kyuubi.jdbc.hive.KyuubiStatement.executeWithConfOverlay(KyuubiStatement.java:195)
at org.apache.kyuubi.jdbc.hive.KyuubiStatement.execute(KyuubiStatement.java:190)
at org.apache.kyuubi.jdbc.hive.KyuubiStatement.executeQuery(KyuubiStatement.java:397)
at org.apache.kyuubi.engine.spark.operation.PySparkTests.$anonfun$runPySparkTest$1(PySparkTests.scala:66)
at org.apache.kyuubi.engine.spark.operation.PySparkTests.$anonfun$runPySparkTest$1$adapted(PySparkTests.scala:64)
at org.apache.kyuubi.operation.JDBCTestHelper.$anonfun$withMultipleConnectionJdbcStatement$3(JDBCTestHelper.scala:60)
at org.apache.kyuubi.operation.JDBCTestHelper.$anonfun$withMultipleConnectionJdbcStatement$3$adapted(JDBCTestHelper.scala:60)
...
Cause: java.lang.IllegalStateException: SPARK_HOME not found!
at org.apache.kyuubi.engine.spark.operation.ExecutePython$.$anonfun$defaultSparkHome$6(ExecutePython.scala:153)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.spark.operation.ExecutePython$.defaultSparkHome(ExecutePython.scala:153)
at org.apache.kyuubi.engine.spark.operation.ExecutePython$.$anonfun$createSessionPythonWorker$2(ExecutePython.scala:128)
at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:596)
at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:594)
at scala.collection.immutable.HashMap.getOrElse(HashMap.scala:73)
at org.apache.kyuubi.engine.spark.operation.ExecutePython$.createSessionPythonWorker(ExecutePython.scala:128)
at org.apache.kyuubi.engine.spark.operation.SparkSQLOperationManager.$anonfun$newExecuteStatementOperation$7(SparkSQLOperationManager.scala:96)
at scala.collection.concurrent.Map.getOrElseUpdate(Map.scala:97)
...
```
we can disable the PySpark-related tests for `Kyuubi Spark Nightly`
### Are you willing to submit PR?
- [ ] Yes. I can submit a PR independently to improve.
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] cfmcgrady commented on issue #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on issue #3805:
URL: https://github.com/apache/incubator-kyuubi/issues/3805#issuecomment-1315550459
we can refer to what we did in the Data Lake unit test
https://github.com/apache/incubator-kyuubi/blob/6548036453239a42b1428f134843ae8f16d2d9b6/.github/workflows/master.yml#L63
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] pan3793 closed issue #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
Posted by GitBox <gi...@apache.org>.
pan3793 closed issue #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
URL: https://github.com/apache/incubator-kyuubi/issues/3805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] bowenliang123 commented on issue #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
Posted by GitBox <gi...@apache.org>.
bowenliang123 commented on issue #3805:
URL: https://github.com/apache/incubator-kyuubi/issues/3805#issuecomment-1315400154
Not sure how to skip Pyspark tests for nightly builds. Any suggestion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] cfmcgrady commented on issue #3805: [GA] `Kyuubi Spark Nightly` failed with PySpark-related tests.
Posted by GitBox <gi...@apache.org>.
cfmcgrady commented on issue #3805:
URL: https://github.com/apache/incubator-kyuubi/issues/3805#issuecomment-1314867951
cc @bowenliang123 if you are interested in this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org