You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@linkis.apache.org by GitBox <gi...@apache.org> on 2022/01/04 12:51:16 UTC
[GitHub] [incubator-linkis] Zosimer opened a new pull request #1269: Pyspark support CDH and HDP versions
Zosimer opened a new pull request #1269:
URL: https://github.com/apache/incubator-linkis/pull/1269
Because pyspark obtains the spark version number through ‘spark submit -- version’, and converts the version number into an integer. For example, spark-2.4.1 has a version number of 241, but the spark version of CDH is 2.4.1-cdh6 3.2 this type cannot be converted to integer. The spark version of HDP is 2.3.2.3.1.4.0-315. After conversion, it is 2.3.2.3.1.4.0-315, and then an error will be reported。
Therefore, I withdraw the top five digits of the version number, such as 2.4.1-cdh6 3.2, take 2.4.1 as the version number, which is similar to HDP and can meet the requirements。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For additional commands, e-mail: dev-help@linkis.apache.org
[GitHub] [incubator-linkis] peacewong merged pull request #1269: Pyspark support CDH and HDP versions
Posted by GitBox <gi...@apache.org>.
peacewong merged pull request #1269:
URL: https://github.com/apache/incubator-linkis/pull/1269
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For additional commands, e-mail: dev-help@linkis.apache.org