[jira] [Commented] (BIGTOP-4053) Remove Python as Spark Installation Dependency

["caijialiang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839461#comment-17839461 ] 

caijialiang commented on BIGTOP-4053:
-------------------------------------

[~iwasakims] Simply using `python-unversioned-command` does not seem to be enough. Removing the Python dependency from Spark has led to a situation where Python 3.7.5+ needs to be manually installed on all operating systems in order to pass the related PySpark smoketests. I am puzzled about how to best handle this. How should the dependencies for the PySpark smoketest be installed? If Python 3 is installed as part of the tool chain and its version is kept up-to-date with the updates of Spark 3, it would be quite a chore to maintain. Could you offer any advice?

> Remove Python as Spark Installation Dependency
> ----------------------------------------------
>
>                 Key: BIGTOP-4053
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-4053
>             Project: Bigtop
>          Issue Type: Sub-task
>            Reporter: caijialiang
>            Assignee: caijialiang
>            Priority: Major
>
> Python Version Requirements for PySpark:
> Here are the Python version requirements for PySpark:
>  * For Spark 3.3 to Spark 3.4.2: Python 3.7 and above.
>  * For Spark 3.5: Python 3.8 and above.
> You can find detailed information about these requirements in the [official documentation|https://spark.apache.org/docs/3.3.3/api/python/getting_started/install.html#python-version-supported].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)