You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/12/29 01:14:03 UTC
[GitHub] [spark] Shooter23 commented on a change in pull request #35043: Remove characters added by IPython
Shooter23 commented on a change in pull request #35043:
URL: https://github.com/apache/spark/pull/35043#discussion_r776118212
##########
File path: bin/find-spark-home
##########
@@ -37,5 +37,5 @@ else
if [[ -z "$PYSPARK_DRIVER_PYTHON" ]]; then
PYSPARK_DRIVER_PYTHON="${PYSPARK_PYTHON:-"python3"}"
fi
- export SPARK_HOME=$($PYSPARK_DRIVER_PYTHON "$FIND_SPARK_HOME_PYTHON_SCRIPT")
+ export SPARK_HOME=$($PYSPARK_DRIVER_PYTHON "$FIND_SPARK_HOME_PYTHON_SCRIPT" | cut -d $'\007' -f 2)
Review comment:
It appears to be the same problem but with a different solution.
I hadn't tried it with jupyter before so I don't know what I should expect.
Running it with `PYSPARK_DRIVER_PYTHON=jupyter` made it attempt to run `jupyter-/path/to/find_spark_home.py`, which failed with it failing to find /bin/load-spark-env.sh or /bin/spark-submit.
Running it with `PYSPARK_DRIVER_PYTHON='jupyter notebook'` started the kernel but initially opened a 403 forbidden page in the browser, complaining of lacking a referrer. I pasted the URL with the token appearing in the jupyter log and it took me to the bin directory under the virtual environment.
It doesn't appear to me that `$PYSPARK_DRIVER_PYTHON "$FIND_SPARK_HOME_PYTHON_SCRIPT"` should run with PYSPARK_DRIVER_PYTHON set to "jupyter".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org