You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datalab.apache.org by "Vira Vitanska (Jira)" <ji...@apache.org> on 2021/03/26 12:46:00 UTC
[jira] [Updated] (DATALAB-1365) Migrate to Python3
[ https://issues.apache.org/jira/browse/DATALAB-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vira Vitanska updated DATALAB-1365:
-----------------------------------
Description:
# Retrieve available lib list group -error -ModuleNotFoundError: No module named 'xmlrpclib' -{color:#00875a} fixed 04/02/2021.{color}
# Get rid of Python2 during SSN deployment, Notebook creation.
*SSN:*
{code:java}
Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y install python3-pip python-dev groff vim less git wget nano libssl-dev libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
---> Running in e349ec65c115{code}
(i) {color:#00875a}Fixed{color} in 'tep 4/28 : RUN apt-get update &&' (03/02/2021).
{code:java}
sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build --build-arg OS=debian --build-arg SRC_PATH= --file general/files/gcp/base_Dockerfile -t docker.datalab-base:latest
{code}
(i) {color:#00875a}Fixed{color} in 'sudo: cd /opt/datalab/sources/' (03/02/2021).
{code:java}
sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
(i) {color:#00875a}Fixed{color} in 'apt-get -y install supervisor' (03/02/2021).
(-) Script for Keycloak client creation should use Python3.
*Jupyter:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
*RStudio:*
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Py2 is still available in 'apt-get -y install nodejs' (03/02/2021).
*Zeppelin:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Fixed in 'apt-get -y build-dep python-matplotlib' (03/02/2021).
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Py2 Is still available in 'apt-get -y install nodejs' (03/02/2021).
*3.* Performance for SSN deployment. It has to take up to 40-50 minutes. But now it takes 2 hours.
*4.1.* It's impossible to retrieve available lib list for any group. Docker runs with '1' (26/03/2021):
{code:java}
No idea what 'list_libs' is!
Traceback (most recent call last):
File "/root/entrypoint.py", line 154, in <module>
subprocess.run("/bin/list_libs.py", shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/bin/list_libs.py' returned non-zero exit status 1.
{code}
*4.2.* Available lib list retrieving:
+RStudio:+
||Apt/Yum|Still in process: is too long, > 50 min.|
||Py3|Passed|
||R packages|Docker runs with 1, Error: package or namespace load failed for ‘SparkR’:
package ‘SparkR’ was installed before R 4.0.0: please re-install it|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Still in process: is too long, > 50 min. |
||Py3| Passed|
||R packages| Passed|
||Others| Passed|
*5.* Lib installation:
+RStudio:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
||Py3|Passed|
||R packages|Failed by lib list retrieving|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
||Py3|Passed|
||R packages|Passed|
||Java dependency|Passed|
||Others|Passed|
*6.* Get rid of kernel python2 for notebooks.
was:
# Retrieve available lib list group -error -ModuleNotFoundError: No module named 'xmlrpclib' - fixed 04/02/2021.
# Get rid of Python2 during SSN deployment, Notebook creation.
*SSN:*
{code:java}
Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y install python3-pip python-dev groff vim less git wget nano libssl-dev libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
---> Running in e349ec65c115{code}
(i) Fixed in 'tep 4/28 : RUN apt-get update &&' (03/02/2021).
{code:java}
sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build --build-arg OS=debian --build-arg SRC_PATH= --file general/files/gcp/base_Dockerfile -t docker.datalab-base:latest
{code}
(i) Fixed in 'sudo: cd /opt/datalab/sources/' (03/02/2021).
{code:java}
sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
(i) Fixed in 'apt-get -y install supervisor' (03/02/2021).
(-) Script for Keycloak client creation should use Python3.
*Jupyter:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
*RStudio:*
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Py2 is still available in 'apt-get -y install nodejs' (03/02/2021).
*Zeppelin:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Fixed in 'apt-get -y build-dep python-matplotlib' (03/02/2021).
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
{code}
(i) Py2 Is still available in 'apt-get -y install nodejs' (03/02/2021).
*3.* Performance for SSN deployment. It has to take up to 40-50 minutes. But now it takes 2 hours.
*4.* Available lib list retrieving:
+RStudio:+
||Apt/Yum|Still in process: is too long, > 50 min.|
||Py3|Passed|
||R packages|Docker runs with 1, Error: package or namespace load failed for ‘SparkR’:
package ‘SparkR’ was installed before R 4.0.0: please re-install it|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Still in process: is too long, > 50 min. |
||Py3| Passed|
||R packages| Passed|
||Others| Passed|
*5.* Lib installation:
+RStudio:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
||Py3|Passed|
||R packages|Failed by lib list retrieving|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
||Py3|Passed|
||R packages|Passed|
||Java dependency|Passed|
||Others|Passed|
*6.* Get rid of kernel python2 for notebooks.
> Migrate to Python3
> ------------------
>
> Key: DATALAB-1365
> URL: https://issues.apache.org/jira/browse/DATALAB-1365
> Project: Apache DataLab
> Issue Type: Task
> Components: DataLab Main
> Reporter: Vira Vitanska
> Assignee: Leonid Frolov
> Priority: Major
> Labels: AWS, AZURE, Debian, DevOps, GCP, RedHat
> Fix For: v.2.5
>
> Attachments: Jupyter creation1.png, Jupyter creation2.png, P4-Rstudio-r-package-lib-list.png, RStudio creation.png, Zeppelin creation1.png, Zeppelin creation2.png
>
>
> # Retrieve available lib list group -error -ModuleNotFoundError: No module named 'xmlrpclib' -{color:#00875a} fixed 04/02/2021.{color}
> # Get rid of Python2 during SSN deployment, Notebook creation.
> *SSN:*
> {code:java}
> Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y install python3-pip python-dev groff vim less git wget nano libssl-dev libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
> ---> Running in e349ec65c115{code}
> (i) {color:#00875a}Fixed{color} in 'tep 4/28 : RUN apt-get update &&' (03/02/2021).
> {code:java}
> sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build --build-arg OS=debian --build-arg SRC_PATH= --file general/files/gcp/base_Dockerfile -t docker.datalab-base:latest
> {code}
> (i) {color:#00875a}Fixed{color} in 'sudo: cd /opt/datalab/sources/' (03/02/2021).
> {code:java}
> sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
> (i) {color:#00875a}Fixed{color} in 'apt-get -y install supervisor' (03/02/2021).
> (-) Script for Keycloak client creation should use Python3.
> *Jupyter:*
> {code:java}
> sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
> {code}
> {code:java}
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi{code}
> *RStudio:*
> {code:java}
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
> {code}
> (i) Py2 is still available in 'apt-get -y install nodejs' (03/02/2021).
> *Zeppelin:*
> {code:java}
> sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
> {code}
> (i) Fixed in 'apt-get -y build-dep python-matplotlib' (03/02/2021).
> {code:java}
>
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" > /tmp/apt.log;fi
> {code}
> (i) Py2 Is still available in 'apt-get -y install nodejs' (03/02/2021).
> *3.* Performance for SSN deployment. It has to take up to 40-50 minutes. But now it takes 2 hours.
> *4.1.* It's impossible to retrieve available lib list for any group. Docker runs with '1' (26/03/2021):
> {code:java}
> No idea what 'list_libs' is!
> Traceback (most recent call last):
> File "/root/entrypoint.py", line 154, in <module>
> subprocess.run("/bin/list_libs.py", shell=True, check=True)
> File "/usr/lib/python3.8/subprocess.py", line 512, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command '/bin/list_libs.py' returned non-zero exit status 1.
> {code}
> *4.2.* Available lib list retrieving:
> +RStudio:+
> ||Apt/Yum|Still in process: is too long, > 50 min.|
> ||Py3|Passed|
> ||R packages|Docker runs with 1, Error: package or namespace load failed for ‘SparkR’:
> package ‘SparkR’ was installed before R 4.0.0: please re-install it|
> ||Others|Passed|
> +Zeppelin:+
> ||Apt/Yum|Still in process: is too long, > 50 min. |
> ||Py3| Passed|
> ||R packages| Passed|
> ||Others| Passed|
> *5.* Lib installation:
> +RStudio:+
> ||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
> ||Py3|Passed|
> ||R packages|Failed by lib list retrieving|
> ||Others|Passed|
> +Zeppelin:+
> ||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be installed|
> ||Py3|Passed|
> ||R packages|Passed|
> ||Java dependency|Passed|
> ||Others|Passed|
> *6.* Get rid of kernel python2 for notebooks.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datalab.apache.org
For additional commands, e-mail: dev-help@datalab.apache.org