You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2022/09/01 02:46:28 UTC
[GitHub] [submarine] huang06 opened a new pull request, #994: SUBMARINE-1320. Drop supports for Python2 and Python 3.6
huang06 opened a new pull request, #994:
URL: https://github.com/apache/submarine/pull/994
### What is this PR for?
<!-- A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://submarine.apache.org/contribution/contributions.html
-->
### What type of PR is it?
Improvement, Documentation
### Todos
* [ ] - Task
### What is the Jira issue?
<https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-1320>
### How should this be tested?
<!--
* First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
-->
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? Yes/No
* Are there breaking changes for older versions? Yes/No
* Does this need new documentation? Yes/No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971413193
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
The protobuf for Tensorflow 2.9.2 is limited to 3.20.
<https://github.com/tensorflow/tensorflow/blob/v2.9.2/tensorflow/tools/pip_package/setup.py#L97>
I will later list the protobuf version constraints for each Tensorflow2 version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r970933466
##########
website/versioned_docs/version-0.7.0/userDocs/submarine-sdk/pysubmarine/development.md:
##########
@@ -24,7 +24,7 @@ This is required for developing & testing changes, we recommend installing pysub
in its own conda environment by running the following
```bash
-conda create --name submarine-dev python=3.6
+conda create --name submarine-dev python=3.7
Review Comment:
Thank you for the feedback.
I didn't notice that this document is under the version-0.7.0 directory!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971477537
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
This [setup.py](https://github.com/tensorflow/tensorflow/blob/v2.9.2/tensorflow/tools/pip_package/setup.py#L90-L97) shows that protobuf 3.20 encounters errors on Windows.
Currently, all the TF2 patch versions are set to `protobuf >= 3.9.2, < 3.20`.
```python
# TODO(b/182876485): Protobuf 3.20 results in linker errors on Windows
# Protobuf 4.0 is binary incompatible with what C++ TF uses.
# We need ~1 quarter to update properly.
# See also: https://github.com/tensorflow/tensorflow/issues/53234
# See also: https://github.com/protocolbuffers/protobuf/issues/9954
# See also: https://github.com/tensorflow/tensorflow/issues/56077
# This is a temporary patch for now, to patch previous TF releases.
'protobuf >= 3.9.2, < 3.20',
```
This [table](https://gist.github.com/huang06/811270ba210f1a0f858e7f1d9138d6fc#file-ml-package-deps-md) lists detailed dependencies for each version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1235073183
> The Tensorflow 1.15 is not available for Python 3.8 or higher.
>
> [tensorflow/tensorflow#39768](https://github.com/tensorflow/tensorflow/issues/39768)
>
> If we want to adapt higher Python versions, We have to drop support for Tensorflow 1.15
My personal thoughts at the moment are to keep at least 2-3 python versions available.
But it looks like it would be a relatively big change to upgrade the python version. This could probably be discussed at next week's [community meeting](https://cloudera.zoom.us/j/97264903288), or if you have time next Tuesday you could join us.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
pingsutw commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971517261
##########
.github/workflows/python.yml:
##########
@@ -40,33 +42,73 @@ jobs:
timeout-minutes: 30
strategy:
matrix:
- python-version: [3.6, 3.7]
- tf-version: [1.15.0, 2.6.0]
+ python-version: ['3.7', '3.8', '3.9', '3.10']
+ tf-version: ['tf15', 'tf26', 'tf27', 'tf28', 'tf29']
+ exclude:
+ - python-version: '3.8'
+ tf-version: 'tf15'
+ - python-version: '3.9'
+ tf-version: 'tf15'
+ - python-version: '3.10'
+ tf-version: 'tf15'
+ - python-version: '3.10'
+ tf-version: 'tf26'
+ - python-version: '3.10'
+ tf-version: 'tf27'
fail-fast: false
steps:
- - uses: actions/checkout@v2
+ - uses: actions/checkout@v3
- name: Setup python environment
- uses: actions/setup-python@v2
+ uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install requirements
run: |
- pip install --upgrade pip
+ pip install --upgrade pip setuptools wheel
pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
- - name: Install pysubmarine with tf1 and pytorch
- if: matrix.tf-version == '1.15.0'
+ - name: Install pysubmarine with tf1.5 and pytorch
+ if: ${{ matrix.tf-version == 'tf15' }}
run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf,pytorch]
- - name: Install pysubmarine with tf2 and pytorch
- if: matrix.tf-version == '2.6.0'
- run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.6 and pytorch
+ if: ${{ matrix.tf-version == 'tf26' }}
+ run: pip install --no-cache-dir tensorflow~=2.6 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.7 and pytorch
+ if: ${{ matrix.tf-version == 'tf27' }}
+ run: pip install --no-cache-dir tensorflow~=2.7 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.8 and pytorch
+ if: ${{ matrix.tf-version == 'tf28' }}
+ run: pip install --no-cache-dir tensorflow~=2.8 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.9 and pytorch
+ if: ${{ matrix.tf-version == 'tf29' }}
Review Comment:
we can change `tf26` to `2.6`, and update the task to something like
```
- name: Install pysubmarine with tf2.6 and pytorch
run: pip install --no-cache-dir tensorflow~=${{ matrix.tf-version }} && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
```
same for the tf28, tf29
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1234951542
The Tensorflow 1.15 is not available for Python 3.8 or higher.
<https://github.com/tensorflow/tensorflow/issues/39768>
If we want to adapt higher Python versions, We have to drop support for Tensorflow 1.15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 closed pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 closed pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
URL: https://github.com/apache/submarine/pull/994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r970780159
##########
website/versioned_docs/version-0.7.0/userDocs/submarine-sdk/pysubmarine/development.md:
##########
@@ -24,7 +24,7 @@ This is required for developing & testing changes, we recommend installing pysub
in its own conda environment by running the following
```bash
-conda create --name submarine-dev python=3.6
+conda create --name submarine-dev python=3.7
Review Comment:
I think the python codes with existing version documentation (0.6/0.7) can be left unchanged. Our focus can be more on master (0.8 or future).
##########
.github/workflows/python.yml:
##########
@@ -40,33 +42,73 @@ jobs:
timeout-minutes: 30
strategy:
matrix:
- python-version: [3.6, 3.7]
- tf-version: [1.15.0, 2.6.0]
+ python-version: ['3.7', '3.8', '3.9', '3.10']
+ tf-version: ['tf15', 'tf26', 'tf27', 'tf28', 'tf29']
+ exclude:
+ - python-version: '3.8'
+ tf-version: 'tf15'
+ - python-version: '3.9'
+ tf-version: 'tf15'
+ - python-version: '3.10'
+ tf-version: 'tf15'
+ - python-version: '3.10'
+ tf-version: 'tf26'
+ - python-version: '3.10'
+ tf-version: 'tf27'
fail-fast: false
steps:
- - uses: actions/checkout@v2
+ - uses: actions/checkout@v3
- name: Setup python environment
- uses: actions/setup-python@v2
+ uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install requirements
run: |
- pip install --upgrade pip
+ pip install --upgrade pip setuptools wheel
pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
- - name: Install pysubmarine with tf1 and pytorch
- if: matrix.tf-version == '1.15.0'
+ - name: Install pysubmarine with tf1.5 and pytorch
+ if: ${{ matrix.tf-version == 'tf15' }}
run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf,pytorch]
- - name: Install pysubmarine with tf2 and pytorch
- if: matrix.tf-version == '2.6.0'
- run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.6 and pytorch
+ if: ${{ matrix.tf-version == 'tf26' }}
+ run: pip install --no-cache-dir tensorflow==2.6.5 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.7 and pytorch
+ if: ${{ matrix.tf-version == 'tf27' }}
+ run: pip install --no-cache-dir tensorflow==2.7.3 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.8 and pytorch
+ if: ${{ matrix.tf-version == 'tf28' }}
+ run: pip install --no-cache-dir tensorflow==2.8.2 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
+ - name: Install pysubmarine with tf2.9 and pytorch
+ if: ${{ matrix.tf-version == 'tf29' }}
+ run: pip install --no-cache-dir tensorflow==2.9.2 && pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
Review Comment:
The version of tensorflow may also be updated, and I was wondering if it could be defined in the following way so that pip would be smarter about downloading the latest packages to keep the version up-to-date/reliable.
```shell
# if I want to install latest 2.9.x
pip install tensorflow<2.10.0
```
##########
dev-support/style-check/python/lint-requirements.txt:
##########
@@ -13,7 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-click==8.0.4 # For black to work properly. See SUBMARINE-1249.
-black[jupyter]==21.8b0
-flake8==3.9.2
-isort==5.9.3
+black[jupyter]
Review Comment:
Can we still define a new dependency versions in order to avoid the problem of version conflicts that will arise again in the future?
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<3.20"],
Review Comment:
3.20.1 should be the final supported version.
https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971477537
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
This [setup.py](https://github.com/tensorflow/tensorflow/blob/v2.9.2/tensorflow/tools/pip_package/setup.py#L90-L97) shows that protobuf 3.20 encounters errors on Windows.
Currently, all the TF2 patch versions use `protobuf >= 3.9.2, < 3.20`.
```python
# TODO(b/182876485): Protobuf 3.20 results in linker errors on Windows
# Protobuf 4.0 is binary incompatible with what C++ TF uses.
# We need ~1 quarter to update properly.
# See also: https://github.com/tensorflow/tensorflow/issues/53234
# See also: https://github.com/protocolbuffers/protobuf/issues/9954
# See also: https://github.com/tensorflow/tensorflow/issues/56077
# This is a temporary patch for now, to patch previous TF releases.
'protobuf >= 3.9.2, < 3.20',
```
This [table](https://gist.github.com/huang06/811270ba210f1a0f858e7f1d9138d6fc#file-ml-package-deps-md) lists detailed dependencies for each version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971398834
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
`protobuf` in tf2 should also be limited to 3.20.1 ?
I don't know what the latest versions of all these dependencies are yet, and it may take me some time to confirm whether there is some potential impact of the interdependencies between these versions. Or if you have time on your side could you run a test locally?
What do you think? @pingsutw @KUAN-HSUN-LI
##########
website/versioned_docs/version-0.6.0/userDocs/yarn/docker/pytorch/with-cifar10-models/cifar10_tutorial.py:
##########
@@ -71,7 +71,12 @@
Using ``torchvision``, it’s extremely easy to load CIFAR10.
"""
+import matplotlib.pyplot as plt
Review Comment:
These codes under 0.6.0 documents `website/versioned_docs/version-0.6.0/**` should also be rolled back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971413193
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
I will later list the protobuf version constraints for each Tensorflow2 version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971477537
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
This [setup.py](https://github.com/tensorflow/tensorflow/blob/v2.9.2/tensorflow/tools/pip_package/setup.py#L90-L97) shows that protobuf 3.20 encounters errors on Windows.
Currently, all the TF2 patches use `protobuf >= 3.9.2, < 3.20`.
```python
# TODO(b/182876485): Protobuf 3.20 results in linker errors on Windows
# Protobuf 4.0 is binary incompatible with what C++ TF uses.
# We need ~1 quarter to update properly.
# See also: https://github.com/tensorflow/tensorflow/issues/53234
# See also: https://github.com/protocolbuffers/protobuf/issues/9954
# See also: https://github.com/tensorflow/tensorflow/issues/56077
# This is a temporary patch for now, to patch previous TF releases.
'protobuf >= 3.9.2, < 3.20',
```
This [table](https://gist.github.com/huang06/811270ba210f1a0f858e7f1d9138d6fc#file-ml-package-deps-md) lists detailed dependencies for each version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r970933466
##########
website/versioned_docs/version-0.7.0/userDocs/submarine-sdk/pysubmarine/development.md:
##########
@@ -24,7 +24,7 @@ This is required for developing & testing changes, we recommend installing pysub
in its own conda environment by running the following
```bash
-conda create --name submarine-dev python=3.6
+conda create --name submarine-dev python=3.7
Review Comment:
Thank you for the feedback.
I didn't notice that this document is under the v0.0.7 directory!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1250484449
@cdmikechen @pingsutw
Please review the changes and approve the workflows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971398834
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
`protobuf` in tf2 should also be limited to 3.20.1 ?
I don't know what the latest versions of all these dependencies are yet, and it may take me some time to confirm whether there is some potential impact of the interdependencies between these versions. Or if you have time on your side could you run a test locally?
What are your opinions? @pingsutw @KUAN-HSUN-LI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1246082608
@huang06
Hi~ I've checked the python checks failed in submarine-sdk/pysubmarine/tests/tracking/test_tracking.py. And you need to change the parameters of `save_model`, the current ones are indeed wrong.
```python
@pytest.mark.skipif(tensorflow.version.VERSION < "2.0", reason="using tensorflow 2")
def test_save_model(self):
input_arr = tensorflow.random.uniform((1, 5))
model = LinearNNModel()
model(input_arr)
self.client.save_model(model, "tensorflow", REGISTERED_MODEL_NAME)
self.client.save_model(model, "tensorflow", REGISTERED_MODEL_NAME)
# Validate model_versions
model_versions = self.model_registry.list_model_versions(REGISTERED_MODEL_NAME)
assert len(model_versions) == 2
assert model_versions[0].name == REGISTERED_MODEL_NAME
assert model_versions[0].version == 1
assert model_versions[1].name == REGISTERED_MODEL_NAME
assert model_versions[1].version == 2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] huang06 commented on a diff in pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
huang06 commented on code in PR #994:
URL: https://github.com/apache/submarine/pull/994#discussion_r971581639
##########
submarine-sdk/pysubmarine/setup.py:
##########
@@ -28,41 +28,41 @@
packages=find_packages(exclude=["tests", "tests.*"]),
package_data={"submarine.cli.config": ["cli_config.yaml"]},
install_requires=[
- "six>=1.10.0",
- "numpy==1.19.2",
+ "numpy",
"pandas",
"sqlalchemy>=1.4.0",
"sqlparse",
"pymysql",
- "requests==2.26.0",
+ "requests",
"urllib3>=1.15.1",
"certifi>=14.05.14",
"python-dateutil>=2.5.3",
"pyarrow>=6.0.1",
"boto3>=1.17.58",
- "click==8.0.3",
- "rich==10.15.2",
- "dacite==1.6.0",
- "dataclasses>=0.6",
- "pyaml==21.10.1",
+ "click>=8.1.0",
+ "rich",
+ "dacite",
+ "pyaml",
],
extras_require={
- "tf": ["tensorflow==1.15.0", "protobuf<=3.20.1"],
+ "tf": ["tensorflow==1.15.5", "numpy>=1.16.0,<1.19.0", "protobuf>=3.6.1,<=3.20.1"],
"tf2": [
- "tensorflow==2.6.0",
+ "tensorflow>=2.6.0,<2.10.0",
+ "numpy>=1.14.5",
+ "keras>=2.6.0",
+ "protobuf>=3.9.2,<3.20",
Review Comment:
Should we remain `protobuf>=3.9.2,<3.20` or relax the version constraint (e.g. `protobuf>=3.9.2`) for future Tensorflow patches?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] codecov[bot] commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1234445981
# [Codecov](https://codecov.io/gh/apache/submarine/pull/994?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#994](https://codecov.io/gh/apache/submarine/pull/994?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8a8e1d1) into [master](https://codecov.io/gh/apache/submarine/commit/806fcb2181e5cef5f40f4e57aa941fae53ee5217?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (806fcb2) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #994 +/- ##
============================================
- Coverage 14.27% 14.26% -0.01%
+ Complexity 992 991 -1
============================================
Files 241 241
Lines 23897 23897
Branches 3473 3473
============================================
- Hits 3412 3410 -2
- Misses 20279 20280 +1
- Partials 206 207 +1
```
| [Impacted Files](https://codecov.io/gh/apache/submarine/pull/994?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../submarine/server/websocket/ConnectionManager.java](https://codecov.io/gh/apache/submarine/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VibWFyaW5lLXNlcnZlci9zZXJ2ZXItY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3VibWFyaW5lL3NlcnZlci93ZWJzb2NrZXQvQ29ubmVjdGlvbk1hbmFnZXIuamF2YQ==) | `53.65% <0.00%> (-4.88%)` | :arrow_down: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] cdmikechen commented on pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
cdmikechen commented on PR #994:
URL: https://github.com/apache/submarine/pull/994#issuecomment-1234945895
Hi @huang06 Thanks for your contribution~
The current PR has a lot of syntax changes and may take longer to review. My personal preference is that we can adapt the 3.7, 3.8 and 3.9 versions in this PR.
What do you think? @pingsutw
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw merged pull request #994: SUBMARINE-1320. Drop supports for Python2.7 and Python3.6
Posted by GitBox <gi...@apache.org>.
pingsutw merged PR #994:
URL: https://github.com/apache/submarine/pull/994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org