You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2021/06/29 06:49:43 UTC

[GitHub] [submarine] featherchen opened a new pull request #619: SUBMARINE-861. Trim the size of pysubmarine

featherchen opened a new pull request #619:
URL: https://github.com/apache/submarine/pull/619


   ### What is this PR for?
   <!-- A few sentences describing the overall goals of the pull request's commits.
   First time? Check out the contributing guide - https://submarine.apache.org/contribution/contributions.html
   -->
   Trim the size of pysubmarine, in other words,we let the user install the package such as pytorch and Tensorflow by themself instead of pre-installation from pysubmarine
   ### What type of PR is it?
    Improvement 
   
   ### Todos
    [ ] - Deal with Jupiter Lab
   
   ### What is the Jira issue?
   https://issues.apache.org/jira/browse/SUBMARINE-861
   ### How should this be tested?
   <!--
   * First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
   * Strongly recommended: add automated unit tests for any new or changed behavior
   * Outline any manual steps to test the PR here.
   -->
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Do the license files need updating? No
   * Are there breaking changes for older versions? No
   * Does this need new documentation? No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu edited a comment on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu edited a comment on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870415206






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu edited a comment on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu edited a comment on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870415206


   Thanks for your contribution. 
   I thought that we can rely on `extras_require` to get rid of explicitly installing ML framework.
   `extras_require` means the dependencies for additional features, pip will not install them by default.
   Users can use something like `pip install apache-submarine[feature1, feature2]` to explicitly install the dependencies for features.
   If users preferred to use TensorFlow, they might want to do `pip install apache-submarine[tf]` when installing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870415206






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873335881


   @featherchen There are two workflows in python.yml: unit test and integration test. Only unit test install pytorch and tf beforehand, but integration did not. You can check here: https://github.com/apache/submarine/blob/master/.github/workflows/python.yml#L133.
   I wonder why `pip install --no-cache-dir ./submarine-sdk/pysubmarine/.[tf.pytorch]` cannot solve the problem?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870417135


   @featherchen Yes, you should modify the jupyter dockerfile too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] asfgit closed pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #619:
URL: https://github.com/apache/submarine/pull/619


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873370482


   Thanks for your contribution! LGTM.
   Can you open an issue to update the usage of submarine-sdk?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870296551


   @ByronHsu 
   I'm not sure if I finished all the requirement, and  whether I should deal with the installation of package in Jupiter Lab or not.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873339402


   > @featherchen There are two workflows in python.yml: unit test and integration test. Only unit test install pytorch and tf beforehand, but integration did not. You can check here: https://github.com/apache/submarine/blob/master/.github/workflows/python.yml#L133.
   > I wonder why `pip install --no-cache-dir ./submarine-sdk/pysubmarine/.[tf.pytorch]` cannot solve the problem?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-871648984


   @ByronHsu  I failed on the Integration test,maybe it is the bug you mentioned earlier?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-871246509


   @ByronHsu 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870296551


   @ByronHsu 
   I'm not sure if I finished all the requirement, and  whether I should deal with the installation of package in Jupiter Lab or not.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870415206


   Thanks for your contribution. 
   I thought that we can rely on `extras_require` to get rid of explicitly installing ML framework.
   `extras_require` means the dependencies for additional features, pip will not install them by default.
   Users can use something like `pip install apache-submarine[feature1, feature2]` to explicitly install the dependencies for features.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] ByronHsu edited a comment on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
ByronHsu edited a comment on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-870415206


   Thanks for your contribution. 
   I thought that we can rely on `extras_require` to get rid of explicitly installing ML framework.
   `extras_require` means the dependencies for additional features, pip will not install them by default.
   Users can use something like `pip install apache-submarine[feature1, feature2]` to explicitly install the dependencies for features.
   If users preferred to use TensorFlow, they might want to do `pip install apache-submarine[tf]` when installing.
   For more detail you can check the `setup.py` of mlflow. (https://github.com/mlflow/mlflow/blob/master/setup.py#L89), and the spec of `pip extras`. (https://www.python.org/dev/peps/pep-0508/#extras)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen commented on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen commented on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873143469


   @ByronHsu  I wonder why in integration test it didn't pip install pytorch and Tensorflow,which are be wriiten in ./github/workflow/python.yaml.An alternative solution is that use pip install --no-cache-dir ./submarine-sdk/pysubmarine/.[tf.pytorch] , but it didn't really fix the problem though.
   ![Screenshot from 2021-07-03 01-09-32](https://user-images.githubusercontent.com/57944334/124308457-e565cd00-db9b-11eb-9a32-d0fa59c41943.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen closed pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen closed pull request #619:
URL: https://github.com/apache/submarine/pull/619


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen edited a comment on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen edited a comment on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873143469


   @ByronHsu  I wonder why in integration test it didn't pip install pytorch and Tensorflow,which are  wriiten in ./github/workflow/python.yaml.An alternative solution is that use pip install --no-cache-dir ./submarine-sdk/pysubmarine/.[tf.pytorch] , but it didn't really fix the problem though.
   ![Screenshot from 2021-07-03 01-09-32](https://user-images.githubusercontent.com/57944334/124308457-e565cd00-db9b-11eb-9a32-d0fa59c41943.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [submarine] featherchen edited a comment on pull request #619: SUBMARINE-861. Trim the size of pysubmarine

Posted by GitBox <gi...@apache.org>.
featherchen edited a comment on pull request #619:
URL: https://github.com/apache/submarine/pull/619#issuecomment-873339402


   @ByronHsu  Sorry that my word is not clear enough,I mean that I want to know the real reason of this problem,and after your explanation,I have known it.
   It pass all the tests now.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org