You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by karth295 <gi...@git.apache.org> on 2018/01/22 02:59:21 UTC
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
GitHub user karth295 opened a pull request:
https://github.com/apache/zeppelin/pull/2738
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
### What is this PR for?
Support saving notebooks to Google Cloud Storage, similar to implementations for S3 and Azure. It uses the same authentication mechanisms as the BigQuery interpreter.
I am new to Maven, so please check my work on the pom.xml files. In particular, I upgraded Guava to 23.0, which was required for `google-cloud-java`. Going through hello-world with my changes seems to work.
Also, I modified the BigQuery interpreter docs to point to the **latest** GCS storage docs. Is it more appropriate to pin to the version you are viewing? How can I do that?
### What type of PR is it?
Improvement
### Todos
* [Low priority] Support encryption keys
I don't this is particularly important, at least for v1.
### How should this be tested?
* I added unit tests for the core functionality
* I manually tested the authentication instructions (but that could use a second pair of eyes)
### Questions:
* Does the licenses files need update?
* No idea. `google-cloud-java` is Apache 2: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/LICENSE
* Is there breaking changes for older versions?
* Nope.
* Does this needs documentation?
* Yes, and I tried to update the docs (but there are likely other things that need to be updated)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karth295/zeppelin master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2738.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2738
----
commit 494aa606c15d7d08c7de6a8e4350be6d7384c931
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-16T07:52:12Z
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
----
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2738
The failure should due to jar file corrupted, could you clean travis cache and trigger the failed build again ? (I mean only rerun the failed build, not all the travis build matrix)
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 closed the pull request at:
https://github.com/apache/zeppelin/pull/2738
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2738
CI still fails, could you retrigger it again ? Just close and reopen this PR.
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
The TravisCI failure doesn't look right to me:
```
[WARNING]
Dependency convergence error for com.google.guava:guava:15.0 paths to dependency are:
+-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
+-org.apache.zeppelin:zeppelin-interpreter:0.8.0-20170802.043121-7
+-com.google.guava:guava:15.0
and
+-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
+-com.google.guava:guava:23.0-android
[WARNING] The requested profile "web-ci" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (enforce) on project zeppelin-zengine: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
[ERROR] Failed while enforcing releasability the error(s) are [
[ERROR] Dependency convergence error for com.google.guava:guava:15.0 paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.8.0-20170802.043121-7
[ERROR] +-com.google.guava:guava:15.0
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
[ERROR] +-com.google.guava:guava:23.0-android
```
If you run `mvn clean package`, it should not use the cached `org.apache.zeppelin:zeppelin-interpreter:0.8.0-20170802.043121-7`, and the dependencies should be correct. (Though maven still scares/confuses me)
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/2738
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 closed the pull request at:
https://github.com/apache/zeppelin/pull/2738
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
@zjffdu I noticed that you just bumped the version to 0.9 -- can this feature be integrated into branch-0.8 release as well?
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
GitHub user karth295 reopened a pull request:
https://github.com/apache/zeppelin/pull/2738
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
### What is this PR for?
Support saving notebooks to Google Cloud Storage, similar to implementations for S3 and Azure. It uses the same authentication mechanisms as the BigQuery interpreter.
I am new to Maven, so please check my work on the pom.xml files. In particular, I upgraded Guava to 23.0, which was required for `google-cloud-java`. Going through hello-world with my changes seems to work.
Also, I modified the BigQuery interpreter docs to point to the **latest** GCS storage docs. Is it more appropriate to pin to the version you are viewing? How can I do that?
### What type of PR is it?
Improvement
### Todos
* [Low priority] Support encryption keys
I don't this is particularly important, at least for v1.
### How should this be tested?
* I added unit tests for the core functionality
* I manually tested the authentication instructions (but that could use a second pair of eyes)
### Questions:
* Does the licenses files need update?
* No idea. `google-cloud-java` is Apache 2: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/LICENSE
* Is there breaking changes for older versions?
* Nope.
* Does this needs documentation?
* Yes, and I tried to update the docs (but there are likely other things that need to be updated)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karth295/zeppelin master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2738.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2738
----
commit 8dc819ee4a322d3616f82f6c720eac221fc32ea0
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-29T03:10:40Z
Unify logic to clear notebook runtime state on load from storage
commit 68b1693568e49e15d47a1d597a97cb5ee70805be
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-29T03:21:34Z
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
----
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
GitHub user karth295 reopened a pull request:
https://github.com/apache/zeppelin/pull/2738
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
### What is this PR for?
Support saving notebooks to Google Cloud Storage, similar to implementations for S3 and Azure. It uses the same authentication mechanisms as the BigQuery interpreter.
I am new to Maven, so please check my work on the pom.xml files. In particular, I upgraded Guava to 23.0, which was required for `google-cloud-java`. Going through hello-world with my changes seems to work.
Also, I modified the BigQuery interpreter docs to point to the **latest** GCS storage docs. Is it more appropriate to pin to the version you are viewing? How can I do that?
### What type of PR is it?
Improvement
### Todos
* [Low priority] Support encryption keys
I don't this is particularly important, at least for v1.
### How should this be tested?
* I added unit tests for the core functionality
* I manually tested the authentication instructions (but that could use a second pair of eyes)
### Questions:
* Does the licenses files need update?
* No idea. `google-cloud-java` is Apache 2: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/LICENSE
* Is there breaking changes for older versions?
* Nope.
* Does this needs documentation?
* Yes, and I tried to update the docs (but there are likely other things that need to be updated)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karth295/zeppelin master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2738.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2738
----
commit 72d1f54b33b1a249e094461fd5d2c80bc8b67fed
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-24T07:03:38Z
Move common logic to clear notebook runtime state into NotebookRepo
commit 2b7e9930f8260cea492defc9d1cd404ec94a2d38
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-16T07:52:12Z
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
----
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
These two commits are rebased over master.
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
FYI I finally resolved the dependency issues by using guava 20.0, which is the only version that works for both the `google-cloud-java` libraries as well as Hadoop (the deprecated `Objects` class).
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2738
@karth295 Please try to rebase your PR, I think the build issue you mentioned above has been fixed.
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
Nice, it's green now (though Jenkins didn't update).
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
The new CI failure doesn't seem real to me -- going to close/reopen to make it run again.
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 closed the pull request at:
https://github.com/apache/zeppelin/pull/2738
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2738
@karth295 The only concern is guava, if that doens't affect other function, it is fine to me to merge into 0.8 as well
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2738
Thanks @karth295 will merge if no more comments
---
[GitHub] zeppelin issue #2738: [ZEPPELIN-3182] Support saving notebooks to Google Clo...
Posted by karth295 <gi...@git.apache.org>.
Github user karth295 commented on the issue:
https://github.com/apache/zeppelin/pull/2738
@zjffdu do you know what's causing these failure? (I can't tell whether my PR is breaking the tests)
---
[GitHub] zeppelin pull request #2738: [ZEPPELIN-3182] Support saving notebooks to Goo...
Posted by karth295 <gi...@git.apache.org>.
GitHub user karth295 reopened a pull request:
https://github.com/apache/zeppelin/pull/2738
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
### What is this PR for?
Support saving notebooks to Google Cloud Storage, similar to implementations for S3 and Azure. It uses the same authentication mechanisms as the BigQuery interpreter.
I am new to Maven, so please check my work on the pom.xml files. In particular, I upgraded Guava to 23.0, which was required for `google-cloud-java`. Going through hello-world with my changes seems to work.
Also, I modified the BigQuery interpreter docs to point to the **latest** GCS storage docs. Is it more appropriate to pin to the version you are viewing? How can I do that?
### What type of PR is it?
Improvement
### Todos
* [Low priority] Support encryption keys
I don't this is particularly important, at least for v1.
### How should this be tested?
* I added unit tests for the core functionality
* I manually tested the authentication instructions (but that could use a second pair of eyes)
### Questions:
* Does the licenses files need update?
* No idea. `google-cloud-java` is Apache 2: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/LICENSE
* Is there breaking changes for older versions?
* Nope.
* Does this needs documentation?
* Yes, and I tried to update the docs (but there are likely other things that need to be updated)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karth295/zeppelin master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2738.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2738
----
commit 00d516cd957188cf872b393d86ec7d80d74b4589
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-24T07:03:38Z
Unify logic to clear notebook runtime state on load from storage
commit 71adaed0f3ceb319eb2b0064c58743b53790bc3e
Author: Karthik Palaniappan <ka...@...>
Date: 2018-01-29T01:05:52Z
[ZEPPELIN-3182] Support saving notebooks to Google Cloud Storage
----
---