You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by astroshim <gi...@git.apache.org> on 2016/06/16 15:15:18 UTC
[GitHub] zeppelin pull request #1030: [ZEPPELIN-1023] Add more credential apis.
GitHub user astroshim opened a pull request:
https://github.com/apache/zeppelin/pull/1030
[ZEPPELIN-1023] Add more credential apis.
### What is this PR for?
This PR is for supporting various Credential APIs for users.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1023
### How should this be tested?
- to create credential information.
```
curl -XPUT -H "Content-Type: application/json" "http://localhost:8080/api/credential" -d '{"entity" : "e1", "username" : "user1", "password" : "testpass"}'
```
- to get credential information.
```
curl -XGET -H "Content-Type: application/json" "http://localhost:8080/api/credential"
```
- to remove credential entity information.
```
curl -vvv -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential/e1"
```
- to remove all credential information.
```
curl -vvv -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential"
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1023
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1030.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1030
----
commit 03717018b88d4a4d02f0ce153bb006c5474f7dd7
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-16T15:05:42Z
add more credential apis.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
Thank you for your review @bzz and @AhyoungRyu and @cloverhearts !
And It really makes sense @bzz comments.
Let me fix them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@bzz @AhyoungRyu Could you review?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1030
PR template does have the JIRA link ask? I thought not every contributor in JIRA can change "assign" or "fix version" and it's up to the admin resolving the JIRA to set them?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1030
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1030
There are multiple CI failures
- JSON parsing failure somewhere in websocket communication
```
Running org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.protocol.ZeppelinhubMessageTest
22:05:10,932 ERROR org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.Client:79 - Cannot deserialize zeppelinhub message
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 12
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
at com.google.gson.Gson.fromJson(Gson.java:791)
at com.google.gson.Gson.fromJson(Gson.java:757)
at com.google.gson.Gson.fromJson(Gson.java:706)
at com.google.gson.Gson.fromJson(Gson.java:678)
at org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.protocol.ZeppelinhubMessage.deserialize(ZeppelinhubMessage.java:77)
at org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.protocol.ZeppelinhubMessageTest.testThatInvalidStringReturnEmptyZeppelinhubMessage(ZeppelinhubMessageTest.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 12
at com.google.gson.stream.JsonReader.expect(JsonReader.java:339)
at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:322)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:165)
... 29 more
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.protocol.ZeppelinhubMessageTest
```
- with something, related to `ConcurrentHashMap`
```
2:05:16,571 ERROR org.apache.zeppelin.scheduler.Job:182 - Job failed
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
at org.apache.zeppelin.user.Credentials.getUserCredentials(Credentials.java:56)
at org.apache.zeppelin.notebook.Paragraph.getInterpreterContext(Paragraph.java:354)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:277)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
22:05:16,571 ERROR org.apache.zeppelin.scheduler.JobProgressPoller:54 - Can not get or update progress
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
at org.apache.zeppelin.user.Credentials.getUserCredentials(Credentials.java:56)
at org.apache.zeppelin.notebook.Paragraph.getInterpreterContext(Paragraph.java:354)
at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgressPoller.java:51)
22:05:16,574 INFO org.apache.zeppelin.notebook.Paragraph:252 - run paragraph 20160624-220515_1683752381 using null
```
Have never seen any of those before, @astroshim could you take a look please and make sure it's not related to the changes?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by neil4dong <gi...@git.apache.org>.
Github user neil4dong commented on the issue:
https://github.com/apache/zeppelin/pull/1030
I checkout the branch "branch-0.6" , but it seems that this bug fix has not merged to this branch.
This problem makes cron jobschedule unusable. Really appreciate for this bug fix.
Should I build from source at the master branch or wait for the release 0.7.* ? Thanks .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1030
Merging if there is no further discussion
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim closed the pull request at:
https://github.com/apache/zeppelin/pull/1030
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@bzz okay I will. Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
GitHub user astroshim reopened a pull request:
https://github.com/apache/zeppelin/pull/1030
[ZEPPELIN-1023] Add more credential apis.
### What is this PR for?
This PR is for supporting various Credential APIs for users.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1023
### How should this be tested?
- to create credential information.
```
curl -XPUT -H "Content-Type: application/json" "http://localhost:8080/api/credential" -d '{"entity" : "e1", "username" : "user1", "password" : "testpass"}'
```
- to get credential information.
```
curl -XGET -H "Content-Type: application/json" "http://localhost:8080/api/credential"
```
- to remove credential entity information.
```
curl -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential/e1"
```
- to remove all credential information.
```
curl -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential"
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1023
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1030.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1030
----
commit 03717018b88d4a4d02f0ce153bb006c5474f7dd7
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-16T15:05:42Z
add more credential apis.
commit 9b2c1c98db582b88e93d66d7ccbe3427b4add607
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-18T13:40:15Z
add checking null and blank values in the putCredentials method.
commit 26433f2945104e17cdbc70de13bfa3f0ca5395e8
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-23T05:34:01Z
change using syncronizedMap.
commit f54653282b26cb087ce52e1942ed513999d35c8a
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-24T15:23:41Z
add rest-credential document
commit 80d8a30ee2912b4952a305b88d9a12617a3bd325
Author: AhyoungRyu <fb...@hanmail.net>
Date: 2016-06-24T16:18:08Z
Add rest-credential.md to index.md & dropdown menu
commit 7b2a7c5709dd31ecfb3180f0d8a236325d5e8b4f
Author: HyungSung <hs...@nflabs.com>
Date: 2016-06-24T16:47:35Z
Merge pull request #7 from AhyoungRyu/ZEPPELIN-1023-docs
[ZEPPELIN-1023] Add rest-credential.md to index.md & dropdown menu
commit d0c2bac7a42c3a0b4d61a1e0dd2901870925a1c0
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-24T21:30:10Z
changed syncronizedmap to concurrenthashmap
commit 0f503fdf4c46b1b3de8f4a42d982830b3ccca3a7
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-27T11:42:34Z
to fix ci
commit 06099313cda64de3850652b4b5d5d4b1d1647d02
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-27T13:08:05Z
update credentialsMap
commit a58047624b0c1171a3586edc8eeb00e7f12985ff
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-27T13:38:43Z
erase comment code
commit 1b94ae12b00cba565f879cc44c76758432b66331
Author: astroshim <hs...@nflabs.com>
Date: 2016-06-27T13:38:47Z
Merge branch 'master' into ZEPPELIN-1023
rebase
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by corneadoug <gi...@git.apache.org>.
Github user corneadoug commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@bzz How about editing our current PR template to state that JIRA issue is mandatory and that you should set both assigned and Fix Version? We may still have to repeat it, but at least it would be written somewhere people see at each PR :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@bzz I fixed code and the build problem that you mentioned is gone.
but another following build fail occurred.
I think following problem seems not related this PR, what do you think?
```
14:10:13,190 ERROR org.apache.zeppelin.scheduler.RemoteScheduler:280 - Unknown status
java.util.ConcurrentModificationException
at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:953)
at java.util.LinkedList$ListItr.next(LinkedList.java:886)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:95)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
at com.google.gson.Gson.toJson(Gson.java:582)
at com.google.gson.Gson.toJson(Gson.java:561)
at com.google.gson.Gson.toJson(Gson.java:516)
at com.google.gson.Gson.toJson(Gson.java:496)
at org.apache.zeppelin.notebook.repo.VFSNotebookRepo.save(VFSNotebookRepo.java:212)
at org.apache.zeppelin.notebook.repo.NotebookRepoSync.save(NotebookRepoSync.java:145)
at org.apache.zeppelin.notebook.Note.persist(Note.java:543)
at org.apache.zeppelin.socket.NotebookServer$ParagraphListenerImpl.afterStatusChange(NotebookServer.java:1195)
at org.apache.zeppelin.scheduler.Job.setStatus(Job.java:150)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.afterStatusChange(RemoteScheduler.java:385)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:267)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:341)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
14:10:13,191 INFO org.apache.zeppelin.socket.NotebookServer:1192 - Job 20160627-141012_944285838 is finished
14:10:13,191 INFO org.apache.zeppelin.notebook.Paragraph:252 - run paragraph 20160627-141013_1822933698 using r org.apache.zeppelin.interpreter.LazyOpenInterpreter@38605c72
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1030
\U0001f44d for having tests!
Two things:
#### 1. Code
Instead of locking on the map (`synchronized(credentialsMap){...}`), would it make sense to rather replace `credentialsMap` with concurrent map implementation?
The client code gets simpler and implementation usually is more efficient than explicit locking.
#### 2. Docs
In PR description you also have
>Does this needs documentation? no
But do not you think it also shall be documented somewhere under http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/rest-api/ ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by ChrisAdvance <gi...@git.apache.org>.
Github user ChrisAdvance commented on the issue:
https://github.com/apache/zeppelin/pull/1030
Does the "Connection refused" problem fixed yet?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by AhyoungRyu <gi...@git.apache.org>.
Github user AhyoungRyu commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@neil4dong This PR is not for bug fix but for adding new credential Apis. Could you elaborate more about your problem? You can also create new [Jira](https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN) ticket if you want :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by cloverhearts <gi...@git.apache.org>.
Github user cloverhearts commented on the issue:
https://github.com/apache/zeppelin/pull/1030
LGTM +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
The CI fail has gone.. I just did trigger CI again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1030
Looks great to me!
Let's create a JIRA issue for current failure with label `flaky-tests` and merge this!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by neil4dong <gi...@git.apache.org>.
Github user neil4dong commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@AhyoungRyu It looks like I am comment on a wrong place.
But I am still have some problem about "spark interpreter group", When multiple notebook use timed-scheduling sharing one spark interpreter instance , because of the interpreter use REPL to process the notebook ,and REPL only allow one task in the same time .
So the code looks like this :
```
public InterpreterResult interpret(String[] lines, InterpreterContext context) {
synchronized (this) {
z.setGui(context.getGui());
sc.setJobGroup(getJobGroup(context), "Zeppelin", false);
InterpreterResult r = interpretInput(lines, context);
sc.clearJobGroup();
return r;
}
}
```
Then the blocked processing will effect each other , eventually leading the interpreter jvm died in somehow without any suspicious output in the interpreter log.
Meanwhile the zeppelin server is constantly checking the status of the interpreter by calling `RemoteInterpreterService#getStatus().` Because the interpreter is already died , the zeppelin server always found :
```
ERROR [2016-12-26 15:39:00,715] ({pool-1-thread-84} RemoteScheduler.java[getStatus]:255) - Can't get status information
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:253)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:341)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 15 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 16 more
ERROR [2016-12-26 15:39:00,715] ({pool-1-thread-84} NotebookServer.java[afterStatusChange]:1145) - Error
org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:250)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:279)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:248)
... 11 more
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 18 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 19 more
```
**Should we need to find out what lead the interpreter died?**
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@bzz I'll do that and It's great idea @corneadoug !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by AhyoungRyu <gi...@git.apache.org>.
Github user AhyoungRyu commented on the issue:
https://github.com/apache/zeppelin/pull/1030
For the docs, yeah I agree with @bzz. Maybe need to create a new page for this `http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/rest-api/rest-credential` :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1030: [ZEPPELIN-1023] Add more credential apis.
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1030
@astroshim Merged, next time please check JIRA issue - it should be assigned to somebody (you, in this case) and has a "Fix Version" set.
This simplified life of the release manager a lot!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---