You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by 1ambda <gi...@git.apache.org> on 2016/11/04 13:00:56 UTC
[GitHub] zeppelin pull request #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
GitHub user 1ambda opened a pull request:
https://github.com/apache/zeppelin/pull/1598
[ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
### What is this PR for?
ElasticSearch just released in Oct 26 and It requires 5.x client driver but zeppelin currently use 2.x only.
- we need 2 different profiles for 2.x and 5.x (there are no 3.x, 4.x)
- 5.x client require different codebase. In this case we should keep 2 version of code at the same time like the spark code.
- 5.x client requires JAVA 8 at least
### What type of PR is it?
Improvement
### Todos
* [ ] - create profiles for 2.x and 5.x
* [ ] - setup JAVA 8 validation for 5.x
* [ ] - create the 5.x interpreter
* [ ] - update license
* [ ] - update docs (build, usage)
* [ ] - update default `elasticsearch.version` to 2.4.x (the most recent for 2.x)
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1618
### How should this be tested?
Some tests will be included.
### Screenshots (if appropriate)
<img width="404" alt="screen shot 2016-11-04 at 9 46 23 pm" src="https://cloud.githubusercontent.com/assets/4968473/20006211/ef9d9f96-a2d9-11e6-9b89-efc8f38b7acf.png">
<img width="828" alt="elasticsearch" src="https://cloud.githubusercontent.com/assets/4968473/20006213/f1053236-a2d9-11e6-80ab-e9e4c954bb64.png">
### Questions:
* Does the licenses files need update? - YES
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/1ambda/zeppelin feat/support-es-5.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1598.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1598
----
commit d973cdece3175c7ca16dccf752f51516fb1efe59
Author: 1ambda <1a...@gmail.com>
Date: 2016-11-04T12:55:11Z
chore: Create elastic profile 2, 5
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1598
Great work @1ambda ! Do you know if there are any ETA for Elasticsearch 5.1?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
Ref - https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_50_java_api_changes.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
@stevenmccord Hi :)
I am trying to fix other urgent issue. So it's ok to close this PR if you want to implement.
The reason I am keeping this issue is sharing info about ES 5.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda closed the pull request at:
https://github.com/apache/zeppelin/pull/1598
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
Blocked until 5.1 release due to too many undocumented breaking changes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by stevenmccord <gi...@git.apache.org>.
Github user stevenmccord commented on the issue:
https://github.com/apache/zeppelin/pull/1598
@1ambda were you still working through this one? I was going to give it a test run if this is something that might be getting in. Noticed it hasn't been updated with master in a while, so just wanted to check to see if it might be worthwhile to give it a try. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
@jaspinderdineout Hi :)
Zeppelin doesn't support es 5.0 currently as i know.
I will close this PR. This issue can be easily handled after #1902.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
@bbonnin Thanks for the quick review :)
1. Duplicated code in `executeDeleteQuery` is due to incompatible API between 2.x and 5.x
```
// 2.x
public boolean isFound() { ... }
// 5.x
RestStatus.NOT_FOUND != response.status())
```
But we can extract duplicated validation logic to the base class as you mentioned.
2. @bzz, @bbonnin Regarding to update docs,
Are there more files I should update? I am asking because
- new maven profile is added `elasticsearch-2` (activated by default), `elasticsearch-5`
- `elasticsearch-5` profile requires java 8 +
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1598
\cc @1ambda
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/1598
2+ Solutions.
#### 1. Use just reflection
- Pros: We can have only 1 interpreter `ElasticsearchInterpreter`
- Cons: but too much reflection code :(
#### 2. Create `Elasticsearch5Interpreter` for version 5.x
- Pros: We can remove reflection code by ignoring the interpreter code based on the maven profile
- Cons: Code might be duplicated but we can extract them as an interface. For example, `ElasticsearchInterpeterBase`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by jaspinderdineout <gi...@git.apache.org>.
Github user jaspinderdineout commented on the issue:
https://github.com/apache/zeppelin/pull/1598
Is it live? Does zeppelin support elasricsearch 5.0 ???
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1598: [ZEPPELIN-1618] [WIP] Support ElasticSearch 5.0
Posted by bbonnin <gi...@git.apache.org>.
Github user bbonnin commented on the issue:
https://github.com/apache/zeppelin/pull/1598
I have just one comment: it's about some duplicate lines of codes. I think some lines that we can find in both ElasticSearchXConnectors could be put in the base class, in order to keep only the specific part ES v2 and V5 in the sub-classes. An example is the method `executeDeleteQuery` that is almost the same in both classes.
Otherwise, looks good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---