[jira] [Work logged] (GRIFFIN-322) Add SQL mode for ES connector

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GRIFFIN-322?focusedWorklogId=374548&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-374548 ]

ASF GitHub Bot logged work on GRIFFIN-322:
------------------------------------------

                Author: ASF GitHub Bot
            Created on: 20/Jan/20 13:29
            Start Date: 20/Jan/20 13:29
    Worklog Time Spent: 10m 
      Work Description: XiaoyuBD commented on pull request #567: [GRIFFIN-322] Add SQL mode for ES connector
URL: https://github.com/apache/griffin/pull/567
 
 
   As  [GRIFFIN-322](https://issues.apache.org/jira/projects/GRIFFIN/issues/GRIFFIN-322?filter=allopenissues) , we want add sql mode for es connector.   
   
   **The sql mode would more effective and user-friendly.** 
   
   Current mode config:
   {   "class": "org.apache.griffin.measure.datasource.connector.batch.ElasticSearchGriffinDataConnector",
       "index": "index-xxx",
       "type": "metric",
       "host": "xxxxxxxxxx",
       "port": "xxxx",
       "fields": ["col_a", "col_b", "col_c"],
       "size": 100}
   
   SQL mode config: 
   {    "class": "org.apache.griffin.measure.datasource.connector.batch.ElasticSearchGriffinDataConnector",
        "host": "xxxxxxxxxx",
        "port": "xxxx",
        "sql": "select col_a, col_b, col_c from index-xx limit 100"}
   
   Compared with current mode, SQL mode could support other types except number type.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 374548)
    Remaining Estimate: 0h
            Time Spent: 10m

> Add SQL mode for ES connector
> -----------------------------
>
>                 Key: GRIFFIN-322
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-322
>             Project: Griffin
>          Issue Type: Improvement
>            Reporter: XIAOYU YU
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The ElasticSearch connector was added in Griffin. We want to enrich it by adding sql mode.
> ES SQL plugins are very popular, such as :  [https://github.com/NLPchina/elasticsearch-sql/wiki |https://github.com/NLPchina/elasticsearch-sql/wiki]and  [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/xpack-sql.html].
> For example: 
> {code:java}
> curl -X POST 'http://{es_url}/_sql?format=csv'  -H 'Content-Type: application/json'  -d 'select a, b, c from {index} where a is not null limit 2'{code}
> would get csv response as follows:
>  
> {code:java}
> a,b,c
> 1,test,1.55
> 2,test,1.57{code}
>  
> So we want Griffin ES Connector could load data by ES SQL in sql mode.  In sql mode, user could config Connector for ES (with sql plugin) like: 
>  
> {{{ "type": "custom",}}
> {{  "config": {}}
> {{    "class": "org.apache.griffin.measure.datasource.connector.batch.ElasticSearchGriffinDataConnector",}}
> {{     "host": "xxxxxxx",}}
> {{     "port": "xxxx",}}
> {{     "sql": "select a, b, c from xxx where a = 2 limit 100"}}}
> {{}}}
>  
>  Which has the following benefits:
>  # For case that only part of the ES data is needed, or only aggregated result is needed, it could significantly reduce IO and improve performance by handle filter and aggregation to ES. 
>  # It is more user-friendly, since we don't need to define data size and schema. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)