[jira] [Created] (GRIFFIN-322) Add SQL mode for ES connector

["XIAOYU YU (Jira)" <ji...@apache.org>]

XIAOYU YU created GRIFFIN-322:
---------------------------------

             Summary: Add SQL mode for ES connector
                 Key: GRIFFIN-322
                 URL: https://issues.apache.org/jira/browse/GRIFFIN-322
             Project: Griffin
          Issue Type: Improvement
            Reporter: XIAOYU YU


The ElasticSearch connector was added in Griffin. We want to enrich it by adding sql mode.

ES SQL plugins are very popular, such as :  [https://github.com/NLPchina/elasticsearch-sql/wiki |https://github.com/NLPchina/elasticsearch-sql/wiki]and  [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/xpack-sql.html].

For example: 
{{}}
{code:java}
curl -X POST 'http://{es_url}/_sql?format=csv'  -H 'Content-Type: application/json'  -d 'select a, b, c from {index} where a is not null limit 2'{code}
{{}}
would get csv response as follows:

 
{code:java}
a,b,c
1,test,1.55
2,test,1.57{code}
 

So we want Griffin ES Connector to load data by ES SQL like:

 

{{}}
{code:java}

{code}
{{"connectors": [ { }}
{{"type": "ElasticSearch",  }}
{{"config": \{ "host": "xxxx", "port": xxxx, "sql": "select a, b, c from xxx where a is not null" } }}
{{} ]}}

 

 which has the following benefits:
 # For case that only part of the ES data is needed, or only aggregated result is needed, it could significantly reduce IO and improve performance by handle filter and aggregation to ES. 
 # It is more user-friendly, since we don't need to define data size and schema. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)