You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by dz...@apache.org on 2021/02/15 12:57:19 UTC

[drill] branch gh-pages updated (07b1902 -> 571cba8)

This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a change to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git.


    from 07b1902  [DOC UPDATE] Fix invalid link for 110-troubleshooting.md (#2161)
     new 1f1a7e8  Correct typos in 060-querying-the-information-schema.md
     new f5f4d2f  Fix Drill output formatting in 053-describe.md
     new ad62f65  Add links to Slack channels to menu bar and selected pages.
     new bdc00b6  Add 030-rdbms-metastore.md
     new b9af195  Convert <span>s to <a>s in 110-troubleshooting.md to make them linkable.
     new 27a2a57  Document the ElasticSearch storage plugin.
     new 571cba8  Update README.md to reflect new docs software versions.

The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 README.md                                          |  22 ++-
 _data/docs.json                                    | 120 +++++++++++--
 _docs/110-troubleshooting.md                       |   6 +-
 .../plugins/130-elasticsearch.md                   |  55 ++++++
 ...metastore.md => 020-drill-iceberg-metastore.md} |   0
 .../drill-metastore/030-rdbms-metastore.md         | 161 +++++++++++++++++
 .../060-querying-the-information-schema.md         |  13 +-
 _docs/sql-reference/sql-commands/053-describe.md   | 199 +++++++++++----------
 _includes/menu.html                                |   4 +-
 _sass/_site-main.scss                              |   7 +-
 community-resources.md                             |   3 +-
 images/slack-logo.svg                              |  99 ++++++++++
 poweredBy.html                                     |   4 +-
 13 files changed, 555 insertions(+), 138 deletions(-)
 create mode 100644 _docs/connect-a-data-source/plugins/130-elasticsearch.md
 rename _docs/performance-tuning/drill-metastore/{030-drill-iceberg-metastore.md => 020-drill-iceberg-metastore.md} (100%)
 create mode 100644 _docs/performance-tuning/drill-metastore/030-rdbms-metastore.md
 create mode 100644 images/slack-logo.svg


[drill] 07/07: Update README.md to reflect new docs software versions.

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit 571cba89aa9f77cc63db8b9adbe84395b2b12f49
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 14:56:13 2021 +0200

    Update README.md to reflect new docs software versions.
---
 README.md | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/README.md b/README.md
index 30e496c..3f2df7d 100644
--- a/README.md
+++ b/README.md
@@ -2,29 +2,35 @@ The Apache Drill website is built using [Jekyll](http://jekyllrb.com/).
 
 # Configuring env
 1. Install `ruby`
-2. Install `bundler` and `jekyll` `v2.5.2`:
+2. Install `bundler` and `jekyll` `v3.9.0`:
 ```
-gem install bundler jekyll:2.5.2
+gem install bundler jekyll:3.9.0
 ```
 3. Install `jekyll-redirect-from` `v0.9.1`:
 ```
 gem install jekyll-redirect-from:0.9.1
 ```
-4. Install Python v2.7.15.
+4. Install Python 3
 
 Please make sure that specific versions of libraries are installed since building the site with other versions may cause some issues like including md document index into the references, etc.
 
+## Note for existing contributors
+The software version numbers above underwent a major increase in 2020 and the Markdown processor
+changed from Redcarpet to Kramdown.  Please check the versions in your environment if you're having
+trouble generating the site.
+
 # Documentation Guidelines
 
-The documentation pages are placed under `_docs`. You can modify existing .md files, or you can create new .md files to add to the Apache Drill documentation site. Create pull requests to submit your documentation updates. The Kramdown MarkDown processor employed by Jekyll supports [a dialect of MarkDown](https://kramdown.gettalong.org/quickref.html) which is a superset of standard MarkDown.
+The documentation pages are placed under `_docs`. You can modify existing .md files, or you can create new .md files to add to the Apache Drill documentation site. Create pull requests to submit your documentation updates. The Kramdown Markdown processor employed by Jekyll supports [a dialect of Markdown](https://kramdown.gettalong.org/quickref.html) which is a superset of standard Markdown.
 
-## Creating New MarkDown Files
+## Creating New Markdown Files
 
-If you create new MarkDown (.md) files, include the required YAML front matter and name the file using the methods described in this section. 
+If you create new Markdown (.md) files, include the required YAML front matter and name the file using the methods described in this section. 
 
 The YAML front matter has three important parameters:
 
 * `title:` - This is the title of the page enclosed in quotation marks. Each page must have a *unique* title
+* `slug:` - Set this to the same value as `title`, it will be slugified automatically by Jekyll.
 * `date:` - This field is needed for Jekyll to write a last-modified date. Initially, leave this field blank.
 * `parent:` - This is the title of the page's parent page. It should be empty for top-level sections/guides, and be identical to the title attribute of another page in all other cases.
 
@@ -33,7 +39,7 @@ The name of the file itself doesn't matter except for the alphanumeric order of
 Best practices:
 
 * Prefix the filenames with `010-foo.md`, `020-bar.md`, `030-baz.md`, etc. This allows room to add files in-between (eg, `005-qux.md`).  
-* Use the slug of the title as the filename. For example, if the title is "Getting Started with Drill", name the file `...-getting-started-with-drill.md`. If you're not sure what the slug is, you should be able to see it in the URL and then adjust (the URLs are auto-generated based on the title attribute).
+* Use the slugified title as the filename. For example, if the title is "Getting Started with Drill", name the file `...-getting-started-with-drill.md`. If you're not sure what the slug is, you should be able to see it in the URL and then adjust (the URLs are auto-generated based on the title attribute).
 
 # Developing and Previewing the Website
 
@@ -42,7 +48,7 @@ To preview the website on your local machine:
 ```bash
 jekyll build --config _config.yml,_config-prod.yml
 _tools/createdatadocs.py
-jekyll serve --config _config.yml,_config-prod.yml
+jekyll serve --config _config.yml,_config-prod.yml [--livereload] [--incremental]
 ```
 Note that you can skip the first two commands (and only run `jekyll serve`) if you haven't changed the title or path of any of the documentation pages.
 


[drill] 05/07: Convert s to s in 110-troubleshooting.md to make them linkable.

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit b9af19567379d691ec48c7d63b9018aff3dd4682
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 13:06:46 2021 +0200

    Convert <span>s to <a>s in 110-troubleshooting.md to make them linkable.
---
 _docs/110-troubleshooting.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/_docs/110-troubleshooting.md b/_docs/110-troubleshooting.md
index 93d27c2..88816f0 100644
--- a/_docs/110-troubleshooting.md
+++ b/_docs/110-troubleshooting.md
@@ -105,7 +105,7 @@ Solutions:
   * Parquet
   * JSON
 
-### <span id='access-nested-fields-without-table-name-alias'>Access Nested Fields without Table Name/Alias</span>
+### <a id='access-nested-fields-without-table-name-alias'>Access Nested Fields without Table Name/Alias</a>
 Symptom: 
 
        SELECT x.y …  
@@ -179,7 +179,7 @@ Symptom: ODBC errors.
 Solution: Make sure that the ODBC driver version is compatible with the server version. [Driver installation instructions]({{site.baseurl}}/docs/installing-the-odbc-driver) include how to check the driver version. 
 Turn on ODBC driver debug logging to better understand failure.  
 
-### <span id='jdbc-odbc-connection-issues-with-zookeeper'>JDBC/ODBC Connection Issues with ZooKeeper</span>
+### <a id='jdbc-odbc-connection-issues-with-zookeeper'>JDBC/ODBC Connection Issues with ZooKeeper</a>
 
 Symptom: Client cannot resolve ZooKeeper host names for JDBC/ODBC.
 Symptom: "IllegalStateException: No active Drillbit endpoint found from ZooKeeper. Check connection parameters?" 
@@ -211,7 +211,7 @@ Symptom: UNSUPPORTED\_OPERATION ERROR: Null values are not supported in lists by
 
 Solution: Avoid selecting fields that are arrays containing nulls. Change Drill session settings to enable all_text_mode. Set store.json.all\_text_mode to true, so Drill treats JSON null values as a string containing the word 'null'.
 
-### <span id='select-count-takes-a-long-time-to-run'>SELECT COUNT (\*) Takes a Long Time to Run</span>
+### <a id='select-count-takes-a-long-time-to-run'>SELECT COUNT (\*) Takes a Long Time to Run</a>
 
 Solution: In some cases, the underlying storage format does not have a built-in capability to return a count of records in a table.  In these cases, Drill does a full scan of the data to verify the number of records.
 


[drill] 01/07: Correct typos in 060-querying-the-information-schema.md

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit 1f1a7e826677f99d6e7fcbd57589be6ba0543602
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Fri Jan 15 17:02:25 2021 +0200

    Correct typos in 060-querying-the-information-schema.md
---
 _docs/query-data/060-querying-the-information-schema.md | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/_docs/query-data/060-querying-the-information-schema.md b/_docs/query-data/060-querying-the-information-schema.md
index f55fa68..06ca877 100644
--- a/_docs/query-data/060-querying-the-information-schema.md
+++ b/_docs/query-data/060-querying-the-information-schema.md
@@ -71,14 +71,14 @@ type) for each column in each table or view.
 
 ## VIEWS
 
-The VIEWS table returns the name and definition for each view in your
-databases. Note that file schemas are the canonical repository for views in
-Drill. Depending on how you create a view, they may only be displayed in Drill
-after it has been used.
+The VIEWS table returns the name and definition for each view in your databases.  Note that file
+schemas are the canonical repository for views in Drill.  Depending on how you create it, a view may
+only be displayed in Drill after it has been used.
 
 ## PARTITIONS
 
-The PARTITIONS table returns information about table partitions including partition columns, partition keys, etc. for each table.
+The PARTITIONS table returns information about table partitions, including partition columns,
+partition keys, etc. for each table.
 
 ## FILES
 
@@ -142,7 +142,8 @@ For this example, S3 buckets were configured as data sources in Drill. Storage p
 	| DRILL         | s3_years_bucket.root     | <owner>       | file  | NO          |
 	|---------------|--------------------------|---------------|-------|-------------|  
 
-Querying the FILES table and filtering on the SCHEMA_NAME provides information about the files that exist within a workspace:  
+Querying the FILES table and filtering on the SCHEMA_NAME provides information about the files
+within a workspace:  
 
 **Note:** The word “files” is a reserved word in Drill and requires backticks (``).   
 


[drill] 04/07: Add 030-rdbms-metastore.md

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit bdc00b63923507ec217880ce75ca795f6be4ed29
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 13:05:35 2021 +0200

    Add 030-rdbms-metastore.md
---
 _data/docs.json                                    | 120 ++++++++++++---
 ...metastore.md => 020-drill-iceberg-metastore.md} |   0
 .../drill-metastore/030-rdbms-metastore.md         | 161 +++++++++++++++++++++
 _sass/_site-main.scss                              |   1 +
 4 files changed, 264 insertions(+), 18 deletions(-)

diff --git a/_data/docs.json b/_data/docs.json
index f764bcf..a7a7cab 100644
--- a/_data/docs.json
+++ b/_data/docs.json
@@ -4708,12 +4708,12 @@
                 }
             ],
             "children": [],
-            "next_title": "Performance Tuning Introduction",
-            "next_url": "/docs/performance-tuning-introduction/",
+            "next_title": "RDBMS Metastore",
+            "next_url": "/docs/rdbms-metastore/",
             "parent": "Drill Metastore",
             "previous_title": "Using Drill Metastore",
             "previous_url": "/docs/using-drill-metastore/",
-            "relative_path": "_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md",
+            "relative_path": "_docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md",
             "title": "Drill Iceberg Metastore",
             "url": "/docs/drill-iceberg-metastore/"
         },
@@ -4775,14 +4775,35 @@
                         }
                     ],
                     "children": [],
-                    "next_title": "Performance Tuning Introduction",
-                    "next_url": "/docs/performance-tuning-introduction/",
+                    "next_title": "RDBMS Metastore",
+                    "next_url": "/docs/rdbms-metastore/",
                     "parent": "Drill Metastore",
                     "previous_title": "Using Drill Metastore",
                     "previous_url": "/docs/using-drill-metastore/",
-                    "relative_path": "_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md",
+                    "relative_path": "_docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md",
                     "title": "Drill Iceberg Metastore",
                     "url": "/docs/drill-iceberg-metastore/"
+                },
+                {
+                    "breadcrumbs": [
+                        {
+                            "title": "Drill Metastore",
+                            "url": "/docs/drill-metastore/"
+                        },
+                        {
+                            "title": "Performance Tuning",
+                            "url": "/docs/performance-tuning/"
+                        }
+                    ],
+                    "children": [],
+                    "next_title": "Performance Tuning Introduction",
+                    "next_url": "/docs/performance-tuning-introduction/",
+                    "parent": "Drill Metastore",
+                    "previous_title": "Drill Iceberg Metastore",
+                    "previous_url": "/docs/drill-iceberg-metastore/",
+                    "relative_path": "_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md",
+                    "title": "RDBMS Metastore",
+                    "url": "/docs/rdbms-metastore/"
                 }
             ],
             "next_title": "Using Drill Metastore",
@@ -8419,14 +8440,35 @@
                                 }
                             ],
                             "children": [],
-                            "next_title": "Performance Tuning Introduction",
-                            "next_url": "/docs/performance-tuning-introduction/",
+                            "next_title": "RDBMS Metastore",
+                            "next_url": "/docs/rdbms-metastore/",
                             "parent": "Drill Metastore",
                             "previous_title": "Using Drill Metastore",
                             "previous_url": "/docs/using-drill-metastore/",
-                            "relative_path": "_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md",
+                            "relative_path": "_docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md",
                             "title": "Drill Iceberg Metastore",
                             "url": "/docs/drill-iceberg-metastore/"
+                        },
+                        {
+                            "breadcrumbs": [
+                                {
+                                    "title": "Drill Metastore",
+                                    "url": "/docs/drill-metastore/"
+                                },
+                                {
+                                    "title": "Performance Tuning",
+                                    "url": "/docs/performance-tuning/"
+                                }
+                            ],
+                            "children": [],
+                            "next_title": "Performance Tuning Introduction",
+                            "next_url": "/docs/performance-tuning-introduction/",
+                            "parent": "Drill Metastore",
+                            "previous_title": "Drill Iceberg Metastore",
+                            "previous_url": "/docs/drill-iceberg-metastore/",
+                            "relative_path": "_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md",
+                            "title": "RDBMS Metastore",
+                            "url": "/docs/rdbms-metastore/"
                         }
                     ],
                     "next_title": "Using Drill Metastore",
@@ -8449,8 +8491,8 @@
                     "next_title": "Partition Pruning",
                     "next_url": "/docs/partition-pruning/",
                     "parent": "Performance Tuning",
-                    "previous_title": "Drill Iceberg Metastore",
-                    "previous_url": "/docs/drill-iceberg-metastore/",
+                    "previous_title": "RDBMS Metastore",
+                    "previous_url": "/docs/rdbms-metastore/",
                     "relative_path": "_docs/performance-tuning/010-performance-tuning-introduction.md",
                     "title": "Performance Tuning Introduction",
                     "url": "/docs/performance-tuning-introduction/"
@@ -8944,8 +8986,8 @@
             "next_title": "Partition Pruning",
             "next_url": "/docs/partition-pruning/",
             "parent": "Performance Tuning",
-            "previous_title": "Drill Iceberg Metastore",
-            "previous_url": "/docs/drill-iceberg-metastore/",
+            "previous_title": "RDBMS Metastore",
+            "previous_url": "/docs/rdbms-metastore/",
             "relative_path": "_docs/performance-tuning/010-performance-tuning-introduction.md",
             "title": "Performance Tuning Introduction",
             "url": "/docs/performance-tuning-introduction/"
@@ -10827,6 +10869,27 @@
             "title": "Querying the INFORMATION SCHEMA",
             "url": "/docs/querying-the-information-schema/"
         },
+        "RDBMS Metastore": {
+            "breadcrumbs": [
+                {
+                    "title": "Drill Metastore",
+                    "url": "/docs/drill-metastore/"
+                },
+                {
+                    "title": "Performance Tuning",
+                    "url": "/docs/performance-tuning/"
+                }
+            ],
+            "children": [],
+            "next_title": "Performance Tuning Introduction",
+            "next_url": "/docs/performance-tuning-introduction/",
+            "parent": "Drill Metastore",
+            "previous_title": "Drill Iceberg Metastore",
+            "previous_url": "/docs/drill-iceberg-metastore/",
+            "relative_path": "_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md",
+            "title": "RDBMS Metastore",
+            "url": "/docs/rdbms-metastore/"
+        },
         "RDBMS Storage Plugin": {
             "breadcrumbs": [
                 {
@@ -20035,14 +20098,35 @@
                                 }
                             ],
                             "children": [],
-                            "next_title": "Performance Tuning Introduction",
-                            "next_url": "/docs/performance-tuning-introduction/",
+                            "next_title": "RDBMS Metastore",
+                            "next_url": "/docs/rdbms-metastore/",
                             "parent": "Drill Metastore",
                             "previous_title": "Using Drill Metastore",
                             "previous_url": "/docs/using-drill-metastore/",
-                            "relative_path": "_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md",
+                            "relative_path": "_docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md",
                             "title": "Drill Iceberg Metastore",
                             "url": "/docs/drill-iceberg-metastore/"
+                        },
+                        {
+                            "breadcrumbs": [
+                                {
+                                    "title": "Drill Metastore",
+                                    "url": "/docs/drill-metastore/"
+                                },
+                                {
+                                    "title": "Performance Tuning",
+                                    "url": "/docs/performance-tuning/"
+                                }
+                            ],
+                            "children": [],
+                            "next_title": "Performance Tuning Introduction",
+                            "next_url": "/docs/performance-tuning-introduction/",
+                            "parent": "Drill Metastore",
+                            "previous_title": "Drill Iceberg Metastore",
+                            "previous_url": "/docs/drill-iceberg-metastore/",
+                            "relative_path": "_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md",
+                            "title": "RDBMS Metastore",
+                            "url": "/docs/rdbms-metastore/"
                         }
                     ],
                     "next_title": "Using Drill Metastore",
@@ -20065,8 +20149,8 @@
                     "next_title": "Partition Pruning",
                     "next_url": "/docs/partition-pruning/",
                     "parent": "Performance Tuning",
-                    "previous_title": "Drill Iceberg Metastore",
-                    "previous_url": "/docs/drill-iceberg-metastore/",
+                    "previous_title": "RDBMS Metastore",
+                    "previous_url": "/docs/rdbms-metastore/",
                     "relative_path": "_docs/performance-tuning/010-performance-tuning-introduction.md",
                     "title": "Performance Tuning Introduction",
                     "url": "/docs/performance-tuning-introduction/"
diff --git a/_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md b/_docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md
similarity index 100%
rename from _docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md
rename to _docs/performance-tuning/drill-metastore/020-drill-iceberg-metastore.md
diff --git a/_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md b/_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md
new file mode 100644
index 0000000..e10ce68
--- /dev/null
+++ b/_docs/performance-tuning/drill-metastore/030-rdbms-metastore.md
@@ -0,0 +1,161 @@
+---
+title: "RDBMS Metastore"
+slug: "RDBMS Metastore"
+parent: "Drill Metastore"
+---
+
+The RDBMS Metastore implementation allows you store Drill Metastore metadata in a configured RDBMS.
+
+## Configuration
+
+Currently, the RDBMS Metastore is not the default implementation.
+To enable the RDBMS Metastore create the `drill-metastore-override.conf` file 
+in your config directory and specify the RDBMS Metastore class:
+
+```yaml
+drill.metastore: {
+  implementation.class: "org.apache.drill.metastore.rdbms.RdbmsMetastore"
+}
+```
+
+### Connection properties
+
+Use the connection properties to specify how Drill should connect to your Metastore database.
+
+`drill.metastore.rdbms.data_source.driver` - driver class name. Required. 
+Note: the driver class must be included into the Drill classpath. 
+The easiest way to do that is to put the driver jar file into the `$DRILL_HOME/jars/3rdparty` folder.
+Or, to make upgrades easier, in your `$DRILL_SITE/jars` folder.  Drill includes the driver for SQLite.
+
+`drill.metastore.rdbms.data_source.url` - connection url. Required.
+
+`drill.metastore.rdbms.data_source.username` - database user on whose behalf the connection is
+being made. Optional, if database does not require user to connect. 
+
+`drill.metastore.rdbms.data_source.password` - database user's password. 
+Optional, if database does not require user's password to connect.
+
+`drill.metastore.rdbms.data_source.properties` - specifies properties which will be used
+during data source creation. See list of available [Hikari properties](https://github.com/brettwooldridge/HikariCP)
+for more details.
+
+### Default configuration 
+
+Out of the box, the Drill RDBMS Metastore is configured to use the embedded file system based SQLite database.
+It will be created locally in user's home directory under `${drill.exec.zk.root}"/metastore` location.
+
+Default setup can be used only in Drill embedded mode. SQLite is an embedded database; is not distributed. 
+SQLite is good for trying out the feature, for testing, for a running Drill in embedded mode, 
+and perhaps for a single-node Drill "cluster". If should not be used in a multi-node cluster. 
+Each Drillbit will have its own version of the truth and behavior will be undefined and incorrect.
+
+### Custom configuration
+
+`drill-metastore-override.conf` is used to customize connection details to the Drill Metastore database.
+See `drill-metastore-override-example.conf` for more details.
+
+#### Example of PostgreSQL configuration
+
+```yaml
+drill.metastore: {
+  implementation.class: "org.apache.drill.metastore.rdbms.RdbmsMetastore",
+  rdbms: {
+    data_source: {
+      driver: "org.postgresql.Driver",
+      url: "jdbc:postgresql://localhost:1234/mydb?currentSchema=drill_metastore",
+      username: "user",
+      password: "password"
+    }
+  }
+}
+```
+
+Note: as mentioned above, the PostgreSQL JDBC driver must be present in the Drill classpath.
+
+#### Example of MySQL configuration
+
+```yaml
+drill.metastore: {
+  implementation.class: "org.apache.drill.metastore.rdbms.RdbmsMetastore",
+  rdbms: {
+    data_source: {
+      driver: "com.mysql.cj.jdbc.Driver",
+      url: "jdbc:mysql://localhost:1234/drill_metastore",
+      username: "user",
+      password: "password"
+    }
+  }
+}
+```
+
+Note: as mentioned above, the MySQL JDBC driver must be present in the Drill classpath.
+
+{% include startnote.html %}
+For MySQL connector version 6+, use the <code>com.mysql.cj.jdbc.Driver</code> driver class,
+for older versions use the <code>com.mysql.jdbc.Driver</code>.
+{% include endnote.html %}
+
+## Tables structure
+
+The Drill Metastore stores several types of metadata, called components. Currently, only the `tables` component is implemented.
+The `tables` component provides metadata about Drill tables, including their segments, files, row groups and partitions.
+In Drill `tables` component unit is represented by `TableMetadataUnit` class which is applicable to any metadata type.
+The `TableMetadataUnit` class holds fields for all five metadata types within the `tables` component. 
+Any fields not applicable to a particular metadata type are simply ignored and remain unset.
+
+In the RDBMS implementation of the Drill Metastore, the tables component includes five tables, one for each metadata type. 
+The five tables are: `TABLES`, `SEGMENTS`, `FILES`, `ROW_GROUPS`, and `PARTITIONS`.
+See `src/main/resources/db/changelog/changes/initial_ddls.yaml` for the schema and indexes of each table.
+
+The Drill Metastore API has the following semantics:
+* most of the time all data about component is accessed;
+* data is filtered by non-complex fields, like storage plugin, workspace, table name, etc;
+* data is overwritten fully, there is no update by certain fields.
+
+Taking into account the Drill Metastore API semantics, the RDBMS Drill Metastore schema is slightly denormalized.
+Having normalized structure would lead to unnecessary joins during select, index re-indexing during update.
+
+### Table creation
+
+The RDBMS Metastore uses [Liquibase](https://www.liquibase.org/documentation/core-concepts/index.html)
+to create the needed tables during the RDBMS Metastore initialization. Users should not create any tables manually.
+
+### Database schema
+
+Liquibase uses a yaml configuration file to apply changes to the database schema: `src/main/resources/db/changelog/changelog.yaml`.
+Liquibase converts the yaml specification into the DDL / DML commands suitable required for the configured database.
+See list of supported databases: https://www.liquibase.org/databases.html.
+
+The Drill Metastore tables are created in the database schema indicated in the connection URL.
+This will be the default schema unless you specify a different schema. Drill will not create the schema, however. 
+Best practice is to create a schema within your database for the Drill metastore before initializing the Metastore.
+
+Example:
+
+PostgreSQL: `jdbc:postgresql://localhost:1234/mydb?currentSchema=drill_metastore`
+
+MySQL: `jdbc:mysql://localhost:1234/drill_metastore`
+
+Since Drill will create the required tables, ensure that the database user has the following permissions in the metastore schema:
+* read and write tables;
+* create and modify database objects (tables, indexes, views, etc.).
+
+### Liquibase tables
+
+During Drill RDBMS Metastore initialization, Liquibase will create two internal tracking tables:
+`DATABASECHANGELOG` and `DATABASECHANGELOGLOCK`. They are needed to track schema changes and concurrent updates.
+See https://www.liquibase.org/get_started/how-lb-works.html for more details.
+
+## Query execution
+
+SQL queries issued to RDBMS Metastore tables are generated using [JOOQ](https://www.jooq.org/doc/3.13/manual/getting-started/).
+Drill uses the open-source version of JOOQ to generate the queries sent to the configured Metastore database.
+
+JOOQ generates SQL statements based on SQL dialect determined by database connection details.
+List of supported dialects: https://www.jooq.org/javadoc/3.13.x/org.jooq/org/jooq/SQLDialect.html.
+Note: dialects annotated with `@Pro` are not supported, since open-source version of JOOQ is used.
+
+## Supported databases
+
+The RDBMS Metastore was tested with `SQLite`, `PostreSQL` and `MySQL`. Other databases should also work
+if there is Liquibase and JOOQ support for them.
diff --git a/_sass/_site-main.scss b/_sass/_site-main.scss
index 8374782..5ebdf39 100644
--- a/_sass/_site-main.scss
+++ b/_sass/_site-main.scss
@@ -147,6 +147,7 @@ a.anchor {
 
 #menu ul li.social-menu-item img {
   width: 22px;
+  padding-bottom: 10px;
 }
 
 #menu ul li ul {


[drill] 03/07: Add links to Slack channels to menu bar and selected pages.

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit ad62f65cf76c357986bd95f9fe17f9d01619dd2a
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 12:02:57 2021 +0200

    Add links to Slack channels to menu bar and selected pages.
---
 _includes/menu.html    |  4 +-
 _sass/_site-main.scss  |  6 +--
 community-resources.md |  3 +-
 images/slack-logo.svg  | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++
 poweredBy.html         |  4 +-
 5 files changed, 109 insertions(+), 7 deletions(-)

diff --git a/_includes/menu.html b/_includes/menu.html
index 0b36b34..dfac142 100644
--- a/_includes/menu.html
+++ b/_includes/menu.html
@@ -34,7 +34,9 @@
   </li>
   <li class='nav'><a href="{{ site.baseurl }}/faq/">FAQ</a></li>
   <li class='nav'><a href="{{ site.baseurl }}/blog/">Blog</a></li>
-  <li id="twitter-menu-item"><a href="https://twitter.com/apachedrill" title="apachedrill on twitter" target="_blank"><img src="{{ site.baseurl }}/images/twitter_32_26_white.png" alt="twitter logo" align="center"></a> </li>
+  <li class="social-menu-item"><a href="https://twitter.com/apachedrill" title="apachedrill on twitter" target="_blank"><img src="{{ site.baseurl }}/images/twitter_32_26_white.png" alt="twitter logo" align="center"></a> </li>
+  <li class="social-menu-item"><a href="https://apache-drill.slack.com/" title="Apache Drill Slack channels"
+      target="_blank"><img src="{{ site.baseurl }}/images/slack-logo.svg" alt="Slack logo" align="center"></a> </li>
   <li class='search-bar'>
     <form id="drill-search-form">
       <input type="text" placeholder="Search Apache Drill" id="drill-search-term" />
diff --git a/_sass/_site-main.scss b/_sass/_site-main.scss
index 65c4ce8..8374782 100644
--- a/_sass/_site-main.scss
+++ b/_sass/_site-main.scss
@@ -135,17 +135,17 @@ a.anchor {
   background-color:#145aa8;
 }
 
-#menu ul li#twitter-menu-item {
+#menu ul li.social-menu-item {
   width:30px;
   padding-left: 2px;
   padding-right:10px;
 }
 
-#menu ul li#twitter-menu-item a {
+#menu ul li.social-menu-item a {
   padding: 10px;
 }
 
-#menu ul li#twitter-menu-item img {
+#menu ul li.social-menu-item img {
   width: 22px;
 }
 
diff --git a/community-resources.md b/community-resources.md
index 4712781..7f5875b 100644
--- a/community-resources.md
+++ b/community-resources.md
@@ -6,6 +6,7 @@ title: "Community Resources"
 ## User Resources
 
 * [Documentation]({{ site.baseurl }}/docs/)
+* [Slack channels for users and developers](https://apache-drill.slack.com/)
 * Twitter: [@ApacheDrill](https://twitter.com/ApacheDrill)
 * Meetups:
     * [Bay Area](http://www.meetup.com/Bay-Area-Apache-Drill-User-Group/)
@@ -27,4 +28,4 @@ title: "Community Resources"
 * See the generous supporters of the Apache Software Foundation at [http://www.apache.org/foundation/thanks.html](http://www.apache.org/foundation/thanks.html).
 * Find out how you can become a sponsor of the Apache Software Foundation at [http://www.apache.org/foundation/sponsorship.html](http://www.apache.org/foundation/sponsorship.html).
 * Get Apache Software Foundation licensing information at [http://www.apache.org/licenses/](http://www.apache.org/licenses/).
-* Report security vulnerabilities at [http://www.apache.org/security/](http://www.apache.org/security/). 
\ No newline at end of file
+* Report security vulnerabilities at [http://www.apache.org/security/](http://www.apache.org/security/). 
diff --git a/images/slack-logo.svg b/images/slack-logo.svg
new file mode 100644
index 0000000..db4ff9c
--- /dev/null
+++ b/images/slack-logo.svg
@@ -0,0 +1,99 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   version="1.1"
+   id="Layer_1"
+   x="0px"
+   y="0px"
+   viewBox="0 0 122.8 122.8"
+   xml:space="preserve"
+   sodipodi:docname="slack-logo.svg"
+   width="122.8"
+   height="122.8"
+   inkscape:version="1.0.2 (e86c870879, 2021-01-15)"><metadata
+   id="metadata35"><rdf:RDF><cc:Work
+       rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
+         rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
+   id="defs33" /><sodipodi:namedview
+   pagecolor="#ffffff"
+   bordercolor="#666666"
+   borderopacity="1"
+   objecttolerance="10"
+   gridtolerance="10"
+   guidetolerance="10"
+   inkscape:pageopacity="0"
+   inkscape:pageshadow="2"
+   inkscape:window-width="1916"
+   inkscape:window-height="1029"
+   id="namedview31"
+   showgrid="false"
+   inkscape:zoom="1.6685185"
+   inkscape:cx="61.4"
+   inkscape:cy="61.4"
+   inkscape:window-x="0"
+   inkscape:window-y="0"
+   inkscape:window-maximized="1"
+   inkscape:current-layer="Layer_1" />
+<style
+   type="text/css"
+   id="style2">
+	.st0{fill:#E01E5A;}
+	.st1{fill:#36C5F0;}
+	.st2{fill:#2EB67D;}
+	.st3{fill:#ECB22E;}
+</style>
+<g
+   id="g28"
+   transform="translate(-73.6,-73.6)">
+	<g
+   id="g8">
+		<path
+   class="st0"
+   d="m 99.4,151.2 c 0,7.1 -5.8,12.9 -12.9,12.9 -7.1,0 -12.9,-5.8 -12.9,-12.9 0,-7.1 5.8,-12.9 12.9,-12.9 h 12.9 z"
+   id="path4" />
+		<path
+   class="st0"
+   d="m 105.9,151.2 c 0,-7.1 5.8,-12.9 12.9,-12.9 7.1,0 12.9,5.8 12.9,12.9 v 32.3 c 0,7.1 -5.8,12.9 -12.9,12.9 -7.1,0 -12.9,-5.8 -12.9,-12.9 z"
+   id="path6" />
+	</g>
+	<g
+   id="g14">
+		<path
+   class="st1"
+   d="m 118.8,99.4 c -7.1,0 -12.9,-5.8 -12.9,-12.9 0,-7.1 5.8,-12.9 12.9,-12.9 7.1,0 12.9,5.8 12.9,12.9 v 12.9 z"
+   id="path10" />
+		<path
+   class="st1"
+   d="m 118.8,105.9 c 7.1,0 12.9,5.8 12.9,12.9 0,7.1 -5.8,12.9 -12.9,12.9 H 86.5 c -7.1,0 -12.9,-5.8 -12.9,-12.9 0,-7.1 5.8,-12.9 12.9,-12.9 z"
+   id="path12" />
+	</g>
+	<g
+   id="g20">
+		<path
+   class="st2"
+   d="m 170.6,118.8 c 0,-7.1 5.8,-12.9 12.9,-12.9 7.1,0 12.9,5.8 12.9,12.9 0,7.1 -5.8,12.9 -12.9,12.9 h -12.9 z"
+   id="path16" />
+		<path
+   class="st2"
+   d="m 164.1,118.8 c 0,7.1 -5.8,12.9 -12.9,12.9 -7.1,0 -12.9,-5.8 -12.9,-12.9 V 86.5 c 0,-7.1 5.8,-12.9 12.9,-12.9 7.1,0 12.9,5.8 12.9,12.9 z"
+   id="path18" />
+	</g>
+	<g
+   id="g26">
+		<path
+   class="st3"
+   d="m 151.2,170.6 c 7.1,0 12.9,5.8 12.9,12.9 0,7.1 -5.8,12.9 -12.9,12.9 -7.1,0 -12.9,-5.8 -12.9,-12.9 v -12.9 z"
+   id="path22" />
+		<path
+   class="st3"
+   d="m 151.2,164.1 c -7.1,0 -12.9,-5.8 -12.9,-12.9 0,-7.1 5.8,-12.9 12.9,-12.9 h 32.3 c 7.1,0 12.9,5.8 12.9,12.9 0,7.1 -5.8,12.9 -12.9,12.9 z"
+   id="path24" />
+	</g>
+</g>
+</svg>
diff --git a/poweredBy.html b/poweredBy.html
index 03bb94b..48fec18 100644
--- a/poweredBy.html
+++ b/poweredBy.html
@@ -11,7 +11,7 @@ title: "Powered By Drill"
         resources that discuss their applications in more detail.</p>
     <p>Please note that the list is not comprehensive, and
         only lists users that explicitly gave consent.
-        If you would you like to be included on this page, please reach out by directly send a message to <a
+        If you would you like to be included on this page, please reach out by sending a direct message to <a
                 href="https://twitter.com/apachedrill">@ApacheDrill</a>
         on Twitter or to kunal[at]apache.org
     </p>
@@ -27,4 +27,4 @@ title: "Powered By Drill"
         {% endif %}
     </div>
     {% endfor %}
-</div>
\ No newline at end of file
+</div>


[drill] 02/07: Fix Drill output formatting in 053-describe.md

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit f5f4d2f52701141b6c6b6c9ef3ef6c6e9683a069
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 12:02:16 2021 +0200

    Fix Drill output formatting in 053-describe.md
---
 _docs/sql-reference/sql-commands/053-describe.md | 199 ++++++++++++-----------
 1 file changed, 103 insertions(+), 96 deletions(-)

diff --git a/_docs/sql-reference/sql-commands/053-describe.md b/_docs/sql-reference/sql-commands/053-describe.md
index 08162cc..a66b9e8 100644
--- a/_docs/sql-reference/sql-commands/053-describe.md
+++ b/_docs/sql-reference/sql-commands/053-describe.md
@@ -8,10 +8,11 @@ The DESCRIBE command returns information about columns in a table, view, or sche
 ## Syntax
 
 The DESCRIBE command supports the following syntax:  
-
-       DESCRIBE [workspace.]table_name|view_name [column_name]
-       DESCRIBE SCHEMA|DATABASE <name>[.workspace]
-       DESCRIBE (query)
+```sql
+DESCRIBE [workspace.]table_name|view_name [column_name]
+DESCRIBE SCHEMA|DATABASE <name>[.workspace]
+DESCRIBE (query)
+```
 
 
 
@@ -65,99 +66,105 @@ and HBase tables.
 
 Complete the following steps to use the DESCRIBE command:
 
-  1. Issue the USE command to switch to a particular schema.
-
-        0: jdbc:drill:zk=drilldemo:5181> use hive;
-        |------|----------------------------------|
-        | ok   | summary                          |
-        |------|----------------------------------|
-        | true | Default schema changed to 'hive' |
-        |------|----------------------------------|
-        1 row selected (0.025 seconds)
-
-  2. Issue the SHOW TABLES command to see the existing tables in the schema.
-
-        0: jdbc:drill:zk=drilldemo:5181> show tables;
-        |--------------|------------|
-        | TABLE_SCHEMA | TABLE_NAME |
-        |--------------|------------|
-        | hive.default | orders     |
-        | hive.default | products   |
-        |--------------|------------|
-        2 rows selected (0.438 seconds)
-
-  3. Issue the DESCRIBE command on a table.
-
-        0: jdbc:drill:zk=drilldemo:5181> describe orders;
-        |-------------|-----------|-------------|
-        | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
-        |-------------|-----------|-------------|
-        | order_id    | BIGINT    | YES         |
-        | month       | VARCHAR   | YES         |
-        | purchdate   | TIMESTAMP | YES         |
-        | cust_id     | BIGINT    | YES         |
-        | state       | VARCHAR   | YES         |
-        | prod_id     | BIGINT    | YES         |
-        | order_total | INTEGER   | YES         |
-        |-------------|-----------|-------------|
-        7 rows selected (0.64 seconds)
-
-  4. Issue the DESCRIBE command on a table in another schema from the current schema.
-
-        0: jdbc:drill:zk=drilldemo:5181> describe hbase.customers;
-        |-------------|-----------------------|-------------|
-        | COLUMN_NAME | DATA_TYPE             | IS_NULLABLE |
-        |-------------|-----------------------|-------------|
-        | row_key     | ANY                   | NO          |
-        | address     | (VARCHAR(1), ANY) MAP | NO          |
-        | loyalty     | (VARCHAR(1), ANY) MAP | NO          |
-        | personal    | (VARCHAR(1), ANY) MAP | NO          |
-        |-------------|-----------------------|-------------|
-        4 rows selected (0.671 seconds)
-
-  5. Issue the DESCRIBE command on a view in another schema from the current schema.
-
-        0: jdbc:drill:zk=drilldemo:5181> describe dfs.views.customers_vw;
-        |-------------|-----------|-------------|
-        | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
-        |-------------|-----------|-------------|
-        | cust_id     | BIGINT    | NO          |
-        | name        | VARCHAR   | NO          |
-        | address     | VARCHAR   | NO          |
-        | gender      | VARCHAR   | NO          |
-        | age         | VARCHAR   | NO          |
-        | agg_rev     | VARCHAR   | NO          |
-        | membership  | VARCHAR   | NO          |
-        |-------------|-----------|-------------|
-        7 rows selected (0.403 seconds)  
- 
-6. Issue the DESCRIBE SCHEMA command on `dfs.tmp` (the `dfs` schema and `tmp` workspace configured within the `dfs` schema) from the current schema.  
-  
-        0: jdbc:drill:zk=drilldemo:5181> describe schema dfs.tmp;  
-       
-              {
-                "type" : "file",
-                "enabled" : true,
-                "connection" : "file:///",
-                "config" : null,
-                "formats" : {
-                  "psv" : {
-                    "type" : "text",
-                    "extensions" : [ "tbl" ],
-                    "delimiter" : "|"
-                  },
-                  "csv" : {
-                    "type" : "text",
-                    "extensions" : [ "csv", "bcp" ],
-                    "delimiter" : ","
-                  },
-                 ... 
-                },
-                "location" : "/tmp",
-                "writable" : true,
-                "defaultInputFormat" : null
-              }  
+1. Issue the USE command to switch to a particular schema.
+```
+0: jdbc:drill:zk=drilldemo:5181> use hive;
+|------|----------------------------------|
+| ok   | summary                          |
+|------|----------------------------------|
+| true | Default schema changed to 'hive' |
+|------|----------------------------------|
+1 row selected (0.025 seconds)
+```
+
+2. Issue the SHOW TABLES command to see the existing tables in the schema.
+```
+0: jdbc:drill:zk=drilldemo:5181> show tables;
+|--------------|------------|
+| TABLE_SCHEMA | TABLE_NAME |
+|--------------|------------|
+| hive.default | orders     |
+| hive.default | products   |
+|--------------|------------|
+2 rows selected (0.438 seconds)
+```
+
+3. Issue the DESCRIBE command on a table.
+```
+0: jdbc:drill:zk=drilldemo:5181> describe orders;
+|-------------|-----------|-------------|
+| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+|-------------|-----------|-------------|
+| order_id    | BIGINT    | YES         |
+| month       | VARCHAR   | YES         |
+| purchdate   | TIMESTAMP | YES         |
+| cust_id     | BIGINT    | YES         |
+| state       | VARCHAR   | YES         |
+| prod_id     | BIGINT    | YES         |
+| order_total | INTEGER   | YES         |
+|-------------|-----------|-------------|
+7 rows selected (0.64 seconds)
+```
+
+4. Issue the DESCRIBE command on a table in another schema from the current schema.
+```
+0: jdbc:drill:zk=drilldemo:5181> describe hbase.customers;
+|-------------|-----------------------|-------------|
+| COLUMN_NAME | DATA_TYPE             | IS_NULLABLE |
+|-------------|-----------------------|-------------|
+| row_key     | ANY                   | NO          |
+| address     | (VARCHAR(1), ANY) MAP | NO          |
+| loyalty     | (VARCHAR(1), ANY) MAP | NO          |
+| personal    | (VARCHAR(1), ANY) MAP | NO          |
+|-------------|-----------------------|-------------|
+4 rows selected (0.671 seconds)
+```
+
+5. Issue the DESCRIBE command on a view in another schema from the current schema.
+```
+0: jdbc:drill:zk=drilldemo:5181> describe dfs.views.customers_vw;
+|-------------|-----------|-------------|
+| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+|-------------|-----------|-------------|
+| cust_id     | BIGINT    | NO          |
+| name        | VARCHAR   | NO          |
+| address     | VARCHAR   | NO          |
+| gender      | VARCHAR   | NO          |
+| age         | VARCHAR   | NO          |
+| agg_rev     | VARCHAR   | NO          |
+| membership  | VARCHAR   | NO          |
+|-------------|-----------|-------------|
+7 rows selected (0.403 seconds)  
+```
 
+6. Issue the DESCRIBE SCHEMA command on `dfs.tmp` (the `dfs` schema and `tmp` workspace configured within the `dfs` schema) from the current schema.  
+```  
+0: jdbc:drill:zk=drilldemo:5181> describe schema dfs.tmp;  
+```
+```json
+{
+  "type" : "file",
+  "enabled" : true,
+  "connection" : "file:///",
+  "config" : null,
+  "formats" : {
+    "psv" : {
+      "type" : "text",
+      "extensions" : [ "tbl" ],
+      "delimiter" : "|"
+    },
+    "csv" : {
+      "type" : "text",
+      "extensions" : [ "csv", "bcp" ],
+      "delimiter" : ","
+    },
+   ... 
+  },
+  "location" : "/tmp",
+  "writable" : true,
+  "defaultInputFormat" : null
+}  
+```
 
 
        


[drill] 06/07: Document the ElasticSearch storage plugin.

Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git

commit 27a2a57a5087bdacf83307145da367ce9d7e6254
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Mon Feb 15 14:55:54 2021 +0200

    Document the ElasticSearch storage plugin.
---
 .../plugins/130-elasticsearch.md                   | 55 ++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/_docs/connect-a-data-source/plugins/130-elasticsearch.md b/_docs/connect-a-data-source/plugins/130-elasticsearch.md
new file mode 100644
index 0000000..b9b76aa
--- /dev/null
+++ b/_docs/connect-a-data-source/plugins/130-elasticsearch.md
@@ -0,0 +1,55 @@
+---
+title: "ElasticSearch Plugin"
+slug: "ElasticSearch Plugin"
+parent: "Connect a Data Source"
+---
+
+**Introduced in release:** 1.19
+
+Drill ElasticSearch storage plugin allows you to perform SQL queries against ElasticSearch indices.
+This storage plugin implementation is based on [Apache Calcite adapter for ElasticSearch](https://calcite.apache.org/docs/elasticsearch_adapter.html).
+
+For more details about supported versions please refer to [Supported versions](https://calcite.apache.org/docs/elasticsearch_adapter.html#supported-versions) page.
+
+### Supported optimizations and features
+
+This storage plugin supports the following optimizations:
+
+- Project pushdown
+- Filter pushdown (only expressions supported by Calcite adapter for ElasticSearch. Filter with unsupported expressions 
+  wouldn't be pushed to ElasticSearch but will be produced by Drill)
+- Limit pushdown
+- Aggregation pushdown
+- Sort pushdown
+
+Besides these optimizations, ElasticSearch storage plugin supports the schema provisioning feature.
+For more details please refer to [Specifying the Schema as Table Function Parameter](https://drill.apache.org/docs/plugin-configuration-basics/#specifying-the-schema-as-table-function-parameter).
+
+### Configuration
+
+The plugin can be registered in Apache Drill using the drill web interface by navigating to the `storage` page.
+Following is the default registration configuration.
+
+```json
+{
+  "type": "elastic",
+  "hosts": [
+    "http://localhost:9200"
+  ],
+  "username": null,
+  "password": null,
+  "enabled": false
+}
+```
+
+### Configuration Options
+
+|----------|-----------------------|----------------------------------------------------|
+| Option   | Default               | Description                                        |
+|----------|-----------------------|----------------------------------------------------|
+| type     | (none)                | Set to "elastic" to use this plugin                |
+| hosts    | http://localhost:9200 | List of ElasticSearch hosts to be queried by Drill |
+| username | null                  | ElasticSearch username to be used by Drill         |
+| password | null                  | ElasticSearch password to be used by Drill         |
+|----------|-----------------------|----------------------------------------------------|
+