You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ra...@apache.org on 2018/10/09 15:50:18 UTC
[16/45] carbondata git commit: [CARBONDATA-2818] Upgrade presto
integration version to 0.210
[CARBONDATA-2818] Upgrade presto integration version to 0.210
Upgrade presto integration version to 0.210
This closes #2733
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/8427771f
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/8427771f
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/8427771f
Branch: refs/heads/branch-1.5
Commit: 8427771fc22071099186d3310500d5fcd6c419a6
Parents: 629d625
Author: chenliang613 <ch...@huawei.com>
Authored: Wed Sep 19 08:18:28 2018 +0800
Committer: Raghunandan S <ca...@gmail.com>
Committed: Fri Sep 28 11:39:48 2018 +0530
----------------------------------------------------------------------
docs/quick-start-guide.md | 29 ++++++++++++--------
.../Presto_Cluster_Setup_For_Carbondata.md | 24 ++++++++--------
integration/presto/README.md | 12 +++++---
integration/presto/pom.xml | 4 +--
4 files changed, 40 insertions(+), 29 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/docs/quick-start-guide.md
----------------------------------------------------------------------
diff --git a/docs/quick-start-guide.md b/docs/quick-start-guide.md
index 37c398c..0fdf055 100644
--- a/docs/quick-start-guide.md
+++ b/docs/quick-start-guide.md
@@ -300,24 +300,24 @@ Once the table is created,it can be queried from Presto.**
### Installing Presto
- 1. Download the 0.187 version of Presto using:
- `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.187/presto-server-0.187.tar.gz`
+ 1. Download the 0.210 version of Presto using:
+ `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz`
- 2. Extract Presto tar file: `tar zxvf presto-server-0.187.tar.gz`.
+ 2. Extract Presto tar file: `tar zxvf presto-server-0.210.tar.gz`.
3. Download the Presto CLI for the coordinator and name it presto.
```
- wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.187/presto-cli-0.187-executable.jar
+ wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar
- mv presto-cli-0.187-executable.jar presto
+ mv presto-cli-0.210-executable.jar presto
chmod +x presto
```
### Create Configuration Files
- 1. Create `etc` folder in presto-server-0.187 directory.
+ 1. Create `etc` folder in presto-server-0.210 directory.
2. Create `config.properties`, `jvm.config`, `log.properties`, and `node.properties` files.
3. Install uuid to generate a node.id.
@@ -363,10 +363,15 @@ Once the table is created,it can be queried from Presto.**
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8086
- query.max-memory=50GB
- query.max-memory-per-node=2GB
+ query.max-memory=5GB
+ query.max-total-memory-per-node=5GB
+ query.max-memory-per-node=3GB
+ memory.heap-headroom-per-node=1GB
discovery-server.enabled=true
- discovery.uri=<coordinator_ip>:8086
+ discovery.uri=http://localhost:8086
+ task.max-worker-threads=4
+ optimizer.dictionary-aggregation=true
+ optimizer.optimize-hash-generation = false
```
The options `node-scheduler.include-coordinator=false` and `coordinator=true` indicate that the node is the coordinator and tells the coordinator not to do any of the computation work itself and to use the workers.
@@ -383,7 +388,7 @@ Then, `query.max-memory=<30GB * number of nodes>`.
```
coordinator=false
http-server.http.port=8086
- query.max-memory=50GB
+ query.max-memory=5GB
query.max-memory-per-node=2GB
discovery.uri=<coordinator_ip>:8086
```
@@ -405,12 +410,12 @@ Then, `query.max-memory=<30GB * number of nodes>`.
### Start Presto Server on all nodes
```
-./presto-server-0.187/bin/launcher start
+./presto-server-0.210/bin/launcher start
```
To run it as a background process.
```
-./presto-server-0.187/bin/launcher run
+./presto-server-0.210/bin/launcher run
```
To run it in foreground.
http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/Presto_Cluster_Setup_For_Carbondata.md
----------------------------------------------------------------------
diff --git a/integration/presto/Presto_Cluster_Setup_For_Carbondata.md b/integration/presto/Presto_Cluster_Setup_For_Carbondata.md
index 082b8fe..975e1fa 100644
--- a/integration/presto/Presto_Cluster_Setup_For_Carbondata.md
+++ b/integration/presto/Presto_Cluster_Setup_For_Carbondata.md
@@ -2,24 +2,24 @@
## Installing Presto
- 1. Download the 0.187 version of Presto using:
- `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.187/presto-server-0.187.tar.gz`
+ 1. Download the 0.210 version of Presto using:
+ `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz`
- 2. Extract Presto tar file: `tar zxvf presto-server-0.187.tar.gz`.
+ 2. Extract Presto tar file: `tar zxvf presto-server-0.210.tar.gz`.
3. Download the Presto CLI for the coordinator and name it presto.
```
- wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.187/presto-cli-0.187-executable.jar
+ wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar
- mv presto-cli-0.187-executable.jar presto
+ mv presto-cli-0.210-executable.jar presto
chmod +x presto
```
## Create Configuration Files
- 1. Create `etc` folder in presto-server-0.187 directory.
+ 1. Create `etc` folder in presto-server-0.210 directory.
2. Create `config.properties`, `jvm.config`, `log.properties`, and `node.properties` files.
3. Install uuid to generate a node.id.
@@ -65,8 +65,10 @@
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8086
- query.max-memory=50GB
- query.max-memory-per-node=2GB
+ query.max-memory=5GB
+ query.max-total-memory-per-node=5GB
+ query.max-memory-per-node=3GB
+ memory.heap-headroom-per-node=1GB
discovery-server.enabled=true
discovery.uri=<coordinator_ip>:8086
```
@@ -85,7 +87,7 @@ Then, `query.max-memory=<30GB * number of nodes>`.
```
coordinator=false
http-server.http.port=8086
- query.max-memory=50GB
+ query.max-memory=5GB
query.max-memory-per-node=2GB
discovery.uri=<coordinator_ip>:8086
```
@@ -107,12 +109,12 @@ Then, `query.max-memory=<30GB * number of nodes>`.
## Start Presto Server on all nodes
```
-./presto-server-0.187/bin/launcher start
+./presto-server-0.210/bin/launcher start
```
To run it as a background process.
```
-./presto-server-0.187/bin/launcher run
+./presto-server-0.210/bin/launcher run
```
To run it in foreground.
http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/README.md
----------------------------------------------------------------------
diff --git a/integration/presto/README.md b/integration/presto/README.md
index 5a44f5c..ceab0fe 100644
--- a/integration/presto/README.md
+++ b/integration/presto/README.md
@@ -18,7 +18,7 @@
Please follow the below steps to query carbondata in presto
### Config presto server
-* Download presto server (0.187 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/
+* Download presto server (0.210 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/
* Finish presto configuration following https://prestodb.io/docs/current/installation/deployment.html.
A configuration example:
```
@@ -27,10 +27,14 @@ Please follow the below steps to query carbondata in presto
node-scheduler.include-coordinator=true
http-server.http.port=8086
query.max-memory=5GB
- query.max-memory-per-node=1GB
+ query.max-total-memory-per-node=5GB
+ query.max-memory-per-node=3GB
+ memory.heap-headroom-per-node=1GB
discovery-server.enabled=true
discovery.uri=http://localhost:8086
- reorder-joins=true
+ task.max-worker-threads=4
+ optimizer.dictionary-aggregation=true
+ optimizer.optimize-hash-generation = false
jvm.config:
@@ -131,7 +135,7 @@ Load data statement in Spark can be used to create carbondata tables. And then y
carbondata files.
### Query carbondata in CLI of presto
-* Download presto cli client following: https://prestodb.io/docs/current/installation/cli.html
+* Download presto cli client of version 0.210 : https://repo1.maven.org/maven2/com/facebook/presto/presto-cli
* Start CLI:
http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/pom.xml
----------------------------------------------------------------------
diff --git a/integration/presto/pom.xml b/integration/presto/pom.xml
index 9a9bc55..72847e2 100644
--- a/integration/presto/pom.xml
+++ b/integration/presto/pom.xml
@@ -31,7 +31,7 @@
<packaging>presto-plugin</packaging>
<properties>
- <presto.version>0.208</presto.version>
+ <presto.version>0.210</presto.version>
<dev.path>${basedir}/../../dev</dev.path>
<jacoco.append>true</jacoco.append>
</properties>
@@ -455,7 +455,7 @@
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<scope>test</scope>
- <version>4.5.3</version>
+ <version>4.7.1</version>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>