You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@griffin.apache.org by gu...@apache.org on 2017/10/30 06:24:01 UTC
incubator-griffin git commit: docker compose document and simple guide
Repository: incubator-griffin
Updated Branches:
refs/heads/master c73349354 -> ed063d920
docker compose document and simple guide
Author: Lionel Liu <bh...@163.com>
Closes #149 from bhlx3lyx7/master.
Project: http://git-wip-us.apache.org/repos/asf/incubator-griffin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-griffin/commit/ed063d92
Tree: http://git-wip-us.apache.org/repos/asf/incubator-griffin/tree/ed063d92
Diff: http://git-wip-us.apache.org/repos/asf/incubator-griffin/diff/ed063d92
Branch: refs/heads/master
Commit: ed063d920eb803f586792b286a4af9f83471e2d1
Parents: c733493
Author: Lionel Liu <bh...@163.com>
Authored: Mon Oct 30 14:23:56 2017 +0800
Committer: Lionel Liu <bh...@163.com>
Committed: Mon Oct 30 14:23:56 2017 +0800
----------------------------------------------------------------------
README.md | 22 +++++------
griffin-doc/docker/svc_msr/docker-compose.yml | 25 ++++++++++++
griffin-doc/dsl-guide.md | 2 +-
.../measure/rule/dsl/parser/BasicParser.scala | 41 ++++++++++++++------
4 files changed, 64 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/ed063d92/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 0f23998..872c367 100644
--- a/README.md
+++ b/README.md
@@ -39,27 +39,23 @@ Snapshot:
Release:
### How to run in docker
-1. Install [docker](https://www.docker.com/).
-2. Pull our built docker image.
+1. Install [docker](https://docs.docker.com/engine/installation/) and [docker compose](https://docs.docker.com/compose/install/).
+2. Pull our built docker image and elasticsearch image.
```
- docker pull bhlx3lyx7/griffin_demo:0.0.1
+ docker pull bhlx3lyx7/svc_msr:0.1.6
+ docker pull elasticsearch:5
```
3. Increase vm.max_map_count of your local machine, to use elasticsearch.
```
sysctl -w vm.max_map_count=262144
- ```
-4. Run this docker image, wait for about one minute, then griffin is ready.
```
- docker run -it -h sandbox --name griffin_demo -m 8G --memory-swap -1 \
- -p 32122:2122 -p 37077:7077 -p 36066:6066 -p 38088:8088 -p 38040:8040 \
- -p 33306:3306 -p 39000:9000 -p 38042:8042 -p 38080:8080 -p 37017:27017 \
- -p 39083:9083 -p 38998:8998 -p 39200:9200 bhlx3lyx7/griffin_demo:0.0.1
+4. Copy [docker-compose.yml](https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/svc_msr/docker-compose.yml) to your work path.
+5. In your work path, start docker containers by using docker compose, wait for about one minutes, then griffin service is ready.
```
-5. Now you can visit UI through your browser, login with account "test" and password "test" if required.
+ docker-compose up -d
```
- http://<your local IP address>:38080/
- ```
- You can also follow the steps using UI [here](https://github.com/apache/incubator-griffin/blob/master/griffin-doc/dockerUIguide.md#webui-test-case-guide).
+6. Now you can try griffin APIs by using postman after importing the [json files](https://github.com/apache/incubator-griffin/blob/master/griffin-doc/postman).
+ In which you need to modify the environment `BASE_PATH` value into `<your local IP address>:38080`.
### How to deploy and run at local
1. Install jdk (1.8 or later versions).
http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/ed063d92/griffin-doc/docker/svc_msr/docker-compose.yml
----------------------------------------------------------------------
diff --git a/griffin-doc/docker/svc_msr/docker-compose.yml b/griffin-doc/docker/svc_msr/docker-compose.yml
new file mode 100644
index 0000000..fd017a2
--- /dev/null
+++ b/griffin-doc/docker/svc_msr/docker-compose.yml
@@ -0,0 +1,25 @@
+griffin:
+ image: bhlx3lyx7/svc_msr:0.1.6
+ hostname: griffin
+ links:
+ - es
+ environment:
+ ES_HOSTNAME: es
+ ports:
+ - 32122:2122
+ - 38088:8088
+ - 33306:3306
+ - 38042:8042
+ - 39083:9083
+ - 38998:8998
+ - 38080:8080
+ tty: true
+ container_name: griffin
+
+es:
+ image: elasticsearch:5
+ hostname: es
+ ports:
+ - 39200:9200
+ - 39300:9300
+ container_name: es
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/ed063d92/griffin-doc/dsl-guide.md
----------------------------------------------------------------------
diff --git a/griffin-doc/dsl-guide.md b/griffin-doc/dsl-guide.md
index c668cb9..e7f8569 100644
--- a/griffin-doc/dsl-guide.md
+++ b/griffin-doc/dsl-guide.md
@@ -144,7 +144,7 @@ For example, the dsl rule is `source.cntry, source.id.count(), source.age.max()
After the translation, the metrics will be persisted in table `profiling`.
-## ALternative Rules
+## Alternative Rules
You can simply use Griffin DSL rule to describe your problem in DQ domain, for some complicate requirement, you can also use some alternative rules supported by Griffin.
### Spark sql
http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/ed063d92/measure/src/main/scala/org/apache/griffin/measure/rule/dsl/parser/BasicParser.scala
----------------------------------------------------------------------
diff --git a/measure/src/main/scala/org/apache/griffin/measure/rule/dsl/parser/BasicParser.scala b/measure/src/main/scala/org/apache/griffin/measure/rule/dsl/parser/BasicParser.scala
index 7d9646d..1b7c374 100644
--- a/measure/src/main/scala/org/apache/griffin/measure/rule/dsl/parser/BasicParser.scala
+++ b/measure/src/main/scala/org/apache/griffin/measure/rule/dsl/parser/BasicParser.scala
@@ -48,15 +48,18 @@ trait BasicParser extends JavaTokenParsers with Serializable {
* <literal-nan> ::= nan
*
* -- selection --
- * <selection> ::= <selection-head> [ <field-sel> | <index-sel> | <function-sel> ]*
- * <selection-head> ::= ("data source name registered") | <function>
+ * <selection> ::= <selection-head> [ <field-sel> | <index-sel> | <function-sel> ]* [<as-alias>]?
+ * <selection-head> ::= ("data source name registered") | <function> | <field-name> | <all-selection>
* <field-sel> ::= "." <field-name> | "[" <quote-field-name> "]"
* <index-sel> ::= "[" <arg> "]"
* <function-sel> ::= "." <function-name> "(" [<arg>]? [, <arg>]* ")"
* <arg> ::= <math-expr>
*
+ * -- as alias --
+ * <as-alias> ::= <as> <field-name>
+ *
* -- math expr --
- * <math-factor> ::= <literal> | <alias-expr> | <function> | <selection> | "(" <math-expr> ")"
+ * <math-factor> ::= <literal> | <function> | <selection> | "(" <math-expr> ")" [<as-alias>]?
* <unary-math-expr> ::= [<unary-opr>]* <math-factor>
* <binary-math-expr> ::= <unary-math-expr> [<binary-opr> <unary-math-expr>]+
* <math-expr> ::= <binary-math-expr>
@@ -69,7 +72,7 @@ trait BasicParser extends JavaTokenParsers with Serializable {
* <is-null-expr> ::= <math-expr> <is> [<not>]? <null>
* <is-nan-expr> ::= <math-expr> <is> [<not>]? <nan>
*
- * <logical-factor> ::= <math-expr> | <in-expr> | <between-expr> | <like-expr> | <is-null-expr> | <is-nan-expr> | "(" <logical-expr> ")"
+ * <logical-factor> ::= <math-expr> | <in-expr> | <between-expr> | <like-expr> | <is-null-expr> | <is-nan-expr> | "(" <logical-expr> ")" [<as-alias>]?
* <unary-logical-expr> ::= [<unary-logical-opr>]* <logical-factor>
* <binary-logical-expr> ::= <unary-logical-expr> [<binary-logical-opr> <unary-logical-expr>]+
* <logical-expr> ::= <binary-logical-expr>
@@ -78,12 +81,22 @@ trait BasicParser extends JavaTokenParsers with Serializable {
* <expr> = <math-expr> | <logical-expr>
*
* -- function expr --
- * <function> ::= <function-name> "(" [<arg>] [, <arg>]+ ")"
+ * <function> ::= <function-name> "(" [<arg>] [, <arg>]+ ")" [<as-alias>]?
* <function-name> ::= ("function name registered")
* <arg> ::= <expr>
*
- * -- alias expr --
- * <alias-expr> = <expr> <as> <name>
+ * -- clauses --
+ * <select-clause> = <expr> [, <expr>]*
+ * <where-clause> = <where> <expr>
+ * <from-clause> = <from> ("data source name registered")
+ * <having-clause> = <having> <expr>
+ * <groupby-clause> = <group> <by> <expr> [ <having-clause> ]?
+ * <orderby-item> = <expr> [ <DESC> ]?
+ * <orderby-clause> = <order> <by> <orderby-item> [ , <orderby-item> ]*
+ * <limit-clause> = <limit> <expr>
+ *
+ * -- combined clauses --
+ * <combined-clauses> = <select-clause> [ <from-clause> ]+ [ <where-clause> ]+ [ <groupby-clause> ]+ [ <orderby-clause> ]+ [ <limit-clause> ]+
*/
protected def genDataSourceNamesParser(names: Seq[String]): Parser[String] = {
@@ -181,8 +194,8 @@ trait BasicParser extends JavaTokenParsers with Serializable {
/**
* -- selection --
- * <selection> ::= <selection-head> [ <field-sel> | <index-sel> | <function-sel> ]*
- * <selection-head> ::= ("data source name registered") | <function>
+ * <selection> ::= <selection-head> [ <field-sel> | <index-sel> | <function-sel> ]* [<as-alias>]?
+ * <selection-head> ::= ("data source name registered") | <function> | <field-name> | <all-selection>
* <field-sel> ::= "." <field-name> | "[" <quote-field-name> "]"
* <index-sel> ::= "[" <arg> "]"
* <function-sel> ::= "." <function-name> "(" [<arg>]? [, <arg>]* ")"
@@ -209,11 +222,15 @@ trait BasicParser extends JavaTokenParsers with Serializable {
case _ ~ name ~ _ ~ args ~ _ => FunctionSelectExpr(name, args)
}
+ /**
+ * -- as alias --
+ * <as-alias> ::= <as> <field-name>
+ */
def asAlias: Parser[String] = AS ~> FieldName
/**
* -- math expr --
- * <math-factor> ::= <literal> | <alias-expr> | <function> | <selection> | "(" <math-expr> ")"
+ * <math-factor> ::= <literal> | <function> | <selection> | "(" <math-expr> ")" [<as-alias>]?
* <unary-math-expr> ::= [<unary-opr>]* <math-factor>
* <binary-math-expr> ::= <unary-math-expr> [<binary-opr> <unary-math-expr>]+
* <math-expr> ::= <binary-math-expr>
@@ -248,7 +265,7 @@ trait BasicParser extends JavaTokenParsers with Serializable {
* <is-null-expr> ::= <math-expr> <is> [<not>]? <null>
* <is-nan-expr> ::= <math-expr> <is> [<not>]? <nan>
*
- * <logical-factor> ::= <math-expr> | <in-expr> | <between-expr> | <like-expr> | <is-null-expr> | <is-nan-expr> | "(" <logical-expr> ")"
+ * <logical-factor> ::= <math-expr> | <in-expr> | <between-expr> | <like-expr> | <is-null-expr> | <is-nan-expr> | "(" <logical-expr> ")" [<as-alias>]?
* <unary-logical-expr> ::= [<unary-logical-opr>]* <logical-factor>
* <binary-logical-expr> ::= <unary-logical-expr> [<binary-logical-opr> <unary-logical-expr>]+
* <logical-expr> ::= <binary-logical-expr>
@@ -301,7 +318,7 @@ trait BasicParser extends JavaTokenParsers with Serializable {
/**
* -- function expr --
- * <function> ::= <function-name> "(" [<arg>] [, <arg>]+ ")"
+ * <function> ::= <function-name> "(" [<arg>] [, <arg>]+ ")" [<as-alias>]?
* <function-name> ::= ("function name registered")
* <arg> ::= <expr>
*/