You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org> on 2016/07/10 09:47:41 UTC

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

abdullah alamoudi has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/988

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................

ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
---
M asterixdb/asterix-app/pom.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
8 files changed, 253 insertions(+), 74 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/88/988/1

diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml
index 43adabf..7d0a9ba 100644
--- a/asterixdb/asterix-app/pom.xml
+++ b/asterixdb/asterix-app/pom.xml
@@ -145,6 +145,35 @@
         </executions>
       </plugin>
     </plugins>
+    <pluginManagement>
+        <plugins>
+            <!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build itself.-->
+            <plugin>
+                <groupId>org.eclipse.m2e</groupId>
+                <artifactId>lifecycle-mapping</artifactId>
+                <version>1.0.0</version>
+                <configuration>
+                    <lifecycleMappingMetadata>
+                        <pluginExecutions>
+                            <pluginExecution>
+                                <pluginExecutionFilter>
+                                    <groupId>org.apache.asterix</groupId>
+                                    <artifactId>asterix-test-datagenerator-maven-plugin</artifactId>
+                                    <versionRange>[0.8.9-SNAPSHOT,)</versionRange>
+                                    <goals>
+                                        <goal>generate-testdata</goal>
+                                    </goals>
+                                </pluginExecutionFilter>
+                                <action>
+                                    <ignore></ignore>
+                                </action>
+                            </pluginExecution>
+                        </pluginExecutions>
+                    </lifecycleMappingMetadata>
+                </configuration>
+            </plugin>
+        </plugins>
+    </pluginManagement>
   </build>
   <dependencies>
     <dependency>
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
new file mode 100644
index 0000000..9fdd9be
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create function printName() { 'AsterixDB Shared nothing parallel BDMS' };
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
new file mode 100644
index 0000000..087788a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+for $x in dataset Metadata.Function
+where $x.DataverseName = "test"
+return $x;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
new file mode 100644
index 0000000..3e20182
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop dataverse test;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
new file mode 100644
index 0000000..5ba00ae
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
@@ -0,0 +1 @@
+{ "DataverseName": "test", "Name": "printName", "Arity": "0", "Params": [  ], "ReturnType": "VOID", "Definition": "'AsterixDB Shared nothing parallel BDMS'", "Language": "AQL", "Kind": "SCALAR" }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
index f7a9b6d..ffa4346 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
@@ -6118,6 +6118,11 @@
   </test-group>
   <test-group name="user-defined-functions">
     <test-case FilePath="user-defined-functions">
+      <compilation-unit name="single-line-definition">
+        <output-dir compare="Text">single-line-definition</output-dir>
+      </compilation-unit>
+    </test-case>
+    <test-case FilePath="user-defined-functions">
       <compilation-unit name="query-issue218-2">
         <output-dir compare="Text">query-issue218-2</output-dir>
       </compilation-unit>
diff --git a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
index b96fb26..e4da011 100644
--- a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
+++ b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
@@ -157,9 +157,7 @@
     private static final String SKIP_SECONDARY_INDEX_SEARCH_HINT = "skip-index";
     private static final String VAL_FILE_HINT = "val-files";
     private static final String VAL_FILE_SAME_INDEX_HINT = "val-file-same-idx";
-
     private static final String GEN_FIELDS_HINT = "gen-fields";
-
     // data generator hints
     private static final String DGEN_HINT = "dgen";
 
@@ -314,7 +312,7 @@
   String dvName = null;
 }
 {
-  "use" "dataverse" dvName = Identifier()
+  <USE> <DATAVERSE> dvName = Identifier()
     {
       defaultDataverse = dvName;
       return new DataverseDecl(new Identifier(dvName));
@@ -328,7 +326,7 @@
   Statement stmt = null;
 }
 {
-  "create"
+  <CREATE>
   (
     {
       hint = getHint(token);
@@ -357,8 +355,8 @@
   TypeExpression typeExpr = null;
 }
 {
-  "type" nameComponents = TypeName() ifNotExists = IfNotExists()
-  "as" typeExpr = TypeExpr()
+  <TYPE> nameComponents = TypeName() ifNotExists = IfNotExists()
+  <AS> typeExpr = TypeExpr()
     {
       long numValues = -1;
       String filename = null;
@@ -384,8 +382,8 @@
   List<Identifier>ncNames = null;
 }
 {
-  "nodegroup" name = Identifier()
-  ifNotExists = IfNotExists() "on" tmp = Identifier()
+  <NODEGROUP> name = Identifier()
+  ifNotExists = IfNotExists() <ON> tmp = Identifier()
     {
       ncNames = new ArrayList<Identifier>();
       ncNames.add(new Identifier(tmp));
@@ -421,13 +419,13 @@
 }
 {
   (
-    "external" <DATASET> nameComponents = QualifiedName()
+    <EXTERNAL> <DATASET> nameComponents = QualifiedName()
     <LEFTPAREN> typeComponents = TypeName() <RIGHTPAREN>
     ifNotExists = IfNotExists()
-    "using" adapterName = AdapterName() properties = Configuration()
-    ("on" nodeGroupName = Identifier() )?
-    ( "hints" hints = Properties() )?
-    ( "using" "compaction" "policy" compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
+    <USING> adapterName = AdapterName() properties = Configuration()
+    (<ON> nodeGroupName = Identifier() )?
+    ( <HINTS> hints = Properties() )?
+    ( <USING> <COMPACTION> <POLICY> compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
       {
         ExternalDetailsDecl edd = new ExternalDetailsDecl();
         edd.setAdapter(adapterName);
@@ -447,7 +445,7 @@
                                    ifNotExists);
       }
 
-    | ("internal" | "temporary" {
+    | (<INTERNAL> | <TEMPORARY> {
             temp = token.image.toLowerCase().equals("temporary");
         }
       )?
@@ -466,11 +464,11 @@
     )?
     ifNotExists = IfNotExists()
     primaryKeyFields = PrimaryKey()
-    ("autogenerated" { autogenerated = true; } )?
-    ("on" nodeGroupName = Identifier() )?
-    ( "hints" hints = Properties() )?
-    ( "using" "compaction" "policy" compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
-    ( "with filter on" filterField = NestedField() )?
+    (<AUTOGENERATED> { autogenerated = true; } )?
+    (<ON> nodeGroupName = Identifier() )?
+    ( <HINTS> hints = Properties() )?
+    ( <USING> <COMPACTION> <POLICY> compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
+    ( <WITH> <FILTER> <ON> filterField = NestedField() )?
       {
         if(filterField!=null && filterField.first!=0){
           throw new ParseException("A filter field can only be a field in the main record of the dataset.");
@@ -507,7 +505,7 @@
   String datasetName = null;
 }
 {
-    "refresh external" <DATASET> nameComponents = QualifiedName()
+    <REFRESH> <EXTERNAL> <DATASET> nameComponents = QualifiedName()
     {
     redss.setDataverseName(nameComponents.first);
     redss.setDatasetName(nameComponents.second);
@@ -524,13 +522,13 @@
   Pair<Identifier,Identifier> nameComponentsTo = null;
 }
 {
-  "run" system = Identifier()<LEFTPAREN> ( tmp = Identifier() [<COMMA>]
+  <RUN> system = Identifier()<LEFTPAREN> ( tmp = Identifier() [<COMMA>]
     {
       parameters.add(tmp);
     }
   )*<RIGHTPAREN>
   <FROM> <DATASET> nameComponentsFrom  = QualifiedName()
-  "to" <DATASET> nameComponentsTo  = QualifiedName()
+  <TO> <DATASET> nameComponentsTo  = QualifiedName()
     {
       return new RunStatement(system, parameters, nameComponentsFrom.first, nameComponentsFrom.second, nameComponentsTo.first, nameComponentsTo.second);
     }
@@ -547,9 +545,9 @@
   boolean enforced = false;
 }
 {
-  "index" indexName = Identifier()
+  <INDEX> indexName = Identifier()
   ifNotExists = IfNotExists()
-  "on" nameComponents = QualifiedName()
+  <ON> nameComponents = QualifiedName()
   <LEFTPAREN> ( fieldPair = OpenField()
     {
        cis.addFieldExprPair(fieldPair.second);
@@ -560,7 +558,7 @@
        cis.addFieldExprPair(fieldPair.second);
        cis.addFieldIndexIndicator(fieldPair.first);
     }
-  )* <RIGHTPAREN> ( "type" indexType = IndexType() )? ( "enforced" { enforced = true; } )?
+  )* <RIGHTPAREN> ( <TYPE> indexType = IndexType() )? ( <ENFORCED> { enforced = true; } )?
     {
       cis.setIndexName(new Identifier(indexName));
       cis.setIfNotExists(ifNotExists);
@@ -603,19 +601,19 @@
   int gramLength = 0;
 }
 {
-  ("btree"
+  (<BTREE>
     {
       type = IndexType.BTREE;
     }
-  | "rtree"
+  | <RTREE>
     {
       type = IndexType.RTREE;
     }
-  | "keyword"
+  | <KEYWORD>
     {
       type = IndexType.LENGTH_PARTITIONED_WORD_INVIX;
     }
-  | "ngram" <LEFTPAREN> <INTEGER_LITERAL>
+  | <NGRAM> <LEFTPAREN> <INTEGER_LITERAL>
     {
       type = IndexType.LENGTH_PARTITIONED_NGRAM_INVIX;
       gramLength = Integer.valueOf(token.image);
@@ -633,9 +631,9 @@
   String format = null;
 }
 {
-  "dataverse" dvName = Identifier()
+  <DATAVERSE> dvName = Identifier()
   ifNotExists = IfNotExists()
-  ( "with format" format = StringLiteral() )?
+  ( <WITH> <FORMAT> format = StringLiteral() )?
     {
       return new CreateDataverseStatement(new Identifier(dvName), format, ifNotExists);
     }
@@ -656,7 +654,7 @@
   createNewScope();
 }
 {
-  "function" fctName = FunctionName()
+  <FUNCTION>fctName = FunctionName()
   ifNotExists = IfNotExists()
   paramList = ParameterList()
   <LEFTBRACE>
@@ -688,15 +686,15 @@
 }
 {
   (
-    "secondary" "feed"  nameComponents = QualifiedName() ifNotExists = IfNotExists()
-      <FROM> "feed" sourceNameComponents = QualifiedName() (appliedFunction = ApplyFunction())?
+    <SECONDARY> <FEED>  nameComponents = QualifiedName() ifNotExists = IfNotExists()
+      <FROM> <FEED> sourceNameComponents = QualifiedName() (appliedFunction = ApplyFunction())?
       {
         cfs = new CreateSecondaryFeedStatement(nameComponents,
                                    sourceNameComponents, appliedFunction, ifNotExists);
       }
      |
-     ("primary")? "feed" nameComponents = QualifiedName() ifNotExists = IfNotExists()
-      "using" adapterName = AdapterName() properties = Configuration() (appliedFunction = ApplyFunction())?
+     (<PRIMARY>)? <FEED> nameComponents = QualifiedName() ifNotExists = IfNotExists()
+      <USING> adapterName = AdapterName() properties = Configuration() (appliedFunction = ApplyFunction())?
        {
         cfs = new CreatePrimaryFeedStatement(nameComponents,
                                     adapterName, properties, appliedFunction, ifNotExists);
@@ -719,14 +717,14 @@
 }
 {
   (
-    "ingestion" "policy"  policyName = Identifier() ifNotExists = IfNotExists()
+    <INGESTION> <POLICY>  policyName = Identifier() ifNotExists = IfNotExists()
       <FROM>
-      ("policy" basePolicyName = Identifier() properties = Configuration() ("definition" definition = StringLiteral())?
+      (<POLICY> basePolicyName = Identifier() properties = Configuration() (<DEFINITION> definition = StringLiteral())?
       {
         cfps = new CreateFeedPolicyStatement(policyName,
                                    basePolicyName, properties, definition, ifNotExists);
       }
-     | "path" sourcePolicyFile = Identifier() ("definition" definition = StringLiteral())?
+     | <PATH> sourcePolicyFile = Identifier() (<DEFINITION> definition = StringLiteral())?
        {
         cfps = new CreateFeedPolicyStatement(policyName, sourcePolicyFile, definition, ifNotExists);
        }
@@ -770,7 +768,7 @@
 {
 }
 {
-  ( "if not exists"
+  ( <IF> <NOT> <EXISTS>
     {
       return true;
     }
@@ -786,7 +784,7 @@
   FunctionSignature funcSig = null;
 }
 {
-  "apply" "function" functioName = FunctionName()
+  <APPLY><FUNCTION>functioName = FunctionName()
     {
        String fqFunctionName = functioName.library == null ? functioName.function : functioName.library + "#" + functioName.function;
        return new FunctionSignature(functioName.dataverse, fqFunctionName, 1);
@@ -798,7 +796,7 @@
   String policy = null;
 }
 {
-   "using" "policy" policy = Identifier()
+   <USING> <POLICY> policy = Identifier()
    {
      return policy;
    }
@@ -811,7 +809,7 @@
   int arity = 0;
 }
 {
-  fctName = FunctionName() "@" <INTEGER_LITERAL>
+  fctName = FunctionName() <SYMBOLAT><INTEGER_LITERAL>
     {
       arity = new Integer(token.image);
       if (arity < 0 && arity != FunctionIdentifier.VARARGS) {
@@ -831,7 +829,7 @@
    List<List<String>> primaryKeyFields = new ArrayList<List<String>>();
 }
 {
-  "primary" "key" tmp = NestedField()
+  <PRIMARY> <KEY> tmp = NestedField()
     {
       keyFieldSourceIndicators.add(tmp.first);
       primaryKeyFields.add(tmp.second);
@@ -857,33 +855,33 @@
   Statement stmt = null;
 }
 {
-  "drop"
+  <DROP>
   (
     <DATASET> pairId = QualifiedName() ifExists = IfExists()
       {
         stmt = new DropStatement(pairId.first, pairId.second, ifExists);
       }
-    | "index" tripleId = DoubleQualifiedName() ifExists = IfExists()
+    | <INDEX> tripleId = DoubleQualifiedName() ifExists = IfExists()
       {
         stmt = new IndexDropStatement(tripleId.first, tripleId.second, tripleId.third, ifExists);
       }
-    | "nodegroup" id = Identifier() ifExists = IfExists()
+    | <NODEGROUP> id = Identifier() ifExists = IfExists()
       {
         stmt = new NodeGroupDropStatement(new Identifier(id), ifExists);
       }
-    | "type" pairId = TypeName() ifExists = IfExists()
+    | <TYPE> pairId = TypeName() ifExists = IfExists()
       {
         stmt = new TypeDropStatement(pairId.first, pairId.second, ifExists);
       }
-    | "dataverse" id = Identifier() ifExists = IfExists()
+    | <DATAVERSE> id = Identifier() ifExists = IfExists()
       {
         stmt = new DataverseDropStatement(new Identifier(id), ifExists);
       }
-    | "function" funcSig = FunctionSignature() ifExists = IfExists()
+    | <FUNCTION>funcSig = FunctionSignature() ifExists = IfExists()
       {
         stmt = new FunctionDropStatement(funcSig, ifExists);
       }
-    | "feed" pairId = QualifiedName() ifExists = IfExists()
+    | <FEED> pairId = QualifiedName() ifExists = IfExists()
       {
         stmt = new FeedDropStatement(pairId.first, pairId.second, ifExists);
       }
@@ -897,7 +895,7 @@
 {
 }
 {
-  ( <IF> "exists"
+  ( <IF> <EXISTS>
     {
       return true;
     }
@@ -914,7 +912,7 @@
   boolean upsert = false;
 }
 {
-  ("insert"|"upsert"{ upsert = true; }) "into" <DATASET> nameComponents = QualifiedName() query = Query()
+  (<INSERT>|<UPSERT>{ upsert = true; }) <INTO> <DATASET> nameComponents = QualifiedName() query = Query()
     {
       query.setTopLevel(true);
       if(upsert){
@@ -936,7 +934,7 @@
 
 }
 {
-  "delete" var = Variable()
+  <DELETE> var = Variable()
     {
       getCurrentScope().addNewVarSymbolToScope(var.getVar());
     }
@@ -963,7 +961,7 @@
   List<UpdateClause> ucs = new ArrayList<UpdateClause>();
 }
 {
-  "update" vars = Variable() <IN> target = Expression()
+  <UPDATE> vars = Variable() <IN> target = Expression()
   <WHERE> condition = Expression()
   <LEFTPAREN> (uc = UpdateClause()
     {
@@ -991,7 +989,7 @@
   UpdateClause elsebranch = null;
 }
 {
-   ("set" target = Expression() <ASSIGN> value = Expression()
+   (<SET> target = Expression() <ASSIGN> value = Expression()
    | is = InsertStatement()
    | ds = DeleteStatement()
    | us = UpdateStatement()
@@ -1010,7 +1008,7 @@
   String pv = null;
 }
 {
-  "set" pn = Identifier() pv = StringLiteral()
+  <SET> pn = Identifier() pv = StringLiteral()
     {
       return new SetStatement(pn, pv);
     }
@@ -1025,8 +1023,8 @@
   Pair<Identifier,Identifier> nameComponents = null;
 }
 {
-  "write" "output" "to" nodeName = Identifier() <COLON> fileName = StringLiteral()
-    ( "using" writerClass = StringLiteral() )?
+  <WRITE> <OUTPUT> <TO> nodeName = Identifier() <COLON> fileName = StringLiteral()
+    ( <USING> writerClass = StringLiteral() )?
     {
       return new WriteStatement(new Identifier(nodeName), fileName, writerClass);
     }
@@ -1042,13 +1040,13 @@
   Pair<Identifier,Identifier> nameComponents = null;
 }
 {
-  "load" <DATASET> nameComponents = QualifiedName()
+  <LOAD> <DATASET> nameComponents = QualifiedName()
     {
       dataverseName = nameComponents.first;
       datasetName = nameComponents.second;
     }
-  "using" adapterName = AdapterName() properties = Configuration()
-  ("pre-sorted"
+  <USING> adapterName = AdapterName() properties = Configuration()
+  (<PRESORTED>
     {
       alreadySorted = true;
     }
@@ -1076,7 +1074,7 @@
   Statement stmt = null;
 }
 {
-  "compact" <DATASET> nameComponents = QualifiedName()
+  <COMPACT> <DATASET> nameComponents = QualifiedName()
     {
       stmt = new CompactStatement(nameComponents.first, nameComponents.second);
     }
@@ -1096,11 +1094,11 @@
 }
 {
   (
-    "connect" "feed" feedNameComponents = QualifiedName() "to" <DATASET> datasetNameComponents = QualifiedName() (policy = GetPolicy())?
+    <CONNECT> <FEED> feedNameComponents = QualifiedName() <TO> <DATASET> datasetNameComponents = QualifiedName() (policy = GetPolicy())?
       {
         stmt = new ConnectFeedStatement(feedNameComponents, datasetNameComponents, policy, getVarCounter());
       }
-    | "disconnect" "feed" feedNameComponents = QualifiedName() <FROM> <DATASET> datasetNameComponents = QualifiedName()
+    | <DISCONNECT> <FEED> feedNameComponents = QualifiedName() <FROM> <DATASET> datasetNameComponents = QualifiedName()
       {
         stmt = new DisconnectFeedStatement(feedNameComponents, datasetNameComponents);
       }
@@ -1219,8 +1217,8 @@
   RecordTypeDefinition.RecordKind recordKind = null;
 }
 {
-  ( "closed" { recordKind = RecordTypeDefinition.RecordKind.CLOSED; }
-    | "open" { recordKind = RecordTypeDefinition.RecordKind.OPEN; } )?
+  ( <CLOSED>{ recordKind = RecordTypeDefinition.RecordKind.CLOSED; }
+    | <OPEN>{ recordKind = RecordTypeDefinition.RecordKind.OPEN; } )?
    <LEFTBRACE>
     {
       String hint = getHint(token);
@@ -1331,7 +1329,7 @@
     {
       secondAfterDot = true;
     }
-  ("#" third = Identifier())? | "#" second = Identifier() )?
+  (<SYMBOLHASH> third = Identifier())? | <SYMBOLHASH> second = Identifier() )?
     {
       if (second == null) {
         result.dataverse = defaultDataverse;
@@ -1380,7 +1378,7 @@
   String lit = null;
 }
 {
-  (<IDENTIFIER>
+  ((<IDENTIFIER>|<NOT>)
     {
       return token.image;
     }
@@ -1510,7 +1508,7 @@
   createNewScope();
 }
 {
-  "declare" "function" functionName = Identifier()
+  <DECLARE><FUNCTION>functionName = Identifier()
   paramList = ParameterList()
   <LEFTBRACE> funcBody = Expression() <RIGHTBRACE>
     {
@@ -2518,33 +2516,87 @@
 <DEFAULT,IN_DBL_BRACE>
 TOKEN :
 {
-    <ASC : "asc">
+    <APPLY : "apply">
+  | <AS : "as">
+  | <ASC : "asc">
   | <AT : "at">
+  | <AUTOGENERATED : "autogenerated">
+  | <BTREE : "btree">
   | <BY : "by">
+  | <CLOSED : "closed">
+  | <COMPACT : "compact">
+  | <COMPACTION : "compaction">
+  | <CONNECT : "connect">
+  | <CREATE : "create">
   | <DATASET : "dataset">
+  | <DATAVERSE : "dataverse">
+  | <DECLARE : "declare">
   | <DECOR : "decor">
+  | <DEFINITION : "definition">
+  | <DELETE : "delete">
   | <DESC : "desc">
+  | <DISCONNECT : "disconnect">
   | <DISTINCT : "distinct">
+  | <DROP : "drop">
   | <ELSE : "else">
+  | <ENFORCED : "enforced">
   | <EVERY : "every">
+  | <EXISTS : "exists">
+  | <EXTERNAL : "external">
+  | <FEED : "feed">
+  | <FILTER : "filter">
   | <FOR : "for">
+  | <FORMAT : "format">
   | <FROM : "from">
+  | <FUNCTION : "function">
   | <GROUP : "group">
+  | <HINTS : "hints">
   | <IF : "if">
   | <IN : "in">
+  | <INDEX : "index">
+  | <INGESTION : "ingestion">
+  | <INSERT : "insert">
+  | <INTERNAL : "internal">
+  | <INTO : "into">
+  | <KEY : "key">
+  | <KEYWORD : "keyword">
+  | <KEEPING : "keeping">
   | <LET : "let">
   | <LIMIT : "limit">
+  | <LOAD : "load">
+  | <NGRAM : "ngram">
+  | <NODEGROUP : "nodegroup">
+  | <NOT : "not">
   | <OFFSET : "offset">
+  | <ON : "on">
+  | <OPEN : "open">
   | <ORDER : "order">
+  | <OUTPUT : "output">
+  | <PATH : "path">
+  | <POLICY : "policy">
+  | <PRESORTED : "pre-sorted">
+  | <PRIMARY : "primary">
+  | <REFRESH : "refresh">
   | <RETURN : "return">
+  | <RTREE : "rtree">
+  | <RUN : "run">
   | <SATISFIES : "satisfies">
+  | <SECONDARY : "secondary">
   | <SELECT : "select">
+  | <SET : "set">
   | <SOME : "some">
+  | <TEMPORARY : "temporary">
   | <THEN : "then">
+  | <TO : "to">
+  | <TYPE : "type">
   | <UNION : "union">
+  | <UPDATE : "update">
+  | <UPSERT : "upsert">
+  | <USE : "use">
+  | <USING : "using">
   | <WHERE : "where">
   | <WITH : "with">
-  | <KEEPING : "keeping">
+  | <WRITE : "write">
 }
 
 <DEFAULT,IN_DBL_BRACE>
@@ -2579,6 +2631,9 @@
 
   | <AND : "and">
   | <OR : "or">
+  
+  | <SYMBOLAT : "@">
+  | <SYMBOLHASH : "#">
 }
 
 <DEFAULT,IN_DBL_BRACE>
diff --git a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
index 6ab82c1..dc0fa93 100644
--- a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
+++ b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
@@ -222,8 +222,11 @@
 
     public String extractFragment(int beginLine, int beginColumn, int endLine, int endColumn) {
         StringBuilder extract = new StringBuilder();
-        extract.append(inputLines[beginLine - 1].trim().length() > 1
-                ? inputLines[beginLine - 1].trim().substring(beginColumn) : "");
+        if (beginLine == endLine) {
+            // special case that we need to handle separately
+            return inputLines[beginLine - 1].substring(beginColumn, endColumn - 1).trim();
+        }
+        extract.append(inputLines[beginLine - 1].substring(beginColumn));
         for (int i = beginLine + 1; i < endLine; i++) {
             extract.append("\n");
             extract.append(inputLines[i - 1]);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 4:

Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/64/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 3:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1857/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2:

(10 comments)

Found a few more spaces, but those are not important. 
The question that confuses me (and that I didn't see last time) is why <NOT> is an identifier?
What's the explanation for this?

https://asterix-gerrit.ics.uci.edu/#/c/988/2/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj:

Line 426:     (<ON> nodeGroupName = Identifier() )?
space missing


Line 467:     (<AUTOGENERATED> { autogenerated = true; } )?
space missing


Line 468:     (<ON> nodeGroupName = Identifier() )?
space missing


Line 604:   (<BTREE>
space missing


Line 657:   <FUNCTION>fctName = FunctionName()
space missing


Line 689:     <SECONDARY> <FEED>  nameComponents = QualifiedName() ifNotExists = IfNotExists()
2 spaces


Line 720:     <INGESTION> <POLICY>  policyName = Identifier() ifNotExists = IfNotExists()
2 spaces


Line 812:   fctName = FunctionName() <SYMBOLAT><INTEGER_LITERAL>
space missing


Line 1332:   (<SYMBOLHASH> third = Identifier())? | <SYMBOLHASH> second = Identifier() )?
space missing


Line 1381:   ((<IDENTIFIER>|<NOT>)
Why is NOT an identifier?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/988

to look at the new patch set (#4).

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................

ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
---
M asterixdb/asterix-app/pom.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
12 files changed, 371 insertions(+), 100 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/88/988/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/59/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 1:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/988/1/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj:

Line 1511:   <DECLARE><FUNCTION>functionName = Identifier()
> spaces between the tokens?
Done


Line 2634:   
> WS
aaaaaaaaah

Done.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/988

to look at the new patch set (#3).

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................

ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
---
M asterixdb/asterix-app/pom.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
8 files changed, 278 insertions(+), 100 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/88/988/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 1:

Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/59/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 5: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/66/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
abdullah alamoudi has submitted this change and it was merged.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Reviewed-on: https://asterix-gerrit.ics.uci.edu/988
Sonar-Qube: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Till Westmann <ti...@apache.org>
---
M asterixdb/asterix-app/pom.xml
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
27 files changed, 396 insertions(+), 123 deletions(-)

Approvals:
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml
index 43adabf..7d0a9ba 100644
--- a/asterixdb/asterix-app/pom.xml
+++ b/asterixdb/asterix-app/pom.xml
@@ -145,6 +145,35 @@
         </executions>
       </plugin>
     </plugins>
+    <pluginManagement>
+        <plugins>
+            <!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build itself.-->
+            <plugin>
+                <groupId>org.eclipse.m2e</groupId>
+                <artifactId>lifecycle-mapping</artifactId>
+                <version>1.0.0</version>
+                <configuration>
+                    <lifecycleMappingMetadata>
+                        <pluginExecutions>
+                            <pluginExecution>
+                                <pluginExecutionFilter>
+                                    <groupId>org.apache.asterix</groupId>
+                                    <artifactId>asterix-test-datagenerator-maven-plugin</artifactId>
+                                    <versionRange>[0.8.9-SNAPSHOT,)</versionRange>
+                                    <goals>
+                                        <goal>generate-testdata</goal>
+                                    </goals>
+                                </pluginExecutionFilter>
+                                <action>
+                                    <ignore></ignore>
+                                </action>
+                            </pluginExecution>
+                        </pluginExecutions>
+                    </lifecycleMappingMetadata>
+                </configuration>
+            </plugin>
+        </plugins>
+    </pluginManagement>
   </build>
   <dependencies>
     <dependency>
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
index bd252ce..75c9578 100644
--- a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
@@ -41,5 +41,5 @@
 group by $aid := $i.key1 with $i
 return {
   "gid": $aid,
-  "avg": avg(for $j in $i where not(is-null($j.value)) return $j.value)
+  "avg": avg(for $j in $i where "not"(is-null($j.value)) return $j.value)
 }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql
index a330dea..e6115bd 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql
@@ -27,5 +27,5 @@
 set import-private-functions 'true';
 
 sql-sum(for $l in dataset('tdst')
-where not(is-null($l.sal))
+where "not"(is-null($l.sal))
 return $l.sal)
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql
index 2d62ed6..5c8570a 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql
@@ -27,5 +27,5 @@
 set import-private-functions 'true';
 
 sum(for $l in dataset('tdst')
-where not(is-null($l.sal))
+where "not"(is-null($l.sal))
 return $l.sal)
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql
index 710bceb..567e8b6 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql
@@ -22,4 +22,6 @@
 let $x := true
 let $y := false
 let $z := null
-return {"not_x": not($x), "not_y": not($y), "not_z": not($z)}
+return {"not_x": "not"($x), "not_y": "not"($y), "not_z": "not"($z)}
+
+
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql
index 8fd8d46..e4bb3fe 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql
@@ -29,10 +29,10 @@
 every $x in [true,false] satisfies $x,
 every $x in [false,true] satisfies $x,
 every $x in [true,true] satisfies $x,
-every $x in [false,false] satisfies not($x),
-every $x in [true,false] satisfies not($x),
-every $x in [false,true] satisfies not($x),
-every $x in [true,true] satisfies not($x)
+every $x in [false,false] satisfies "not"($x),
+every $x in [true,false] satisfies "not"($x),
+every $x in [false,true] satisfies "not"($x),
+every $x in [true,true] satisfies "not"($x)
 ]
 for $i in $x
 return $i
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql
index 41799a9..5f4b54b 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql
@@ -29,10 +29,10 @@
 some $x in [true,false] satisfies $x,
 some $x in [false,true] satisfies $x,
 some $x in [true,true] satisfies $x,
-some $x in [false,false] satisfies not($x),
-some $x in [true,false] satisfies not($x),
-some $x in [false,true] satisfies not($x),
-some $x in [true,true] satisfies not($x)
+some $x in [false,false] satisfies "not"($x),
+some $x in [true,false] satisfies "not"($x),
+some $x in [false,true] satisfies "not"($x),
+some $x in [true,true] satisfies "not"($x)
 ]
 for $i in $x
 return $i
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql
index 62f7690..5bbddf3 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql
@@ -20,6 +20,6 @@
 set import-private-functions 'true';
 
 for $o in dataset('Orders')
-where not(is-missing($o.param1))
+where "not"(is-missing($o.param1))
 order by $o.oid
 return $o
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql
index 356b97b..ac63501 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql
@@ -22,6 +22,6 @@
 use dataverse test;
 
 for $e in dataset log
-where not(is-null($e.RemoteHost))
+where "not"(is-null($e.RemoteHost))
 limit 1
 return substring-after($e.RemoteHost, "@");
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql
index 23e6898..97f8763 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql
@@ -22,6 +22,6 @@
 use dataverse test;
 
 for $e in dataset log
-where not(is-null($e.RemoteHost))
+where "not"(is-null($e.RemoteHost))
 limit 1
 return substring-after($e.RemoteHost, "@");
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql
index b6f302d..573c38c 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql
@@ -24,6 +24,6 @@
 
 
 for $fbu in dataset FacebookUsers
-where (every $e in $fbu.employment satisfies not(is-missing($e.end-date)))
+where (every $e in $fbu.employment satisfies "not"(is-missing($e.end-date)))
 order by $fbu.id
 return $fbu;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql
index 85a3dd3..a262fb4 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql
@@ -26,7 +26,7 @@
       "c_custkey": $c.c_custkey,
       "o_orderkey_count": count(
         from $o in dataset('Orders')
-        where  $c.c_custkey = $o.o_custkey and not(like($o.o_comment,'%special%requests%'))
+        where  $c.c_custkey = $o.o_custkey and "not"(like($o.o_comment,'%special%requests%'))
         select $o.o_orderkey
       )
     }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
index b1c06ad..09be343 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
@@ -23,7 +23,7 @@
     from $ps in dataset('Partsupp')
     from $p in dataset('Part')
     where $p.p_partkey = $ps.ps_partkey and $p.p_brand != 'Brand#45'
-      and not(like($p.p_type, 'MEDIUM POLISHED%'))
+      and "not"(like($p.p_type, 'MEDIUM POLISHED%'))
     select {
       "p_brand": $p.p_brand,
       "p_type": $p.p_type,
@@ -32,7 +32,7 @@
     }
   )
   from $s in dataset('Supplier')
-  where $psp.ps_suppkey = $s.s_suppkey and not(like($s.s_comment, '%Customer%Complaints%'))
+  where $psp.ps_suppkey = $s.s_suppkey and "not"(like($s.s_comment, '%Customer%Complaints%'))
   select {
    "p_brand": $psp.p_brand,
    "p_type": $psp.p_type,
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql
index 2a490d9..16d1791 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql
@@ -26,7 +26,7 @@
       "c_custkey": $c.c_custkey, 
       "o_orderkey_count": count(
         for $o in dataset('Orders')
-        where  $c.c_custkey = $o.o_custkey and not(like($o.o_comment,'%special%requests%'))
+        where  $c.c_custkey = $o.o_custkey and "not"(like($o.o_comment,'%special%requests%'))
         return $o.o_orderkey
       )
     }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
index 8f969fc..f859ade 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
@@ -23,7 +23,7 @@
     for $ps in dataset('Partsupp')
     for $p in dataset('Part')
     where $p.p_partkey = $ps.ps_partkey and $p.p_brand != 'Brand#45' 
-      and not(like($p.p_type, 'MEDIUM POLISHED%'))
+      and "not"(like($p.p_type, 'MEDIUM POLISHED%'))
     return {
       "p_brand": $p.p_brand, 
       "p_type": $p.p_type, 
@@ -32,7 +32,7 @@
     }
   )
   for $s in dataset('Supplier')
-  where $psp.ps_suppkey = $s.s_suppkey and not(like($s.s_comment, '%Customer%Complaints%'))
+  where $psp.ps_suppkey = $s.s_suppkey and "not"(like($s.s_comment, '%Customer%Complaints%'))
   return {
    "p_brand": $psp.p_brand, 
    "p_type": $psp.p_type, 
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
new file mode 100644
index 0000000..9fdd9be
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create function printName() { 'AsterixDB Shared nothing parallel BDMS' };
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
new file mode 100644
index 0000000..087788a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+for $x in dataset Metadata.Function
+where $x.DataverseName = "test"
+return $x;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
new file mode 100644
index 0000000..3e20182
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop dataverse test;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql
index 1c5f6cc..7872722 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql
@@ -25,5 +25,5 @@
 use dataverse test;
 set import-private-functions 'true';
 
-let $val := not(test.fn06())
+let $val := "not"(test.fn06())
 return $val
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp
new file mode 100644
index 0000000..7b4638a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop database test if exists;
+create database test;
+use test;
+
+create function printName() { 'AsterixDB Shared nothing parallel BDMS' };
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp
new file mode 100644
index 0000000..9763645
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+use Metadata;
+
+select value `Function`
+from `Function`
+where DataverseName = 'test';
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp
new file mode 100644
index 0000000..210ad32
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description  : Create a function definition in a single line
+ * See: https://issues.apache.org/jira/browse/ASTERIXDB-1269
+ * Expected Res : Success
+ * Date         : Jul 10th 2016
+ */
+
+
+drop database test;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
new file mode 100644
index 0000000..5ba00ae
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
@@ -0,0 +1 @@
+{ "DataverseName": "test", "Name": "printName", "Arity": "0", "Params": [  ], "ReturnType": "VOID", "Definition": "'AsterixDB Shared nothing parallel BDMS'", "Language": "AQL", "Kind": "SCALAR" }
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
index f7a9b6d..ffa4346 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
@@ -6118,6 +6118,11 @@
   </test-group>
   <test-group name="user-defined-functions">
     <test-case FilePath="user-defined-functions">
+      <compilation-unit name="single-line-definition">
+        <output-dir compare="Text">single-line-definition</output-dir>
+      </compilation-unit>
+    </test-case>
+    <test-case FilePath="user-defined-functions">
       <compilation-unit name="query-issue218-2">
         <output-dir compare="Text">query-issue218-2</output-dir>
       </compilation-unit>
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 96d7c4f..494e830 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -6387,6 +6387,11 @@
   </test-group>
   <test-group name="user-defined-functions">
     <test-case FilePath="user-defined-functions">
+      <compilation-unit name="single-line-definition">
+        <output-dir compare="Text">single-line-definition</output-dir>
+      </compilation-unit>
+    </test-case>
+    <test-case FilePath="user-defined-functions">
       <compilation-unit name="query-issue218-2">
         <output-dir compare="Text">query-issue218-2</output-dir>
       </compilation-unit>
diff --git a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
index b96fb26..fcb6eb4 100644
--- a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
+++ b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
@@ -157,9 +157,7 @@
     private static final String SKIP_SECONDARY_INDEX_SEARCH_HINT = "skip-index";
     private static final String VAL_FILE_HINT = "val-files";
     private static final String VAL_FILE_SAME_INDEX_HINT = "val-file-same-idx";
-
     private static final String GEN_FIELDS_HINT = "gen-fields";
-
     // data generator hints
     private static final String DGEN_HINT = "dgen";
 
@@ -314,7 +312,7 @@
   String dvName = null;
 }
 {
-  "use" "dataverse" dvName = Identifier()
+  <USE> <DATAVERSE> dvName = Identifier()
     {
       defaultDataverse = dvName;
       return new DataverseDecl(new Identifier(dvName));
@@ -328,7 +326,7 @@
   Statement stmt = null;
 }
 {
-  "create"
+  <CREATE>
   (
     {
       hint = getHint(token);
@@ -357,8 +355,8 @@
   TypeExpression typeExpr = null;
 }
 {
-  "type" nameComponents = TypeName() ifNotExists = IfNotExists()
-  "as" typeExpr = TypeExpr()
+  <TYPE> nameComponents = TypeName() ifNotExists = IfNotExists()
+  <AS> typeExpr = TypeExpr()
     {
       long numValues = -1;
       String filename = null;
@@ -384,8 +382,8 @@
   List<Identifier>ncNames = null;
 }
 {
-  "nodegroup" name = Identifier()
-  ifNotExists = IfNotExists() "on" tmp = Identifier()
+  <NODEGROUP> name = Identifier()
+  ifNotExists = IfNotExists() <ON> tmp = Identifier()
     {
       ncNames = new ArrayList<Identifier>();
       ncNames.add(new Identifier(tmp));
@@ -421,13 +419,13 @@
 }
 {
   (
-    "external" <DATASET> nameComponents = QualifiedName()
+    <EXTERNAL> <DATASET> nameComponents = QualifiedName()
     <LEFTPAREN> typeComponents = TypeName() <RIGHTPAREN>
     ifNotExists = IfNotExists()
-    "using" adapterName = AdapterName() properties = Configuration()
-    ("on" nodeGroupName = Identifier() )?
-    ( "hints" hints = Properties() )?
-    ( "using" "compaction" "policy" compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
+    <USING> adapterName = AdapterName() properties = Configuration()
+    ( <ON> nodeGroupName = Identifier() )?
+    ( <HINTS> hints = Properties() )?
+    ( <USING> <COMPACTION> <POLICY> compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
       {
         ExternalDetailsDecl edd = new ExternalDetailsDecl();
         edd.setAdapter(adapterName);
@@ -447,7 +445,7 @@
                                    ifNotExists);
       }
 
-    | ("internal" | "temporary" {
+    | (<INTERNAL> | <TEMPORARY> {
             temp = token.image.toLowerCase().equals("temporary");
         }
       )?
@@ -466,11 +464,11 @@
     )?
     ifNotExists = IfNotExists()
     primaryKeyFields = PrimaryKey()
-    ("autogenerated" { autogenerated = true; } )?
-    ("on" nodeGroupName = Identifier() )?
-    ( "hints" hints = Properties() )?
-    ( "using" "compaction" "policy" compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
-    ( "with filter on" filterField = NestedField() )?
+    ( <AUTOGENERATED> { autogenerated = true; } )?
+    ( <ON> nodeGroupName = Identifier() )?
+    ( <HINTS> hints = Properties() )?
+    ( <USING> <COMPACTION> <POLICY> compactionPolicy = CompactionPolicy() (compactionPolicyProperties = Configuration())? )?
+    ( <WITH> <FILTER> <ON> filterField = NestedField() )?
       {
         if(filterField!=null && filterField.first!=0){
           throw new ParseException("A filter field can only be a field in the main record of the dataset.");
@@ -507,7 +505,7 @@
   String datasetName = null;
 }
 {
-    "refresh external" <DATASET> nameComponents = QualifiedName()
+    <REFRESH> <EXTERNAL> <DATASET> nameComponents = QualifiedName()
     {
     redss.setDataverseName(nameComponents.first);
     redss.setDatasetName(nameComponents.second);
@@ -524,13 +522,13 @@
   Pair<Identifier,Identifier> nameComponentsTo = null;
 }
 {
-  "run" system = Identifier()<LEFTPAREN> ( tmp = Identifier() [<COMMA>]
+  <RUN> system = Identifier()<LEFTPAREN> ( tmp = Identifier() [<COMMA>]
     {
       parameters.add(tmp);
     }
   )*<RIGHTPAREN>
   <FROM> <DATASET> nameComponentsFrom  = QualifiedName()
-  "to" <DATASET> nameComponentsTo  = QualifiedName()
+  <TO> <DATASET> nameComponentsTo  = QualifiedName()
     {
       return new RunStatement(system, parameters, nameComponentsFrom.first, nameComponentsFrom.second, nameComponentsTo.first, nameComponentsTo.second);
     }
@@ -547,9 +545,9 @@
   boolean enforced = false;
 }
 {
-  "index" indexName = Identifier()
+  <INDEX> indexName = Identifier()
   ifNotExists = IfNotExists()
-  "on" nameComponents = QualifiedName()
+  <ON> nameComponents = QualifiedName()
   <LEFTPAREN> ( fieldPair = OpenField()
     {
        cis.addFieldExprPair(fieldPair.second);
@@ -560,7 +558,7 @@
        cis.addFieldExprPair(fieldPair.second);
        cis.addFieldIndexIndicator(fieldPair.first);
     }
-  )* <RIGHTPAREN> ( "type" indexType = IndexType() )? ( "enforced" { enforced = true; } )?
+  )* <RIGHTPAREN> ( <TYPE> indexType = IndexType() )? ( <ENFORCED> { enforced = true; } )?
     {
       cis.setIndexName(new Identifier(indexName));
       cis.setIfNotExists(ifNotExists);
@@ -603,27 +601,29 @@
   int gramLength = 0;
 }
 {
-  ("btree"
+  (
+    <BTREE>
     {
       type = IndexType.BTREE;
     }
-  | "rtree"
+   |<RTREE>
     {
       type = IndexType.RTREE;
     }
-  | "keyword"
+   |<KEYWORD>
     {
       type = IndexType.LENGTH_PARTITIONED_WORD_INVIX;
     }
-  | "ngram" <LEFTPAREN> <INTEGER_LITERAL>
+   |<NGRAM> <LEFTPAREN> <INTEGER_LITERAL>
     {
       type = IndexType.LENGTH_PARTITIONED_NGRAM_INVIX;
       gramLength = Integer.valueOf(token.image);
     }
-  <RIGHTPAREN>)
-    {
-      return new IndexParams(type, gramLength);
-    }
+    <RIGHTPAREN>
+  )
+  {
+    return new IndexParams(type, gramLength);
+  }
 }
 
 CreateDataverseStatement DataverseSpecification() throws ParseException :
@@ -633,9 +633,9 @@
   String format = null;
 }
 {
-  "dataverse" dvName = Identifier()
+  <DATAVERSE> dvName = Identifier()
   ifNotExists = IfNotExists()
-  ( "with format" format = StringLiteral() )?
+  ( <WITH> <FORMAT> format = StringLiteral() )?
     {
       return new CreateDataverseStatement(new Identifier(dvName), format, ifNotExists);
     }
@@ -656,7 +656,7 @@
   createNewScope();
 }
 {
-  "function" fctName = FunctionName()
+  <FUNCTION> fctName = FunctionName()
   ifNotExists = IfNotExists()
   paramList = ParameterList()
   <LEFTBRACE>
@@ -688,23 +688,21 @@
 }
 {
   (
-    "secondary" "feed"  nameComponents = QualifiedName() ifNotExists = IfNotExists()
-      <FROM> "feed" sourceNameComponents = QualifiedName() (appliedFunction = ApplyFunction())?
-      {
-        cfs = new CreateSecondaryFeedStatement(nameComponents,
-                                   sourceNameComponents, appliedFunction, ifNotExists);
-      }
-     |
-     ("primary")? "feed" nameComponents = QualifiedName() ifNotExists = IfNotExists()
-      "using" adapterName = AdapterName() properties = Configuration() (appliedFunction = ApplyFunction())?
-       {
-        cfs = new CreatePrimaryFeedStatement(nameComponents,
-                                    adapterName, properties, appliedFunction, ifNotExists);
-       }
-  )
+    <SECONDARY> <FEED>  nameComponents = QualifiedName() ifNotExists = IfNotExists()
+    <FROM> <FEED> sourceNameComponents = QualifiedName() (appliedFunction = ApplyFunction())?
     {
-      return cfs;
+      cfs = new CreateSecondaryFeedStatement(nameComponents, sourceNameComponents, appliedFunction, ifNotExists);
     }
+   |
+    (<PRIMARY>)? <FEED> nameComponents = QualifiedName() ifNotExists = IfNotExists()
+    <USING> adapterName = AdapterName() properties = Configuration() (appliedFunction = ApplyFunction())?
+     {
+      cfs = new CreatePrimaryFeedStatement(nameComponents, adapterName, properties, appliedFunction, ifNotExists);
+     }
+  )
+  {
+    return cfs;
+  }
 }
 
 CreateFeedPolicyStatement FeedPolicySpecification() throws ParseException:
@@ -719,23 +717,22 @@
 }
 {
   (
-    "ingestion" "policy"  policyName = Identifier() ifNotExists = IfNotExists()
-      <FROM>
-      ("policy" basePolicyName = Identifier() properties = Configuration() ("definition" definition = StringLiteral())?
+    <INGESTION> <POLICY>  policyName = Identifier() ifNotExists = IfNotExists()
+    <FROM>
+    (
+      <POLICY> basePolicyName = Identifier() properties = Configuration() ( <DEFINITION> definition = StringLiteral())?
       {
-        cfps = new CreateFeedPolicyStatement(policyName,
-                                   basePolicyName, properties, definition, ifNotExists);
+        cfps = new CreateFeedPolicyStatement(policyName, basePolicyName, properties, definition, ifNotExists);
       }
-     | "path" sourcePolicyFile = Identifier() ("definition" definition = StringLiteral())?
-       {
+     |<PATH> sourcePolicyFile = Identifier() (<DEFINITION> definition = StringLiteral())?
+      {
         cfps = new CreateFeedPolicyStatement(policyName, sourcePolicyFile, definition, ifNotExists);
-       }
-     )
-
+      }
+    )
   )
-    {
-      return cfps;
-    }
+  {
+    return cfps;
+  }
 }
 
 
@@ -753,7 +750,7 @@
       paramList.add(var);
       getCurrentScope().addNewVarSymbolToScope(var);
     }
-  (<COMMA> <VARIABLE>
+  ( <COMMA> <VARIABLE>
     {
       var = new VarIdentifier();
       var.setValue(token.image);
@@ -770,7 +767,7 @@
 {
 }
 {
-  ( "if not exists"
+  ( <IF> <NOT> <EXISTS>
     {
       return true;
     }
@@ -786,7 +783,7 @@
   FunctionSignature funcSig = null;
 }
 {
-  "apply" "function" functioName = FunctionName()
+  <APPLY> <FUNCTION> functioName = FunctionName()
     {
        String fqFunctionName = functioName.library == null ? functioName.function : functioName.library + "#" + functioName.function;
        return new FunctionSignature(functioName.dataverse, fqFunctionName, 1);
@@ -798,7 +795,7 @@
   String policy = null;
 }
 {
-   "using" "policy" policy = Identifier()
+   <USING> <POLICY> policy = Identifier()
    {
      return policy;
    }
@@ -811,7 +808,7 @@
   int arity = 0;
 }
 {
-  fctName = FunctionName() "@" <INTEGER_LITERAL>
+  fctName = FunctionName() <SYMBOLAT> <INTEGER_LITERAL>
     {
       arity = new Integer(token.image);
       if (arity < 0 && arity != FunctionIdentifier.VARARGS) {
@@ -831,7 +828,7 @@
    List<List<String>> primaryKeyFields = new ArrayList<List<String>>();
 }
 {
-  "primary" "key" tmp = NestedField()
+  <PRIMARY> <KEY> tmp = NestedField()
     {
       keyFieldSourceIndicators.add(tmp.first);
       primaryKeyFields.add(tmp.second);
@@ -857,33 +854,33 @@
   Statement stmt = null;
 }
 {
-  "drop"
+  <DROP>
   (
     <DATASET> pairId = QualifiedName() ifExists = IfExists()
       {
         stmt = new DropStatement(pairId.first, pairId.second, ifExists);
       }
-    | "index" tripleId = DoubleQualifiedName() ifExists = IfExists()
+    | <INDEX> tripleId = DoubleQualifiedName() ifExists = IfExists()
       {
         stmt = new IndexDropStatement(tripleId.first, tripleId.second, tripleId.third, ifExists);
       }
-    | "nodegroup" id = Identifier() ifExists = IfExists()
+    | <NODEGROUP> id = Identifier() ifExists = IfExists()
       {
         stmt = new NodeGroupDropStatement(new Identifier(id), ifExists);
       }
-    | "type" pairId = TypeName() ifExists = IfExists()
+    | <TYPE> pairId = TypeName() ifExists = IfExists()
       {
         stmt = new TypeDropStatement(pairId.first, pairId.second, ifExists);
       }
-    | "dataverse" id = Identifier() ifExists = IfExists()
+    | <DATAVERSE> id = Identifier() ifExists = IfExists()
       {
         stmt = new DataverseDropStatement(new Identifier(id), ifExists);
       }
-    | "function" funcSig = FunctionSignature() ifExists = IfExists()
+    | <FUNCTION> funcSig = FunctionSignature() ifExists = IfExists()
       {
         stmt = new FunctionDropStatement(funcSig, ifExists);
       }
-    | "feed" pairId = QualifiedName() ifExists = IfExists()
+    | <FEED> pairId = QualifiedName() ifExists = IfExists()
       {
         stmt = new FeedDropStatement(pairId.first, pairId.second, ifExists);
       }
@@ -897,7 +894,7 @@
 {
 }
 {
-  ( <IF> "exists"
+  ( <IF> <EXISTS>
     {
       return true;
     }
@@ -914,7 +911,7 @@
   boolean upsert = false;
 }
 {
-  ("insert"|"upsert"{ upsert = true; }) "into" <DATASET> nameComponents = QualifiedName() query = Query()
+  (<INSERT>|<UPSERT>{ upsert = true; }) <INTO> <DATASET> nameComponents = QualifiedName() query = Query()
     {
       query.setTopLevel(true);
       if(upsert){
@@ -936,7 +933,7 @@
 
 }
 {
-  "delete" var = Variable()
+  <DELETE> var = Variable()
     {
       getCurrentScope().addNewVarSymbolToScope(var.getVar());
     }
@@ -963,7 +960,7 @@
   List<UpdateClause> ucs = new ArrayList<UpdateClause>();
 }
 {
-  "update" vars = Variable() <IN> target = Expression()
+  <UPDATE> vars = Variable() <IN> target = Expression()
   <WHERE> condition = Expression()
   <LEFTPAREN> (uc = UpdateClause()
     {
@@ -991,7 +988,7 @@
   UpdateClause elsebranch = null;
 }
 {
-   ("set" target = Expression() <ASSIGN> value = Expression()
+   (<SET> target = Expression() <ASSIGN> value = Expression()
    | is = InsertStatement()
    | ds = DeleteStatement()
    | us = UpdateStatement()
@@ -1010,7 +1007,7 @@
   String pv = null;
 }
 {
-  "set" pn = Identifier() pv = StringLiteral()
+  <SET> pn = Identifier() pv = StringLiteral()
     {
       return new SetStatement(pn, pv);
     }
@@ -1025,8 +1022,8 @@
   Pair<Identifier,Identifier> nameComponents = null;
 }
 {
-  "write" "output" "to" nodeName = Identifier() <COLON> fileName = StringLiteral()
-    ( "using" writerClass = StringLiteral() )?
+  <WRITE> <OUTPUT> <TO> nodeName = Identifier() <COLON> fileName = StringLiteral()
+    ( <USING> writerClass = StringLiteral() )?
     {
       return new WriteStatement(new Identifier(nodeName), fileName, writerClass);
     }
@@ -1042,13 +1039,13 @@
   Pair<Identifier,Identifier> nameComponents = null;
 }
 {
-  "load" <DATASET> nameComponents = QualifiedName()
+  <LOAD> <DATASET> nameComponents = QualifiedName()
     {
       dataverseName = nameComponents.first;
       datasetName = nameComponents.second;
     }
-  "using" adapterName = AdapterName() properties = Configuration()
-  ("pre-sorted"
+  <USING> adapterName = AdapterName() properties = Configuration()
+  (<PRESORTED>
     {
       alreadySorted = true;
     }
@@ -1076,7 +1073,7 @@
   Statement stmt = null;
 }
 {
-  "compact" <DATASET> nameComponents = QualifiedName()
+  <COMPACT> <DATASET> nameComponents = QualifiedName()
     {
       stmt = new CompactStatement(nameComponents.first, nameComponents.second);
     }
@@ -1096,11 +1093,11 @@
 }
 {
   (
-    "connect" "feed" feedNameComponents = QualifiedName() "to" <DATASET> datasetNameComponents = QualifiedName() (policy = GetPolicy())?
+    <CONNECT> <FEED> feedNameComponents = QualifiedName() <TO> <DATASET> datasetNameComponents = QualifiedName() (policy = GetPolicy())?
       {
         stmt = new ConnectFeedStatement(feedNameComponents, datasetNameComponents, policy, getVarCounter());
       }
-    | "disconnect" "feed" feedNameComponents = QualifiedName() <FROM> <DATASET> datasetNameComponents = QualifiedName()
+    | <DISCONNECT> <FEED> feedNameComponents = QualifiedName() <FROM> <DATASET> datasetNameComponents = QualifiedName()
       {
         stmt = new DisconnectFeedStatement(feedNameComponents, datasetNameComponents);
       }
@@ -1219,8 +1216,8 @@
   RecordTypeDefinition.RecordKind recordKind = null;
 }
 {
-  ( "closed" { recordKind = RecordTypeDefinition.RecordKind.CLOSED; }
-    | "open" { recordKind = RecordTypeDefinition.RecordKind.OPEN; } )?
+  ( <CLOSED>{ recordKind = RecordTypeDefinition.RecordKind.CLOSED; }
+    | <OPEN>{ recordKind = RecordTypeDefinition.RecordKind.OPEN; } )?
    <LEFTBRACE>
     {
       String hint = getHint(token);
@@ -1331,7 +1328,7 @@
     {
       secondAfterDot = true;
     }
-  ("#" third = Identifier())? | "#" second = Identifier() )?
+  ( <SYMBOLHASH> third = Identifier())? | <SYMBOLHASH> second = Identifier() )?
     {
       if (second == null) {
         result.dataverse = defaultDataverse;
@@ -1380,7 +1377,7 @@
   String lit = null;
 }
 {
-  (<IDENTIFIER>
+  ((<IDENTIFIER>)
     {
       return token.image;
     }
@@ -1510,7 +1507,7 @@
   createNewScope();
 }
 {
-  "declare" "function" functionName = Identifier()
+  <DECLARE> <FUNCTION> functionName = Identifier()
   paramList = ParameterList()
   <LEFTBRACE> funcBody = Expression() <RIGHTBRACE>
     {
@@ -2518,33 +2515,87 @@
 <DEFAULT,IN_DBL_BRACE>
 TOKEN :
 {
-    <ASC : "asc">
+    <APPLY : "apply">
+  | <AS : "as">
+  | <ASC : "asc">
   | <AT : "at">
+  | <AUTOGENERATED : "autogenerated">
+  | <BTREE : "btree">
   | <BY : "by">
+  | <CLOSED : "closed">
+  | <COMPACT : "compact">
+  | <COMPACTION : "compaction">
+  | <CONNECT : "connect">
+  | <CREATE : "create">
   | <DATASET : "dataset">
+  | <DATAVERSE : "dataverse">
+  | <DECLARE : "declare">
   | <DECOR : "decor">
+  | <DEFINITION : "definition">
+  | <DELETE : "delete">
   | <DESC : "desc">
+  | <DISCONNECT : "disconnect">
   | <DISTINCT : "distinct">
+  | <DROP : "drop">
   | <ELSE : "else">
+  | <ENFORCED : "enforced">
   | <EVERY : "every">
+  | <EXISTS : "exists">
+  | <EXTERNAL : "external">
+  | <FEED : "feed">
+  | <FILTER : "filter">
   | <FOR : "for">
+  | <FORMAT : "format">
   | <FROM : "from">
+  | <FUNCTION : "function">
   | <GROUP : "group">
+  | <HINTS : "hints">
   | <IF : "if">
   | <IN : "in">
+  | <INDEX : "index">
+  | <INGESTION : "ingestion">
+  | <INSERT : "insert">
+  | <INTERNAL : "internal">
+  | <INTO : "into">
+  | <KEY : "key">
+  | <KEYWORD : "keyword">
+  | <KEEPING : "keeping">
   | <LET : "let">
   | <LIMIT : "limit">
+  | <LOAD : "load">
+  | <NGRAM : "ngram">
+  | <NODEGROUP : "nodegroup">
+  | <NOT : "not">
   | <OFFSET : "offset">
+  | <ON : "on">
+  | <OPEN : "open">
   | <ORDER : "order">
+  | <OUTPUT : "output">
+  | <PATH : "path">
+  | <POLICY : "policy">
+  | <PRESORTED : "pre-sorted">
+  | <PRIMARY : "primary">
+  | <REFRESH : "refresh">
   | <RETURN : "return">
+  | <RTREE : "rtree">
+  | <RUN : "run">
   | <SATISFIES : "satisfies">
+  | <SECONDARY : "secondary">
   | <SELECT : "select">
+  | <SET : "set">
   | <SOME : "some">
+  | <TEMPORARY : "temporary">
   | <THEN : "then">
+  | <TO : "to">
+  | <TYPE : "type">
   | <UNION : "union">
+  | <UPDATE : "update">
+  | <UPSERT : "upsert">
+  | <USE : "use">
+  | <USING : "using">
   | <WHERE : "where">
   | <WITH : "with">
-  | <KEEPING : "keeping">
+  | <WRITE : "write">
 }
 
 <DEFAULT,IN_DBL_BRACE>
@@ -2579,6 +2630,9 @@
 
   | <AND : "and">
   | <OR : "or">
+
+  | <SYMBOLAT : "@">
+  | <SYMBOLHASH : "#">
 }
 
 <DEFAULT,IN_DBL_BRACE>
diff --git a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
index 6ab82c1..dc0fa93 100644
--- a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
+++ b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
@@ -222,8 +222,11 @@
 
     public String extractFragment(int beginLine, int beginColumn, int endLine, int endColumn) {
         StringBuilder extract = new StringBuilder();
-        extract.append(inputLines[beginLine - 1].trim().length() > 1
-                ? inputLines[beginLine - 1].trim().substring(beginColumn) : "");
+        if (beginLine == endLine) {
+            // special case that we need to handle separately
+            return inputLines[beginLine - 1].substring(beginColumn, endColumn - 1).trim();
+        }
+        extract.append(inputLines[beginLine - 1].substring(beginColumn));
         for (int i = beginLine + 1; i < endLine; i++) {
             extract.append("\n");
             extract.append(inputLines[i - 1]);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 3:

Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/63/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 5: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Till Westmann (Code Review)" <do...@asterixdb.incubator.apache.org>.
Till Westmann has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 1:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/988/1/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj:

Line 1511:   <DECLARE><FUNCTION>functionName = Identifier()
spaces between the tokens?


Line 2634:   
WS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 5:

Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/66/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/60/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2:

Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/60/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 4:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1858/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 3: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/63/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 5:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1861/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/988

to look at the new patch set (#2).

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................

ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
---
M asterixdb/asterix-app/pom.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
8 files changed, 253 insertions(+), 74 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/88/988/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/988

to look at the new patch set (#5).

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................

ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser

This change fixes the issue ASTERIXDB-1269 and refactor aql.jj
removing qouted strings throughout the file and creating token's
definitions for them instead

Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
---
M asterixdb/asterix-app/pom.xml
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/query-issue697.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate-sql/sum_null-with-pred/sum_null-with-pred.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_null-with-pred/sum_null-with-pred.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/boolean/not_01/not_01.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/everysat_04/everysat_04.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/quantifiers/somesat_06/somesat_06.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/semistructured/has-param1/has-param1.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-5/substring-after-5.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/string/substring-after-6/substring-after-6.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tinysocial/tinysocial-suite/tinysocial-suite.13.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q13_customer_distribution/q13_customer_distribution.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch-sql-like/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q13_customer_distribution/q13_customer_distribution.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/tpch/q16_parts_supplier_relationship/q16_parts_supplier_relationship.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.2.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/single-line-definition/single-line-definition.3.ddl.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/user-defined-functions/udf18/udf18.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/user-defined-functions/single-line-definition/single-line-definition.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/user-defined-functions/single-line-definition/single-line-definition.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
27 files changed, 396 insertions(+), 123 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/88/988/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>.
abdullah alamoudi has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2:

(11 comments)

https://asterix-gerrit.ics.uci.edu/#/c/988/2/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj:

Line 426:     (<ON> nodeGroupName = Identifier() )?
> space missing
Done


Line 467:     (<AUTOGENERATED> { autogenerated = true; } )?
> space missing
Done


Line 468:     (<ON> nodeGroupName = Identifier() )?
> space missing
Done


Line 604:   (<BTREE>
> space missing
Done


Line 657:   <FUNCTION>fctName = FunctionName()
> space missing
Done


Line 689:     <SECONDARY> <FEED>  nameComponents = QualifiedName() ifNotExists = IfNotExists()
> 2 spaces
Done


Line 720:     <INGESTION> <POLICY>  policyName = Identifier() ifNotExists = IfNotExists()
> 2 spaces
Done


Line 812:   fctName = FunctionName() <SYMBOLAT><INTEGER_LITERAL>
> space missing
Done


Line 1332:   (<SYMBOLHASH> third = Identifier())? | <SYMBOLHASH> second = Identifier() )?
> space missing
Done


Line 1381:   ((<IDENTIFIER>|<NOT>)
> Why is NOT an identifier?
NOT is a keyword used in: 
if not exists;
we also have a not function.

So when the parser encounters it, it will think that is the NOT keyword unless you specifically say that NOT can also be an identifier.

This used to work before because if not exists was specified as a single string "if not exists"


Line 1381:   ((<IDENTIFIER>|<NOT>)
> It's better to change test queries, instead of hacking identifiers here.
why is that better?
I really don't like having to add quotes.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Yingyi Bu (Code Review)" <do...@asterixdb.incubator.apache.org>.
Yingyi Bu has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2:

(1 comment)

Can you add a SQL++ query as a regression test? 
It seems to me that the SQL++ query should just work because you fixed that in ScopeChecker.

https://asterix-gerrit.ics.uci.edu/#/c/988/2/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj:

Line 1381:   ((<IDENTIFIER>|<NOT>)
It's better to change test queries, instead of hacking identifiers here.
For those queries calling function not(....),  do the following change:
not -> "not", i.e., to use delimited identifiers.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 1:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1851/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 2:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/1852/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor ...

Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1269 Fix Extraction of Query Segment and Refactor AQL Parser
......................................................................


Patch Set 4: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/64/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/988
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I18ac4f8d86b3c5c7bfe226c98114499671649e93
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>
Gerrit-HasComments: No