You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by sansanichfb <gi...@git.apache.org> on 2016/09/19 21:34:33 UTC
[GitHub] incubator-hawq pull request #913: HAWQ-1048.
GitHub user sansanichfb opened a pull request:
https://github.com/apache/incubator-hawq/pull/913
HAWQ-1048.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sansanichfb/incubator-hawq HAWQ-1048
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/913.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #913
----
commit cd186f6fb3ab6bc74833ef5184da03f113ee7995
Author: Oleksandr Diachenko <od...@pivotal.io>
Date: 2016-09-17T02:00:50Z
HAWQ-1048. Draft implementation.
commit 7ef7e0fbfe332d28755175045b1835f5c1117e25
Author: Oleksandr Diachenko <od...@pivotal.io>
Date: 2016-09-19T21:05:22Z
HAWQ-1048. Added free logic.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #913: HAWQ-1048. Support OR, NOT logical operato...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/913#discussion_r80160834
--- Diff: src/backend/access/external/pxffilters.c ---
@@ -187,38 +208,19 @@ pxf_make_filter_list(List *quals)
{
case T_OpExpr:
{
- OpExpr *expr = (OpExpr *) node;
- PxfFilterDesc *filter;
-
- filter = (PxfFilterDesc *) palloc0(sizeof(PxfFilterDesc));
- elog(DEBUG5, "pxf_make_filter_list: node tag %d (T_OpExpr)", tag);
-
- if (opexpr_to_pxffilter(expr, filter))
- result = lappend(result, filter);
- else
- pfree(filter);
-
+ result = lappend(result, node);
break;
}
case T_BoolExpr:
{
BoolExpr *expr = (BoolExpr *) node;
- BoolExprType boolType = expr->boolop;
- elog(DEBUG5, "pxf_make_filter_list: node tag %d (T_BoolExpr), bool node type %d %s",
- tag, boolType, boolType==AND_EXPR ? "(AND_EXPR)" : "");
-
- /* only AND_EXPR is supported */
- if (expr->boolop == AND_EXPR)
- {
- List *inner_result = pxf_make_filter_list(expr->args);
- elog(DEBUG5, "pxf_make_filter_list: inner result size %d", list_length(inner_result));
- result = list_concat(result, inner_result);
- }
+ List *inner_result = pxf_make_expression_items_list(expr->args);
+ result = list_concat(result, inner_result);
+ result = lappend(result, node);
break;
}
default:
- /* expression not supported. ignore */
- elog(DEBUG5, "pxf_make_filter_list: unsupported node tag %d", tag);
+ elog(DEBUG5, "pxf_make_expression_items_list: unsupported node tag %d", tag);
--- End diff --
lets make this DEBUG1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #913: HAWQ-1048. Support OR, NOT logical operato...
Posted by sansanichfb <gi...@git.apache.org>.
Github user sansanichfb closed the pull request at:
https://github.com/apache/incubator-hawq/pull/913
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #913: HAWQ-1048. Support OR, NOT logical operato...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/913#discussion_r82915549
--- Diff: src/backend/access/external/pxffilters.c ---
@@ -153,72 +186,102 @@ Oid pxf_supported_types[] =
CHAROID,
BYTEAOID,
BOOLOID,
- DATEOID
+ DATEOID,
+ TIMESTAMPOID
};
+static void
+pxf_free_expression_items_list(List *expressionItems, bool freeBoolExprNodes)
+{
+ ListCell *lc = NULL;
+ ExpressionItem *expressionItem = NULL;
+ int previousLength;
+
+ while (list_length(expressionItems) > 0)
+ {
+ expressionItem = (ExpressionItem *) lfirst(list_head(expressionItems));
+ if (freeBoolExprNodes && nodeTag(expressionItem->node) == T_BoolExpr)
+ {
+ pfree((BoolExpr *)expressionItem->node);
+ }
+ pfree(expressionItem);
+
+ /* to avoid freeing already freed items - delete all occurrences of current expression*/
+ previousLength = expressionItems->length + 1;
+ while (expressionItems != NULL && previousLength > expressionItems->length)
+ {
+ previousLength = expressionItems->length;
+ expressionItems = list_delete_ptr(expressionItems, expressionItem);
+ }
+ }
+}
+
/*
- * pxf_make_filter_list
+ * pxf_make_expression_items_list
*
* Given a scan node qual list, find the filters that are eligible to be used
- * by PXF, construct a PxfFilterDesc list that describes the filter information,
+ * by PXF, construct an expressions list, which consists of OpExpr or BoolExpr nodes
--- End diff --
We are no longer checking for the validity of the filter in this function right. It is now checked in pxf_serialize_filter_list. So remove this comment.
You don't have to mention in comment the types of expr supported as this list will be growing with time
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #913: HAWQ-1048. Support OR, NOT logical operato...
Posted by shivzone <gi...@git.apache.org>.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/913#discussion_r83259075
--- Diff: src/backend/access/external/pxffilters.c ---
@@ -153,72 +186,102 @@ Oid pxf_supported_types[] =
CHAROID,
BYTEAOID,
BOOLOID,
- DATEOID
+ DATEOID,
+ TIMESTAMPOID
};
+static void
+pxf_free_expression_items_list(List *expressionItems, bool freeBoolExprNodes)
+{
+ ListCell *lc = NULL;
+ ExpressionItem *expressionItem = NULL;
+ int previousLength;
+
+ while (list_length(expressionItems) > 0)
+ {
+ expressionItem = (ExpressionItem *) lfirst(list_head(expressionItems));
+ if (freeBoolExprNodes && nodeTag(expressionItem->node) == T_BoolExpr)
+ {
+ pfree((BoolExpr *)expressionItem->node);
+ }
+ pfree(expressionItem);
+
+ /* to avoid freeing already freed items - delete all occurrences of current expression*/
+ previousLength = expressionItems->length + 1;
+ while (expressionItems != NULL && previousLength > expressionItems->length)
+ {
+ previousLength = expressionItems->length;
+ expressionItems = list_delete_ptr(expressionItems, expressionItem);
+ }
+ }
+}
+
/*
- * pxf_make_filter_list
+ * pxf_make_expression_items_list
*
* Given a scan node qual list, find the filters that are eligible to be used
- * by PXF, construct a PxfFilterDesc list that describes the filter information,
+ * by PXF, construct an expressions list, which consists of OpExpr or BoolExpr nodes
* and return it to the caller.
*
- * Caller is responsible for pfreeing the returned PxfFilterDesc List.
+ * Basically this function just transforms expression tree to Reversed Polish Notation list.
+ *
+ *
*/
static List *
-pxf_make_filter_list(List *quals)
+pxf_make_expression_items_list(List *quals, Node *parent, bool *logicalOpsNum)
{
+ ExpressionItem *expressionItem = NULL;
List *result = NIL;
ListCell *lc = NULL;
+ ListCell *ilc = NULL;
if (list_length(quals) == 0)
return NIL;
- /*
- * Iterate over all implicitly ANDed qualifiers and add the ones
- * that are supported for push-down into the result filter list.
- */
foreach (lc, quals)
{
Node *node = (Node *) lfirst(lc);
NodeTag tag = nodeTag(node);
+ expressionItem = (ExpressionItem *) palloc0(sizeof(ExpressionItem));
+ expressionItem->node = node;
+ expressionItem->parent = parent;
+ expressionItem->processed = false;
switch (tag)
{
case T_OpExpr:
{
- OpExpr *expr = (OpExpr *) node;
- PxfFilterDesc *filter;
-
- filter = (PxfFilterDesc *) palloc0(sizeof(PxfFilterDesc));
- elog(DEBUG5, "pxf_make_filter_list: node tag %d (T_OpExpr)", tag);
-
- if (opexpr_to_pxffilter(expr, filter))
- result = lappend(result, filter);
- else
- pfree(filter);
-
+ result = lappend(result, expressionItem);
break;
}
case T_BoolExpr:
{
+ (*logicalOpsNum)++;
BoolExpr *expr = (BoolExpr *) node;
- BoolExprType boolType = expr->boolop;
- elog(DEBUG5, "pxf_make_filter_list: node tag %d (T_BoolExpr), bool node type %d %s",
- tag, boolType, boolType==AND_EXPR ? "(AND_EXPR)" : "");
+ List *inner_result = pxf_make_expression_items_list(expr->args, node, logicalOpsNum);
+ result = list_concat(result, inner_result);
+
+ int childNodesNum = 0;
- /* only AND_EXPR is supported */
- if (expr->boolop == AND_EXPR)
+ /* Find number of child nodes on first level*/
+ foreach (ilc, inner_result)
{
- List *inner_result = pxf_make_filter_list(expr->args);
- elog(DEBUG5, "pxf_make_filter_list: inner result size %d", list_length(inner_result));
- result = list_concat(result, inner_result);
+ ExpressionItem *ei = (ExpressionItem *) lfirst(ilc);
+ if (!ei->processed && ei->parent == node)
+ {
+ ei->processed = true;
+ childNodesNum++;
+ }
+ }
+
+ for (int i = 0; i < childNodesNum - 1; i++)
--- End diff --
Why exactly do we need to add children in first level again after we do a list_concat(result, inner_result) ?
I'm not understanding this flow for BoolExpr
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---