You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/07/18 21:50:34 UTC
[GitHub] [iceberg] samredai opened a new pull request, #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
samredai opened a new pull request, #5303:
URL: https://github.com/apache/iceberg/pull/5303
This adds the groundwork for a `BoundBooleanExpressionVisitor`. It only implements visiting a `BoundIn` expression since that's the only bound predicate implemented so far. As more expressions are added, a method must be registered in the `BoundBooleanExpressionVisitor` (using `@visit_bound_predicate.register(<expression class here>)`) and also a function must be registered with the `visit` dispatch function (using `@visit.register(<expression class here>)`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925979830
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
@rdblue sorry that was a mistype, I meant it inherits from `BooleanExpressionVisitor`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r926916214
##########
python/pyiceberg/expressions/base.py:
##########
@@ -431,7 +431,13 @@ def _(obj: And, visitor: BooleanExpressionVisitor[T]) -> T:
@visit.register(In)
def _(obj: In, visitor: BooleanExpressionVisitor[T]) -> T:
"""Visit an In boolean expression with a concrete BooleanExpressionVisitor"""
- return visitor.visit_unbound_predicate(predicate=obj)
+ return visitor.visit_unbound_predicate(obj)
+
+
+@visit.register(BoundIn)
+def _(obj: BoundIn, visitor: BooleanExpressionVisitor[T]) -> T:
Review Comment:
Ah right, makes sense--the whole purpose for the inheritance. Updated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975772854
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
Review Comment:
I see, I noticed the behavior so included this test but what you're saying makes sense. So is it safe to just remove this test and expect that the higher-level APIs would prevent an `IsNaN` expression on a string type? Or do you feel that we need to add some validation here? (sounds like we don't need that validation but just want to confirm)
```py
@visit_bound_predicate.register(BoundIsNaN)
def _(expr: BoundIsNaN, visitor: BoundBooleanExpressionVisitor[T]) -> T:
# validate that `expr.term.field.field_type` is an appropriate type
return visitor.visit_is_nan(term=expr.term)
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
+
+
+@visit_bound_predicate.register(BoundIn)
+def _(obj: BoundIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound In predicate"""
+ return visitor.visit_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundNotIn)
+def _(obj: BoundNotIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
Review Comment:
Updated 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975762653
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.Not(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN"]
+
+
+def test_bound_boolean_expression_visitor_and_in():
+ """Test visiting an And and In expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literals=(StringLiteral("baz"), StringLiteral("qux")),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "IN", "AND"]
+
+
+def test_bound_boolean_expression_visitor_or():
+ """Test visiting an Or expression with a bound boolean expression visitor"""
+ bound_expression = base.Or(
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ {StringLiteral("foo"), StringLiteral("bar")},
+ )
+ ),
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ {StringLiteral("baz"), StringLiteral("qux")},
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "NOT", "IN", "NOT", "OR"]
+
+
+def test_bound_boolean_expression_visitor_equal_and_not_equal():
+ """Test visiting an And expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundNotEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ base.BoundEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["NOT_EQUAL", "EQUAL", "AND"]
Review Comment:
Can you add separate tests that validate not-equal and equal in isolation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975759809
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
Review Comment:
This is suspicious. You can't bind `IsNaN` to a string, so this should never happen. In addition, I don't see why the visitor would handle this any differently. There may be a runtime failure because the expression is improperly bound (users should never do this) but the visitor shouldn't simply ignore it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975755647
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
+
+
+@visit_bound_predicate.register(BoundIn)
+def _(obj: BoundIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound In predicate"""
+ return visitor.visit_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundNotIn)
+def _(obj: BoundNotIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotIn predicate"""
+ return visitor.visit_not_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundIsNaN)
+def _(obj: BoundIsNaN, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound IsNan predicate"""
+ return visitor.visit_is_nan(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundNotNaN)
+def _(obj: BoundNotNaN, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotNan predicate"""
+ return visitor.visit_not_nan(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundIsNull)
+def _(obj: BoundIsNull, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound IsNull predicate"""
+ return visitor.visit_is_null(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundNotNull)
+def _(obj: BoundNotNull, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotNull predicate"""
+ return visitor.visit_not_null(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundEqualTo)
+def _(obj: BoundEqualTo, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound EqualTo predicate"""
Review Comment:
Are these doc strings visible anywhere? It seems pretty clear what's happening and since this is a `@singledispatch` method I'm not sure when we'd ever see them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r954077446
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,55 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, ref: BoundReference[T], literals: set[Literal[Any]]) -> T:
Review Comment:
If we update this to Term then we won't need to implement `handle_non_reference` because visitors will be built to handle non-references.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r964123604
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,42 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, ref: BoundReference[T], literals: set[Literal[Any]]) -> T:
+ """Visit the ref and literals from an In boolean expression
+ Args:
+ ref (BoundReference[T]): The ref used in the In boolean expression
+ literals (Tuple[Literal[T], ...]): The literals used in the In boolean expression
+ Raises:
+ NotImplementedError: If the concrete bound boolean expression visitor does not override this method
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
Review Comment:
Do you want to add methods for the other expression types?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r926800328
##########
python/pyiceberg/expressions/base.py:
##########
@@ -431,7 +431,13 @@ def _(obj: And, visitor: BooleanExpressionVisitor[T]) -> T:
@visit.register(In)
def _(obj: In, visitor: BooleanExpressionVisitor[T]) -> T:
"""Visit an In boolean expression with a concrete BooleanExpressionVisitor"""
- return visitor.visit_unbound_predicate(predicate=obj)
+ return visitor.visit_unbound_predicate(obj)
+
+
+@visit.register(BoundIn)
+def _(obj: BoundIn, visitor: BooleanExpressionVisitor[T]) -> T:
Review Comment:
I don't think this should be registered for the regular expression visitor, just the bound visitor. The regular visitor should always call `visit_bound_predicate` for all bound predicates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r976806503
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -104,17 +109,95 @@ def visit_test_expression_b(self) -> List:
@base.visit.register(ExpressionA)
-def _(obj: ExpressionA, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionA, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionA with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_a()
@base.visit.register(ExpressionB)
-def _(obj: ExpressionB, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionB, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionB with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_b()
+class FooBoundBooleanExpressionVisitor(base.BoundBooleanExpressionVisitor[List]):
Review Comment:
Nit: Rather than `Foo` it would be better to name this `TestVisitor` or something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975772092
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.Not(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN"]
+
+
+def test_bound_boolean_expression_visitor_and_in():
+ """Test visiting an And and In expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literals=(StringLiteral("baz"), StringLiteral("qux")),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "IN", "AND"]
+
+
+def test_bound_boolean_expression_visitor_or():
+ """Test visiting an Or expression with a bound boolean expression visitor"""
+ bound_expression = base.Or(
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ {StringLiteral("foo"), StringLiteral("bar")},
+ )
+ ),
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ {StringLiteral("baz"), StringLiteral("qux")},
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "NOT", "IN", "NOT", "OR"]
+
+
+def test_bound_boolean_expression_visitor_equal_and_not_equal():
+ """Test visiting an And expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundNotEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ base.BoundEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["NOT_EQUAL", "EQUAL", "AND"]
Review Comment:
Done! Much clearer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r964123399
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,55 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, ref: BoundReference[T], literals: set[Literal[Any]]) -> T:
Review Comment:
Rather than `ref` I think this should accept `term: BoundTerm[T]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975777214
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
Review Comment:
We don't need validation in the visitor. We should assume that the bound expression has been created correctly by the binding process and not by people instantiating the expressions directly. Maybe we should also move these into `bound.py` to avoid exposing them?
We also need to understand why the expression was skipped. That sounds like a bug to me. I don't think there's anything that checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925304677
##########
python/pyiceberg/expressions/base.py:
##########
@@ -474,3 +474,77 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(Generic[R], BooleanExpressionVisitor[R], ABC):
+ def visit_in(self, ref: BoundReference[T], literals: Tuple[Literal[T], ...]) -> R:
+ """Visit the ref and literals from an In boolean expression
+
+ Args:
+ ref (BoundReference[T]): The ref used in the In boolean expression
+ literals (Tuple[Literal[T], ...]): The literals used in the In boolean expression
+
+ Raises:
+ NotImplementedError: If the concrete bound boolean expression visitor does not override this method
+
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+ raise NotImplementedError(f"{type(self).__name__} does not support visiting In expressions")
+
+ def handle_non_reference(self, term: BoundTerm) -> R:
+ """Handle a non-reference value in this visitor
+
+ Visitors that specificially require references can use this method to return a default value for expressions with non-references.
+ The default implementation will throw a TypeError because the non-reference is not supported.
+
+ Args:
+ term (BoundTerm): a non-reference term such as a transform
+
+ Raises:
+ TypeError: when handle_non_reference is not implemented in the concrete visitor
+
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+ raise TypeError(f"BoundExpressionVisitor does not support non-reference: {term}")
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ @singledispatchmethod
Review Comment:
Keep in mind that the `singledispatchmethod` is much slower than just single dispatch: https://bugs.python.org/issue40988
I'm not sure how big these expressions trees can grow. In the case of auto-generated SQL from some BI tool, I think it can become significant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r964122636
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,42 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, ref: BoundReference[T], literals: set[Literal[Any]]) -> T:
+ """Visit the ref and literals from an In boolean expression
+ Args:
+ ref (BoundReference[T]): The ref used in the In boolean expression
+ literals (Tuple[Literal[T], ...]): The literals used in the In boolean expression
+ Raises:
+ NotImplementedError: If the concrete bound boolean expression visitor does not override this method
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ Raises:
+ TypeError: When an unsupported predicate is found
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+ if not isinstance(predicate.term, BoundReference):
+ raise TypeError(f"{type(self).__name__} does not support non-reference: {predicate.term}")
Review Comment:
I don't think that we need to fail here. This is most likely an artifact of not having `Transform` expressions at first in the Java implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r978162614
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
Review Comment:
Okay, I looked at what's happening here and it is a bit misleading because we are rewriting expressions in the expression constructors. `BoundIsNaN(...)` actually returns `AlwaysFalse()` because it cannot be true for a string. Then `Not(AlwaysFalse())` returns `AlwaysTrue()` and `And` returns its left argument because the right argument is a constant.
In the end, this is testing expression construction that simplifies automatically, not the visitor. I'm going to just remove this test since it looks like everything functions properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#issuecomment-1252487415
Updated this PR to address the remaining feedback. Thanks for the reminder @Fokko!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975762419
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.Not(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN"]
+
+
+def test_bound_boolean_expression_visitor_and_in():
+ """Test visiting an And and In expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literals=(StringLiteral("baz"), StringLiteral("qux")),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "IN", "AND"]
+
+
+def test_bound_boolean_expression_visitor_or():
+ """Test visiting an Or expression with a bound boolean expression visitor"""
+ bound_expression = base.Or(
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ {StringLiteral("foo"), StringLiteral("bar")},
+ )
+ ),
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ {StringLiteral("baz"), StringLiteral("qux")},
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "NOT", "IN", "NOT", "OR"]
+
+
+def test_bound_boolean_expression_visitor_equal_and_not_equal():
+ """Test visiting an And expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundNotEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ base.BoundEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["NOT_EQUAL", "EQUAL", "AND"]
+
+
+def test_bound_boolean_expression_visitor_always_true():
+ """Test visiting an And expression with a bound boolean expression visitor that resolves to AlwaysTrue"""
+ bound_expression = base.Or(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ )
+ ),
+ base.BoundNotNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["TRUE"]
+
+
+def test_bound_boolean_expression_visitor_always_false():
+ """Test visiting an And expression with a bound boolean expression visitor that resolves to AlwaysFalse"""
+ bound_expression = base.And(
Review Comment:
Why not test the false/true expressions directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #5303:
URL: https://github.com/apache/iceberg/pull/5303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r976808752
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -104,17 +109,95 @@ def visit_test_expression_b(self) -> List:
@base.visit.register(ExpressionA)
-def _(obj: ExpressionA, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionA, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionA with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_a()
@base.visit.register(ExpressionB)
-def _(obj: ExpressionB, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionB, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionB with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_b()
+class FooBoundBooleanExpressionVisitor(base.BoundBooleanExpressionVisitor[List]):
Review Comment:
I originally had it as `TestBoundBooleanExpressionVisitor` but since classes that start with `Test` is a pytest convention, the test suite was getting confused. How about `ExampleVisitor`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925302436
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
I would also prefer to keep it abstract. What's also common is to raise a `NotImplementedError`, see https://docs.python.org/3/library/exceptions.html#NotImplementedError. This way you still need to define all the methods, but you can just raise the exceptions on the methods that don't make any sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r923893110
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -634,3 +651,16 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor():
+ bound_expression = base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN"]
Review Comment:
Just a simple test here for now using the test implementation `FooBoundBooleanExpressionVisitor`. Once more predicates are added we include more robust tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#issuecomment-1188356748
cc: @CircArgs @Fokko @dramaticlly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#issuecomment-1221449212
> @samredai, can you rebase?
Rebased, thanks for the ping! 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975756136
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
+
+
+@visit_bound_predicate.register(BoundIn)
+def _(obj: BoundIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound In predicate"""
+ return visitor.visit_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundNotIn)
+def _(obj: BoundNotIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
Review Comment:
Nit: I'd probably rename `obj` to `expr` since it is an expression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975754455
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
Review Comment:
Don't we want to throw an exception if this case is hit? It is an unknown predicate so we can't handle it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975773101
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
+
+
+@visit_bound_predicate.register(BoundIn)
+def _(obj: BoundIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound In predicate"""
+ return visitor.visit_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundNotIn)
+def _(obj: BoundNotIn, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotIn predicate"""
+ return visitor.visit_not_in(term=obj.term, literals=obj.literals)
+
+
+@visit_bound_predicate.register(BoundIsNaN)
+def _(obj: BoundIsNaN, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound IsNan predicate"""
+ return visitor.visit_is_nan(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundNotNaN)
+def _(obj: BoundNotNaN, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotNan predicate"""
+ return visitor.visit_not_nan(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundIsNull)
+def _(obj: BoundIsNull, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound IsNull predicate"""
+ return visitor.visit_is_null(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundNotNull)
+def _(obj: BoundNotNull, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound NotNull predicate"""
+ return visitor.visit_not_null(term=obj.term)
+
+
+@visit_bound_predicate.register(BoundEqualTo)
+def _(obj: BoundEqualTo, visitor: BoundBooleanExpressionVisitor[T]) -> T:
+ """Visit a bound EqualTo predicate"""
Review Comment:
Good point, removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975774239
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,166 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound In predicate"""
+
+ @abstractmethod
+ def visit_not_in(self, term: BoundTerm[T], literals: set[Literal[Any]]) -> T:
+ """Visit a bound NotIn predicate"""
+
+ @abstractmethod
+ def visit_is_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNan predicate"""
+
+ @abstractmethod
+ def visit_not_nan(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNan predicate"""
+
+ @abstractmethod
+ def visit_is_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound IsNull predicate"""
+
+ @abstractmethod
+ def visit_not_null(self, term: BoundTerm[T]) -> T:
+ """Visit a bound NotNull predicate"""
+
+ @abstractmethod
+ def visit_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound Equal predicate"""
+
+ @abstractmethod
+ def visit_not_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound NotEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_greater_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound GreaterThan predicate"""
+
+ @abstractmethod
+ def visit_less_than(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThan predicate"""
+
+ @abstractmethod
+ def visit_less_than_or_equal(self, term: BoundTerm[T], literal: Literal[Any]) -> T:
+ """Visit a bound LessThanOrEqual predicate"""
+
+ @abstractmethod
+ def visit_true(self) -> T:
+ """Visit a bound True predicate"""
+
+ @abstractmethod
+ def visit_false(self) -> T:
+ """Visit a bound False predicate"""
+
+ @abstractmethod
+ def visit_not(self, child_result: T) -> T:
+ """Visit a bound Not predicate"""
+
+ @abstractmethod
+ def visit_and(self, left_result: T, right_result: T) -> T:
+ """Visit a bound And predicate"""
+
+ @abstractmethod
+ def visit_or(self, left_result: T, right_result: T) -> T:
+ """Visit a bound Or predicate"""
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
+ """Visit a bound predicate
+ Args:
+ predicate (BoundPredicate[T]): A bound predicate
+ """
+ return visit_bound_predicate(predicate, self)
+
+
+@singledispatch
+def visit_bound_predicate(obj, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
+ pass
Review Comment:
I think I misinterpreted your previous comment about not needing to fail, but I see that this here is different. Added raising of a `TypeError`.
```py
@singledispatch
def visit_bound_predicate(expr, visitor: BooleanExpressionVisitor[T]) -> T: # pylint: disable=unused-argument
raise TypeError(f"Unknown predicate: {expr}")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925972944
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
> BoundBooleanExpressionVisitor needs to inherit from BooleanExpression in order to use the visit function defined here
I'm not sure I'm following. Did you mean `BooleanExpression`? I don't see why the visitor would need to be an expression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r923896468
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
I changed `BooleanExpressionVisitor` to no longer be an abstract base class so as not to force expression visitors to have to override the methods for nodes that will be ignored. For example `BoundBooleanExpressionVisitor` takes no action when it visits an `And` node.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r964122235
##########
python/pyiceberg/expressions/base.py:
##########
@@ -672,3 +683,42 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(BooleanExpressionVisitor[T], ABC):
+ @abstractmethod
+ def visit_in(self, ref: BoundReference[T], literals: set[Literal[Any]]) -> T:
+ """Visit the ref and literals from an In boolean expression
+ Args:
+ ref (BoundReference[T]): The ref used in the In boolean expression
+ literals (Tuple[Literal[T], ...]): The literals used in the In boolean expression
+ Raises:
+ NotImplementedError: If the concrete bound boolean expression visitor does not override this method
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ def visit_bound_predicate(self, predicate: BoundPredicate[T]) -> T:
Review Comment:
I think this should call another singledispatch method that accepts a bound predicate type and a bound expression visitor, rather than calling `isinstance` itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r976808752
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -104,17 +109,95 @@ def visit_test_expression_b(self) -> List:
@base.visit.register(ExpressionA)
-def _(obj: ExpressionA, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionA, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionA with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_a()
@base.visit.register(ExpressionB)
-def _(obj: ExpressionB, visitor: BooleanExpressionVisitor) -> List:
+def _(obj: ExpressionB, visitor: FooBooleanExpressionVisitor) -> List:
"""Visit a ExpressionB with a BooleanExpressionVisitor"""
return visitor.visit_test_expression_b()
+class FooBoundBooleanExpressionVisitor(base.BoundBooleanExpressionVisitor[List]):
Review Comment:
I originally had it as `TestBoundBooleanExpressionVisitor` but since classes that start with `Test` is a pytest convention, the test suite was getting confused. Updated the PR to use `ExampleVisitor`. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925184879
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
Yes it would return `None`. What prompted me to do this is that `BoundBooleanExpressionVisitor` needs to inherit from `BooleanExpression` in order to use the `visit` function defined here. But that means it would need to override each method it doesn't use and return `None` (this is my understanding of what the Java one does).
But I guess there's no reason that needs to be a default behavior from the parent class and it's better to explicitly define that for this visitor. I'll change this back and implement those methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925403357
##########
python/pyiceberg/expressions/base.py:
##########
@@ -474,3 +474,77 @@ def visit_unbound_predicate(self, predicate) -> BooleanExpression:
def visit_bound_predicate(self, predicate) -> BooleanExpression:
raise TypeError(f"Found already bound predicate: {predicate}")
+
+
+class BoundBooleanExpressionVisitor(Generic[R], BooleanExpressionVisitor[R], ABC):
+ def visit_in(self, ref: BoundReference[T], literals: Tuple[Literal[T], ...]) -> R:
+ """Visit the ref and literals from an In boolean expression
+
+ Args:
+ ref (BoundReference[T]): The ref used in the In boolean expression
+ literals (Tuple[Literal[T], ...]): The literals used in the In boolean expression
+
+ Raises:
+ NotImplementedError: If the concrete bound boolean expression visitor does not override this method
+
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+ raise NotImplementedError(f"{type(self).__name__} does not support visiting In expressions")
+
+ def handle_non_reference(self, term: BoundTerm) -> R:
+ """Handle a non-reference value in this visitor
+
+ Visitors that specificially require references can use this method to return a default value for expressions with non-references.
+ The default implementation will throw a TypeError because the non-reference is not supported.
+
+ Args:
+ term (BoundTerm): a non-reference term such as a transform
+
+ Raises:
+ TypeError: when handle_non_reference is not implemented in the concrete visitor
+
+ Returns:
+ R: The return type defined by the concrete bound boolean expression visitor
+ """
+ raise TypeError(f"BoundExpressionVisitor does not support non-reference: {term}")
+
+ def visit_unbound_predicate(self, predicate: UnboundPredicate[T]):
+ """Visit an unbound predicate
+
+ Args:
+ predicate (UnboundPredicate[T]): An unbound predicate
+
+ Raises:
+ TypeError: This always raises since an unbound predicate is not expected in a bound boolean expression
+ """
+ raise TypeError(f"Not a bound predicate: {predicate}")
+
+ @singledispatchmethod
Review Comment:
That's right, I forgot about the performance hit when using this. I replaced it with a simple if block that does an `isinstance` check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925413163
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
Ok updated to revert this back to an ABC class. Since `BoundBooleanExpressionVisitor` is also an ABC class, the abstractmethods are just carried forward. I'll keep in mind the `NotImplementedError` option as we add concrete implementations. 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r976805363
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
Review Comment:
@samredai, we still need to find out what's going on here. This appears to be ignoring parts of the bound expression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#issuecomment-1255658731
Thanks, @samredai!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r925116064
##########
python/pyiceberg/expressions/base.py:
##########
@@ -327,30 +328,26 @@ def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
return BoundIn(bound_ref, tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
-class BooleanExpressionVisitor(Generic[T], ABC):
Review Comment:
I don't think this is a good idea. Don't we want to ensure that all of the cases are handled? What is the return value of visit methods that are not implemented? `None`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#discussion_r975772322
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -841,3 +924,290 @@ def test_not_expression_binding(unbound_not_expression, expected_bound_expressio
"""Test that visiting an unbound NOT expression with a bind-visitor returns the expected bound expression"""
bound_expression = base.visit(unbound_not_expression, visitor=base.BindVisitor(schema=table_schema_simple))
assert bound_expression == expected_bound_expression
+
+
+def test_bound_boolean_expression_visitor_ignore_is_nan_for_string_type():
+ """Test ignoring IsNan expressions when the field type is a String"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.Not(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN"]
+
+
+def test_bound_boolean_expression_visitor_and_in():
+ """Test visiting an And and In expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literals=(StringLiteral("foo"), StringLiteral("bar")),
+ ),
+ base.BoundIn[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literals=(StringLiteral("baz"), StringLiteral("qux")),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "IN", "AND"]
+
+
+def test_bound_boolean_expression_visitor_or():
+ """Test visiting an Or expression with a bound boolean expression visitor"""
+ bound_expression = base.Or(
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ {StringLiteral("foo"), StringLiteral("bar")},
+ )
+ ),
+ base.Not(
+ base.BoundIn[str](
+ base.BoundReference[str](
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ {StringLiteral("baz"), StringLiteral("qux")},
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["IN", "NOT", "IN", "NOT", "OR"]
+
+
+def test_bound_boolean_expression_visitor_equal_and_not_equal():
+ """Test visiting an And expression with a bound boolean expression visitor"""
+ bound_expression = base.And(
+ base.BoundNotEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ base.BoundEqualTo[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=2, name="bar", field_type=StringType(), required=False),
+ accessor=Accessor(position=1, inner=None),
+ ),
+ literal=StringLiteral("foo"),
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["NOT_EQUAL", "EQUAL", "AND"]
+
+
+def test_bound_boolean_expression_visitor_always_true():
+ """Test visiting an And expression with a bound boolean expression visitor that resolves to AlwaysTrue"""
+ bound_expression = base.Or(
+ base.BoundIsNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ )
+ ),
+ base.BoundNotNaN[str](
+ term=base.BoundReference(
+ field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
+ accessor=Accessor(position=0, inner=None),
+ )
+ ),
+ )
+ visitor = FooBoundBooleanExpressionVisitor()
+ result = base.visit(bound_expression, visitor=visitor)
+ assert result == ["TRUE"]
+
+
+def test_bound_boolean_expression_visitor_always_false():
+ """Test visiting an And expression with a bound boolean expression visitor that resolves to AlwaysFalse"""
+ bound_expression = base.And(
Review Comment:
Agreed and udated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5303: Add a BoundBooleanExpressionVisitor for visiting bound expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5303:
URL: https://github.com/apache/iceberg/pull/5303#issuecomment-1221414862
@samredai, can you rebase?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org