You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/07/26 22:38:23 UTC
[GitHub] [iceberg] rdblue opened a new pull request, #5362: Python: Refactor unary expressions
rdblue opened a new pull request, #5362:
URL: https://github.com/apache/iceberg/pull/5362
This refactors unary expressions to use base classes that handle binding and inverting. In addition, this updates binding to produce `AlwaysTrue()` or `AlwaysFalse()` values based on the bound type. For example, `IsNull(a)` when bound to a required field a will produce `AlwaysFalse()` because the value is required.
This also adds missing tests for unary expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932531899
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
Review Comment:
If we make `__invert__` abstract, then we can't use a frozen `@dataclass` without duplicating a lot more.
The problem is, as you pointed out, `@dataclass` can't be abstract. If I move this to the child class, `IsNull`, then the constructor no longer supports `term` because it is defined here in the parent. Then we have to implement `__invert__` in each child class and define the `term` separately in each child class.
That basically removes the benefit of using a common base class. We can no longer rely on anything higher than the leaf class having a `term` or a set of `literals`.
I think I'd rather trade the mypy hints for `__invert__` than not be able to rely on inheritance for expressions. That is, if I have a `UnaryPredicate`, I want to be able to use its `term` instead of handling everything at the leaf level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932476485
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
Review Comment:
Good catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932817543
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,235 +243,321 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class BoundPredicate(Bound[T], BooleanExpression):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundPredicate[T]:
+ raise NotImplementedError
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+@dataclass(frozen=True)
+class UnboundPredicate(Unbound[T, BooleanExpression], BooleanExpression):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+ def __invert__(self) -> UnboundPredicate[T]:
+ raise NotImplementedError
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ raise NotImplementedError
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(UnboundPredicate[T]):
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
+
+ def __invert__(self) -> UnaryPredicate[T]:
+ raise NotImplementedError
+
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(BoundPredicate[T]):
+ def __invert__(self) -> BoundUnaryPredicate[T]:
+ raise NotImplementedError
+
+
+@dataclass(frozen=True)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
+
+ def __invert__(self) -> BoundNotNull[T]:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+@dataclass(frozen=True)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
Review Comment:
pylint for the 3.8 CI seems to not like that the argument signature is different between this and the default `super().__new__`. You can change this to match the `*args, **kwargs` from super, but then you'd have to pull out the kwargs and it won't look as clean plus won't allow positional arguments.
```py
@dataclass(frozen=True)
class BoundNotNull(BoundUnaryPredicate[T]):
def __new__(cls, *args, **kwargs):
term: BoundTerm[T] = kwargs["term"]
if term.ref().field.required:
return AlwaysTrue()
return super().__new__(cls)
```
This is just a linting error though and the assumptions made when using the super type hold well so I think it'd be better to just ignore it for these lines.
```py
@dataclass(frozen=True)
class BoundNotNull(BoundUnaryPredicate[T]):
def __new__(cls, term: BoundTerm[T]): # pylint: disable=W0611
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,235 +243,321 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class BoundPredicate(Bound[T], BooleanExpression):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundPredicate[T]:
+ raise NotImplementedError
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+@dataclass(frozen=True)
+class UnboundPredicate(Unbound[T, BooleanExpression], BooleanExpression):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+ def __invert__(self) -> UnboundPredicate[T]:
+ raise NotImplementedError
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ raise NotImplementedError
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(UnboundPredicate[T]):
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
Review Comment:
Should this return type be `BoundUnaryPredicate`? Same question for `SetPredicate.bind(...)` and `LiteralPredicate.bind(...)` which can have return types of `BoundSetPredicate` and `BoundLiteralPredicate`.
##########
python/tests/expressions/test_expressions_base.py:
##########
@@ -139,210 +217,138 @@ def test_strs(op, string):
assert str(op) == string
+def test_ref_binding_case_sensitive(request):
+ schema = request.getfixturevalue("table_schema_simple")
+ ref = base.Reference("foo")
+ bound = base.BoundReference(schema.find_field(1), schema.accessor_for_field(1))
+ assert ref.bind(schema, case_sensitive=True) == bound
+
+
+def test_ref_binding_case_sensitive_failure(request):
+ schema = request.getfixturevalue("table_schema_simple")
+ ref = base.Reference("Foo")
+ with pytest.raises(ValueError):
+ ref.bind(schema, case_sensitive=True)
+
+
+def test_ref_binding_case_insensitive(request):
+ schema = request.getfixturevalue("table_schema_simple")
+ ref = base.Reference("Foo")
+ bound = base.BoundReference(schema.find_field(1), schema.accessor_for_field(1))
+ assert ref.bind(schema, case_sensitive=False) == bound
+
+
+def test_ref_binding_case_insensitive_failure(request):
+ schema = request.getfixturevalue("table_schema_simple")
+ ref = base.Reference("Foot")
+ with pytest.raises(ValueError):
+ ref.bind(schema, case_sensitive=False)
+
+
+def test_in_to_eq():
+ assert base.In(base.Reference("x"), (literal(34.56),)) == base.EqualTo(base.Reference("x"), literal(34.56))
+
+
+def test_bind_in(request):
+ schema = request.getfixturevalue("table_schema_simple")
+ bound = base.BoundIn(
+ base.BoundReference(schema.find_field(1), schema.accessor_for_field(1)), {literal("hello"), literal("world")}
+ )
+ assert base.In(base.Reference("foo"), (literal("hello"), literal("world"))).bind(schema) == bound
+
+
+def test_bind_dedup(request):
+ schema = request.getfixturevalue("table_schema_simple")
Review Comment:
It's definitely not required but instead of using `request` here you can just use the fixture name and pytest will automatically grab it for you.
```py
def test_bind_dedup(table_schema_simple):
bound = base.BoundIn(
base.BoundReference(table_schema_simple.find_field(1), table_schema_simple.accessor_for_field(1)), {literal("hello"), literal("world")}
)
assert base.In(base.Reference("foo"), (literal("hello"), literal("world"), literal("world"))).bind(table_schema_simple) == bound
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r933492649
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,235 +243,321 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class BoundPredicate(Bound[T], BooleanExpression):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundPredicate[T]:
+ raise NotImplementedError
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+@dataclass(frozen=True)
+class UnboundPredicate(Unbound[T, BooleanExpression], BooleanExpression):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+ def __invert__(self) -> UnboundPredicate[T]:
+ raise NotImplementedError
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ raise NotImplementedError
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(UnboundPredicate[T]):
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
Review Comment:
Binding may produce a different type of expression. For example, if `IsNull` is bound to a required type, binding will return `AlwaysFalse`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r931808275
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
Review Comment:
This will actually mutate the `literals`, and turn it into an empty set.
```suggestion
return BoundEq(term, next(iter(literals)))
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
+ return BoundNotEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __invert__(self) -> BooleanExpression:
+ return BoundIn(self.term, self.literals)
-class BoundNotIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundNotIn must contain at least 1 literal.")
+class In(SetPredicate[T]):
+ as_bound = BoundIn
- def __invert__(self) -> BoundIn[T]:
- return BoundIn(self.term, *self.literals)
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
Review Comment:
```suggestion
elif count == 1:
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
Review Comment:
```suggestion
elif count == 1:
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
Review Comment:
I like how this is done, by checking this in the `__new__` method.
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
+ return BoundNotEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __invert__(self) -> BooleanExpression:
+ return BoundIn(self.term, self.literals)
-class BoundNotIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundNotIn must contain at least 1 literal.")
+class In(SetPredicate[T]):
+ as_bound = BoundIn
- def __invert__(self) -> BoundIn[T]:
- return BoundIn(self.term, *self.literals)
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
Review Comment:
How about accumulating the positional arguments like:
```suggestion
def __new__(cls, term: UnboundTerm[T], *literals: Literal[T]) -> BooleanExpression:
```
This way we can write:
```python
def test_in_to_eq():
assert base.In(
base.Reference("x"),
literal(34.56),
literal(19.25)
) == base.Eq(base.Reference("x"), literal(34.56))
```
instead of:
```python
def test_in_to_eq():
assert base.In(
base.Reference("x"),
(literal(34.56), literal(19.25))
) == base.Eq(base.Reference("x"), literal(34.56))
```
I think this makes the API a bit friendlier and less verbose.
You can easily expand the arguments when you want to supply a list:
```python
class NotIn(SetPredicate[T]):
as_bound = BoundNotIn
...
def __invert__(self) -> BooleanExpression:
return In(self.term, *self.literals)
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
+ return BoundNotEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __invert__(self) -> BooleanExpression:
+ return BoundIn(self.term, self.literals)
-class BoundNotIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundNotIn must contain at least 1 literal.")
+class In(SetPredicate[T]):
+ as_bound = BoundIn
- def __invert__(self) -> BoundIn[T]:
- return BoundIn(self.term, *self.literals)
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return Eq(term, literals[0])
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return NotIn(self.term, self.literals)
-class NotIn(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("NotIn must contain at least 1 literal.")
- def __invert__(self) -> In[T]:
- return In(self.term, *self.literals)
+class NotIn(SetPredicate[T]):
+ as_bound = BoundNotIn
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
Review Comment:
```suggestion
elif count == 1:
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
Review Comment:
We could also consider wrapping it into positional arguments:
```python
class BoundIn(BoundSetPredicate[T]):
def __new__(cls, term: BoundTerm[T], *literals: Literal[T]) -> BooleanExpression:
literals_set = set(literals)
count = len(literals)
if count == 0:
return AlwaysFalse()
if count == 1:
return BoundEq(term, literals.pop())
else:
return super().__new__(cls)
def __invert__(self) -> BooleanExpression:
return BoundNotIn(self.term, self.literals)
```
This way we also don't have to care about mutating the set. See comment below 👍🏻
This way we don't have to pass in a set:
```python
BoundIn[str](
base.BoundReference(
field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
accessor=Accessor(position=0, inner=None),
),
StringLiteral("foo"), StringLiteral("bar"), StringLiteral("baz"),
)
```
Instead of:
```python
BoundIn[str](
base.BoundReference(
field=NestedField(field_id=1, name="foo", field_type=StringType(), required=False),
accessor=Accessor(position=0, inner=None),
),
{StringLiteral("foo"), StringLiteral("bar"), StringLiteral("baz")},
)
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
Review Comment:
```suggestion
elif count == 1:
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
+ return BoundNotEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __invert__(self) -> BooleanExpression:
+ return BoundIn(self.term, self.literals)
-class BoundNotIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundNotIn must contain at least 1 literal.")
+class In(SetPredicate[T]):
+ as_bound = BoundIn
- def __invert__(self) -> BoundIn[T]:
- return BoundIn(self.term, *self.literals)
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return Eq(term, literals[0])
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return NotIn(self.term, self.literals)
-class NotIn(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("NotIn must contain at least 1 literal.")
- def __invert__(self) -> In[T]:
- return In(self.term, *self.literals)
+class NotIn(SetPredicate[T]):
+ as_bound = BoundNotIn
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
Review Comment:
```suggestion
def __new__(cls, term: UnboundTerm[T], literals: *Literal[T]) -> BooleanExpression:
```
See the comment at `class In(SetPredicate[T]):`
##########
python/pyiceberg/expressions/base.py:
##########
@@ -19,58 +19,22 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from functools import reduce, singledispatch
-from typing import Generic, TypeVar
+from typing import (
+ ClassVar,
+ Generic,
+ Literal,
Review Comment:
I think we want to import our own literal from:
```python
from pyiceberg.expressions.literals import Literal
```
instead of the one from typing.
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
Review Comment:
An abstract class should not be a dataclass as well, since it cannot be instantiated, it does not carry any data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r931452949
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
Review Comment:
With the other refactoring, I think it was reasonable to put the `__invert__` implementations back in since they're one-line changes. Thanks, @Fokko!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932505874
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
Review Comment:
I'm actually not sure about this. There are mypy errors if you declare the return type from `__new__` to be correct. This is from `In.__new__` for example:
```
451: error: Incompatible return type for "__new__" (returns "BooleanExpression", but must return a subtype of "BoundIn[Any]")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932538475
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
Review Comment:
I solved this by implementing `__invert__` and raising `NotImplementedError`. It's overridden in subclasses just like before, but now `@dataclass` works because these aren't abstract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r931445511
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
-class BoundIsNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNaN:
- return BoundNotNaN(self.term)
+class IsNaN(UnaryPredicate[T]):
+ pass
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+class NotNaN(UnaryPredicate[T]):
+ pass
-class NotNaN(UnboundPredicate[T]):
- def __invert__(self) -> IsNaN:
- return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
Review Comment:
Done.
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
-class BoundIsNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNaN:
- return BoundNotNaN(self.term)
+class IsNaN(UnaryPredicate[T]):
+ pass
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+class NotNaN(UnaryPredicate[T]):
+ pass
-class NotNaN(UnboundPredicate[T]):
- def __invert__(self) -> IsNaN:
- return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
+ return super().__new__(cls)
+ return AlwaysFalse()
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932479836
##########
python/pyiceberg/expressions/base.py:
##########
@@ -19,58 +19,22 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from functools import reduce, singledispatch
-from typing import Generic, TypeVar
+from typing import (
+ ClassVar,
+ Generic,
+ Literal,
Review Comment:
Ah, I was wondering what the problem was. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r930469392
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
Review Comment:
This was my main goal because I think it is easy to validate that the behavior is correct. Reimplementing `__inverse__` etc. in every class seemed like it was going to cause trouble later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r930724220
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
-class BoundIsNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNaN:
- return BoundNotNaN(self.term)
+class IsNaN(UnaryPredicate[T]):
+ pass
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+class NotNaN(UnaryPredicate[T]):
+ pass
-class NotNaN(UnboundPredicate[T]):
- def __invert__(self) -> IsNaN:
- return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
Review Comment:
This way we don't have to fetch the actual `{Float,Double}Type`.
```suggestion
if type(bound_type) in {FloatType, DoubleType}:
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
Review Comment:
I like it that we only store the types. I'm unsure if reimplementing would cause issues. I would probably override the `__invert__` method in `UnaryPredicate`:
```python
@abstractmethod
def __invert__(self) -> BooleanExpression:
pass
```
The return type would then change to `BooleanExpression` as it can return a `UnaryPredicate`, but also a `Always{True,False}`. This is nice because it gives us typing and we don't flatten everything to `type`:
And then implement them in the function itself. This keeps everything in one place, and we make sure that we implement all the methods:
By pytest:
```python
def test_isnull_bind_required():
schema = Schema(NestedField(2, "a", IntegerType(), required=True), schema_id=1)
> assert base.IsNull(base.Reference("a")).bind(schema) == base.AlwaysFalse()
E TypeError: Can't instantiate abstract class IsNull with abstract method __invert__
```
Also by static analysis:
```python
python/pyiceberg/expressions/base.py:393: error: Cannot instantiate abstract class "NotNull" with abstract attribute "__invert__"
```
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
-
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ inverse: ClassVar[type]
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression):
+ inverse: ClassVar[type]
+ term: BoundTerm[T]
+ def __invert__(self) -> inverse:
+ return self.inverse(self.term)
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ pass
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ pass
-class BoundNotNull(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNull:
- return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+IsNull.inverse = NotNull
+IsNull.as_bound = BoundIsNull
+NotNull.inverse = IsNull
+NotNull.as_bound = BoundNotNull
+BoundIsNull.inverse = BoundNotNull
+BoundNotNull.inverse = BoundIsNull
-class BoundIsNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNaN:
- return BoundNotNaN(self.term)
+class IsNaN(UnaryPredicate[T]):
+ pass
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+class NotNaN(UnaryPredicate[T]):
+ pass
-class NotNaN(UnboundPredicate[T]):
- def __invert__(self) -> IsNaN:
- return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
+ return super().__new__(cls)
+ return AlwaysFalse()
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if bound_type == FloatType() or bound_type == DoubleType():
Review Comment:
```suggestion
if type(bound_type) in {FloatType, DoubleType}:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932479543
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysFalse()
-class BoundIsNaN(BoundPredicate[T]):
def __invert__(self) -> BoundNotNaN:
return BoundNotNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
+
+class BoundNotNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
+ if type(bound_type) in {FloatType, DoubleType}:
+ return super().__new__(cls)
+ return AlwaysTrue()
+
+ def __invert__(self) -> BoundIsNaN:
+ return BoundIsNaN(self.term)
-class NotNaN(UnboundPredicate[T]):
+class IsNaN(UnaryPredicate[T]):
+ as_bound = BoundIsNaN
+
+ def __invert__(self) -> NotNaN:
+ return NotNaN(self.term)
+
+
+class NotNaN(UnaryPredicate[T]):
+ as_bound = BoundNotNaN
+
def __invert__(self) -> IsNaN:
return IsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNaN(bound_ref)
+@dataclass(frozen=True)
+class SetPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
+ literals: tuple[Literal[T], ...]
+
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term, {lit.to(bound_term.ref().field.field_type) for lit in self.literals})
-class BoundNotNaN(BoundPredicate[T]):
- def __invert__(self) -> BoundIsNaN:
- return BoundIsNaN(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNaN is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundSetPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
+ literals: set[Literal[T]]
+ @abstractmethod
+ def __invert__(self) -> BooleanExpression:
+ ...
-class BoundIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundIn must contain at least 1 literal.")
- def __invert__(self) -> BoundNotIn[T]:
- return BoundNotIn(self.term, *self.literals)
+class BoundIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysFalse()
+ if count == 1:
+ return BoundEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
+ def __invert__(self) -> BooleanExpression:
+ return BoundNotIn(self.term, self.literals)
-class In(UnboundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("In must contain at least 1 literal.")
- def __invert__(self) -> NotIn[T]:
- return NotIn(self.term, *self.literals)
+class BoundNotIn(BoundSetPredicate[T]):
+ def __new__(cls, term: BoundTerm[T], literals: set[Literal[T]]) -> BooleanExpression:
+ count = len(literals)
+ if count == 0:
+ return AlwaysTrue()
+ if count == 1:
+ return BoundNotEq(term, literals.pop())
+ else:
+ return super().__new__(cls)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIn[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIn(bound_ref, *tuple(lit.to(bound_ref.field.field_type) for lit in self.literals)) # type: ignore
+ def __invert__(self) -> BooleanExpression:
+ return BoundIn(self.term, self.literals)
-class BoundNotIn(BoundPredicate[T]):
- def _validate_literals(self): # pylint: disable=W0238
- if not self.literals:
- raise AttributeError("BoundNotIn must contain at least 1 literal.")
+class In(SetPredicate[T]):
+ as_bound = BoundIn
- def __invert__(self) -> BoundIn[T]:
- return BoundIn(self.term, *self.literals)
+ def __new__(cls, term: UnboundTerm[T], literals: tuple[Literal[T], ...]) -> BooleanExpression:
Review Comment:
I wanted to do this, but I didn't know how to make it work with `@dataclass`. I looked it up and saw that you have to implement `__init__`. But maybe we don't want to use `@dataclass` anyway? It seems to have strange mypy errors:
```
327: error: Only concrete class can be given where "Type[UnaryPredicate[T]]" is expected
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] danielcweeks merged pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
danielcweeks merged PR #5362:
URL: https://github.com/apache/iceberg/pull/5362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932594200
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,151 +313,197 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(Unbound[T, BooleanExpression], BooleanExpression, ABC):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> UnaryPredicate:
+ ...
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+@dataclass(frozen=True)
+class BoundUnaryPredicate(Bound[T], BooleanExpression, ABC):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ @abstractmethod
+ def __invert__(self) -> BoundUnaryPredicate:
+ ...
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundNotNull:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysTrue()
+ return super().__new__(cls)
-class BoundNotNull(BoundPredicate[T]):
def __invert__(self) -> BoundIsNull:
return BoundIsNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+class IsNull(UnaryPredicate[T]):
+ as_bound = BoundIsNull
-class IsNaN(UnboundPredicate[T]):
- def __invert__(self) -> NotNaN:
- return NotNaN(self.term)
+ def __invert__(self) -> NotNull:
+ return NotNull(self.term)
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("IsNaN is a unary predicate and takes no Literals.")
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNaN[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNaN(bound_ref)
+class NotNull(UnaryPredicate[T]):
+ as_bound = BoundNotNull
+
+ def __invert__(self) -> IsNull:
+ return IsNull(self.term)
+
+class BoundIsNaN(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ bound_type = term.ref().field.field_type
Review Comment:
Yes, the `__new__` expects to return to the class it belongs to. I think we can ignore this one because I like the trick of returning a different type very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r930468732
##########
python/pyiceberg/expressions/base.py:
##########
@@ -204,6 +210,7 @@ def __eq__(self, other) -> bool:
)
+# TODO: this should still exist as a parent, but term should be abstract
Review Comment:
The `UnaryPredicate` and `BoundUnaryPredicate` classes introduced in this PR should implement the `UnboundPredicate` and `BoundPredicate` interfaces, but I didn't want to refactor the entire set of expressions in this PR so I added this TODO as a reminder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r930468840
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,107 +349,104 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
Review Comment:
These aren't actually abstract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r933499516
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,235 +243,321 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class BoundPredicate(Bound[T], BooleanExpression):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundPredicate[T]:
+ raise NotImplementedError
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+@dataclass(frozen=True)
+class UnboundPredicate(Unbound[T, BooleanExpression], BooleanExpression):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+ def __invert__(self) -> UnboundPredicate[T]:
+ raise NotImplementedError
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ raise NotImplementedError
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(UnboundPredicate[T]):
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
+
+ def __invert__(self) -> UnaryPredicate[T]:
+ raise NotImplementedError
+
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(BoundPredicate[T]):
+ def __invert__(self) -> BoundUnaryPredicate[T]:
+ raise NotImplementedError
+
+
+@dataclass(frozen=True)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
+
+ def __invert__(self) -> BoundNotNull[T]:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+@dataclass(frozen=True)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
Review Comment:
Thanks! I suppressed the false positives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] samredai commented on a diff in pull request #5362: Python: Refactor unary and set expressions
Posted by GitBox <gi...@apache.org>.
samredai commented on code in PR #5362:
URL: https://github.com/apache/iceberg/pull/5362#discussion_r932817543
##########
python/pyiceberg/expressions/base.py:
##########
@@ -342,235 +243,321 @@ def __str__(self) -> str:
@dataclass(frozen=True)
-class AlwaysTrue(BooleanExpression, ABC, Singleton):
+class AlwaysTrue(BooleanExpression, Singleton):
"""TRUE expression"""
def __invert__(self) -> AlwaysFalse:
return AlwaysFalse()
@dataclass(frozen=True)
-class AlwaysFalse(BooleanExpression, ABC, Singleton):
+class AlwaysFalse(BooleanExpression, Singleton):
"""FALSE expression"""
def __invert__(self) -> AlwaysTrue:
return AlwaysTrue()
-class IsNull(UnboundPredicate[T]):
- def __invert__(self) -> NotNull:
- return NotNull(self.term)
+@dataclass(frozen=True)
+class BoundPredicate(Bound[T], BooleanExpression):
+ term: BoundTerm[T]
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals is not None:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def __invert__(self) -> BoundPredicate[T]:
+ raise NotImplementedError
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundIsNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundIsNull(bound_ref)
+@dataclass(frozen=True)
+class UnboundPredicate(Unbound[T, BooleanExpression], BooleanExpression):
+ as_bound: ClassVar[type]
+ term: UnboundTerm[T]
-class BoundIsNull(BoundPredicate[T]):
- def __invert__(self) -> BoundNotNull:
- return BoundNotNull(self.term)
+ def __invert__(self) -> UnboundPredicate[T]:
+ raise NotImplementedError
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("Null is a unary predicate and takes no Literals.")
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ raise NotImplementedError
-class NotNull(UnboundPredicate[T]):
- def __invert__(self) -> IsNull:
- return IsNull(self.term)
+@dataclass(frozen=True)
+class UnaryPredicate(UnboundPredicate[T]):
+ def bind(self, schema: Schema, case_sensitive: bool = True) -> BooleanExpression:
+ bound_term = self.term.bind(schema, case_sensitive)
+ return self.as_bound(bound_term)
+
+ def __invert__(self) -> UnaryPredicate[T]:
+ raise NotImplementedError
+
- def _validate_literals(self): # pylint: disable=W0238
- if self.literals:
- raise AttributeError("NotNull is a unary predicate and takes no Literals.")
+@dataclass(frozen=True)
+class BoundUnaryPredicate(BoundPredicate[T]):
+ def __invert__(self) -> BoundUnaryPredicate[T]:
+ raise NotImplementedError
+
+
+@dataclass(frozen=True)
+class BoundIsNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
+ if term.ref().field.required:
+ return AlwaysFalse()
+ return super().__new__(cls)
+
+ def __invert__(self) -> BoundNotNull[T]:
+ return BoundNotNull(self.term)
- def bind(self, schema: Schema, case_sensitive: bool) -> BoundNotNull[T]:
- bound_ref = self.term.bind(schema, case_sensitive)
- return BoundNotNull(bound_ref)
+@dataclass(frozen=True)
+class BoundNotNull(BoundUnaryPredicate[T]):
+ def __new__(cls, term: BoundTerm[T]):
Review Comment:
pylint for the 3.8 CI seems to not like that the argument signature is different between this and the default `super().__new__`. You can change this to match the `*args, **kwargs` from super, but then you'd have to pull out the kwargs and it won't look as clean plus won't allow positional arguments.
```py
@dataclass(frozen=True)
class BoundNotNull(BoundUnaryPredicate[T]):
def __new__(cls, *args, **kwargs):
term: BoundTerm[T] = kwargs["term"]
if term.ref().field.required:
return AlwaysTrue()
return super().__new__(cls)
```
This is just a linting error though and the assumptions made when using the super type hold well so I think it'd be better to just ignore it for these lines.
```py
@dataclass(frozen=True)
class BoundNotNull(BoundUnaryPredicate[T]):
def __new__(cls, term: BoundTerm[T]): # pylint: disable=W0221
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org