[I] [BUG] STAC temporal OR pushdown can narrow unbounded predicates [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu opened a new issue, #3126:
URL: https://github.com/apache/sedona/issues/3126

   ## Description
   
   `StacUtils.getFilterTemporal` computes a bounding interval for compound temporal filters before sending the `datetime` parameter to a STAC catalog. When an `OrFilter` child has an unbounded endpoint, `calculateUnionTemporal` discards that unbounded endpoint and can emit a finite interval that is narrower than the original predicate.
   
   Because the Spark filter remains as a residual predicate, the remote interval must always be a superset. A narrower interval can drop matching STAC items before Spark evaluates the residual filter.
   
   This behavior predates #3122 and is not introduced by that PR.
   
   ## Reproducer
   
   ```scala
   val filter = TemporalFilter.OrFilter(
     TemporalFilter.GreaterThanFilter(
       "datetime",
       LocalDateTime.parse("2025-03-06T00:00:00")),
     TemporalFilter.LessThanFilter(
       "datetime",
       LocalDateTime.parse("2025-03-07T00:00:00")))
   
   StacUtils.getFilterTemporal(filter)
   ```
   
   The predicate is a tautology: every timestamp is either after March 6 or before March 7. It currently serializes to a finite interval similar to:
   
   ```text
   datetime=2025-03-06T00:00:00.000000000Z/2025-03-07T00:00:00.000000999Z
   ```
   
   The STAC catalog therefore excludes timestamps outside that interval even though the residual Spark predicate would retain them.
   
   Mixed `OR` expressions such as equality combined with a one-sided inequality have the same problem. The existing `getFilterTemporal with OrFilter` test currently codifies the unsafe finite interval.
   
   ## Expected behavior
   
   Temporal pushdown must not narrow the result set accepted by the residual Spark predicate.
   
   - Preserve an unbounded endpoint when the convex hull of an `OR` expression is unbounded.
   - Skip temporal pushdown when no useful safe interval can be produced, such as a tautology.
   - Do not emit reversed intervals for contradictory compound predicates; let the residual filter determine the empty result if necessary.
   
   ## Suggested tests
   
   - `datetime > a OR datetime < b` where `a < b`
   - equality combined with a one-sided inequality
   - disjoint bounded ranges whose safe convex hull is finite
   - contradictory compound ranges that could otherwise produce a reversed interval
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] STAC temporal OR pushdown can narrow unbounded predicates [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu closed issue #3126: [BUG] STAC temporal OR pushdown can narrow unbounded predicates
URL: https://github.com/apache/sedona/issues/3126


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org