[I] Raster session settings are re-applied from each query's config snapshot (io_concurrency, cache_max_bytes) [sedona-db]

["james-willis (via GitHub)" <gi...@apache.org>]

james-willis opened a new issue, #1483:
URL: https://github.com/apache/sedona-db/issues/1483

   `sedona.raster.io_concurrency` (#1385) and `sedona.raster.cache_max_bytes` are applied to shared session state at call time, from the `ConfigOptions` each call is invoked with: `io_budget_from_config` resizes the session's `RasterIoBudget` and `chunk_cache_from_config` resizes the session's chunk cache, so that a `SET` takes effect on the next `RS_FromPath` or `RS_EnsureLoaded` call.
   
   Each query runs with the options snapshot taken when it was planned. A query planned before a `SET` therefore re-applies the value it was planned with at its next call, and two queries running at once with different settings keep flipping the shared state between them until the older one finishes:
   
   1. Query A is planned with `io_concurrency = 64` (the default) and starts loading.
   2. `SET sedona.raster.io_concurrency = 1`.
   3. Query B is planned, and its first call resizes the budget to 1.
   4. Query A's next batch resizes it back to 64; B's next batch sets it to 1 again, and so on.
   
   The limit never exceeds the larger of the two values, so this is a wobble rather than a leak, but `SHOW sedona.raster.io_concurrency` can advertise a limit the session is not holding, and the same holds for the cache size.
   
   Found in the review of #1385 (@paleolimbot, via Astra).
   
   ## Proposed fix
   
   Apply at `SET` time, so snapshots never write. Make each option a handle type whose `ConfigField::set` resizes the shared budget or cache and whose `Display` prints the number, so `SHOW` and the docs generator keep working. `SedonaOptions` lives in `sedona-common` while the budget and cache live in `sedona-raster`, so either the budget moves down (it is a plain mutex and condvar counter with no dependencies, so it can) or `sedona-common` grows a small hook the raster crate registers into.
   
   An alternative that keeps the call-time apply: stamp each `SET` with a monotonically increasing version, and have `io_budget_from_config` and `chunk_cache_from_config` apply only a stamp newer than the one last applied.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org