[PR] Persistence: DynamoDB (atomic) entity writes, CAS and name uniqueness [polaris]

["ZephyrYWZhou (via GitHub)" <gi...@apache.org>]

ZephyrYWZhou opened a new pull request, #5892:
URL: https://github.com/apache/polaris/pull/5892

   ## Summary
   
   Next step of the optional DynamoDB persistence backend (#5615), building on the skeleton (#5622), the single-table data model (#5707) and the schema + test harness (#5830). This PR adds the first `BasePersistence` implementation: **entity writes, per-entity compare-and-set, and name uniqueness**, plus the by-id and by-name lookups.
   
   Still no runtime wiring, so the backend cannot be selected yet. Operations not covered here throw `UnsupportedOperationException` and land in follow-ups.
   
   ## Design
   
   `DynamoDbBasePersistence` runs under the existing `AtomicOperationMetaStoreManager`. Every correctness decision rides a strongly consistent single-item read or a conditional write:
   
   - **Create** — one `TransactWriteItems` puts the entity item and its name sentinel, both `attribute_not_exists`, so ids and names stay unique.
   - **Update** — conditional `PutItem`, CAS on both `entityVersion` and `grantRecordsVersion` (grant changes bump only the latter, same as JDBC).
   - **Rename / re-parent** — one transaction: CAS the entity, delete the old sentinel, claim the new one.
   - **Delete** — one transaction: delete the entity (version-checked) and its sentinel.
   - **Lookups** — strongly consistent `GetItem` / `BatchGetItem`; by-name goes through the sentinel, never the GSI.
   
   Failures map to the exceptions the manager already handles:
   
   - sentinel condition failed → `EntityAlreadyExistsException` (name collision)
   - entity-id condition failed → `EntityAlreadyExistsException` with the same id (idempotent create retry)
   - version condition failed → `RetryOnConcurrencyException`
   
   A few edge cases are handled explicitly:
   
   - The manager passes `nameOrParentChanged=true` on some updates where the name is unchanged; that stays a plain update rather than deleting and re-putting the same sentinel.
   - A sentinel is only deleted if it still points at this entity, so a delete with a stale copy cannot remove a name another entity now owns.
   - Lookups filter on catalog id and type code, matching JDBC.
   
   ## Changes
   
   - `DynamoDbBasePersistence` — entity writes and lookups as above.
   - `IdGenerator` — positive 63-bit random ids.
   - `TransactionFailures` — reads per-action cancellation reasons of a `TransactWriteItems`.
   - `DynamoDbBasePersistenceIT` — create/read, stale-version rejection, duplicate name, idempotent create, rename and re-parent, rename onto a taken name, delete, and the stale-delete case, against the Floci harness.
   - `TransactionFailuresTest` — unit tests for the cancellation-reason mapping.
   - `build.gradle.kts` — `compileOnly(libs.jspecify)`, as in `relational-jdbc`.
   - `package-info.java` — drop the outdated "skeleton only" note.
   
   ## Testing
   
   `./gradlew :polaris-persistence-dynamodb-atomic:spotlessCheck checkstyleMain checkstyleTest checkstyleIntTest compileIntTestJava test javadoc` passes. `DynamoDbBasePersistenceIT` needs Docker; it was not run in my local environment and is expected to run in CI.
   
   ## Scope / follow-ups
   
   Next: `listEntities` / `hasChildren` (by-name GSI + page token), then `writeEntities`, then grant records with the by-grantee GSI, then principal secrets, policy mappings and the shared `BasePolarisMetaStoreManagerTest`, and finally the factory / CDI / config wiring.
   
   ## Checklist
   - [x] 🛡 Don't disclose security issues! (contact security@apache.org)
   - [x] 🔗 Clearly explained why the changes are needed, or linked related issues: part of #5615, builds on #5622, #5707 and #5830
   - [x] 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
   - [x] 💡 Added comments for complex logic
   - [x] 🧾 Updated `CHANGELOG.md` (if needed)
   - [x] 📚 Updated documentation in `site/content/in-dev/unreleased` (if needed)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org