Re: [PR] Spec: Add finer grained read restrictions as part of loadTable [iceberg]

["GGraziadei (via GitHub)" <gi...@apache.org>]

GGraziadei commented on code in PR #13879:
URL: https://github.com/apache/iceberg/pull/13879#discussion_r3898662340


##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3839,6 +3849,315 @@ components:
           additionalProperties:
             type: string
 
+    ReadRestrictions:
+      type: object
+      description: >
+          Read restrictions for a table.
+
+          A reader evaluates the row filter against original, untransformed column
+          values, then applies required-column-projections to the surviving rows.
+          Each action must produce a value of the same type as the input column.
+          If a reader that supports read-restrictions cannot apply any returned
+          restriction (a filter expression or an action), it must fail the query
+          and must not silently return raw, partial, or empty results.
+
+          A server must not return projections on both a nested-typed field
+          (struct, list, or map) and any field-id nested within it at any
+          depth. A reader that receives such a response must fail the query.
+
+          A missing or empty ReadRestrictions object (no required-column-projections and no
+          required-row-filter) imposes no restrictions.
+      example:
+        required-column-projections:
+          - field-id: 4
+            action: show-last-4
+          - field-id: 6
+            action: replace-with-null
+          - field-id: 8
+            action: truncate-to-year
+          - field-id: 10
+            action: sha-256-global
+          - field-id: 12
+            action: mask-alphanum
+        required-row-filter:
+          type: eq
+          left:
+            type: reference
+            id: 14
+          right:
+            type: literal
+            value: "US"
+      properties:
+        required-column-projections:
+          description: >
+            A list of columns that require specific actions to be applied when reading.
+            A server must not return an action for a column whose type is not listed in
+            that action's "Applicable to" set. If absent or empty, no required actions
+            apply; columns not listed are not subject to any required action.
+
+            1. For each column listed, the reader must apply the specified action before
+              returning values for that column.
+
+            2. The reader must replace all output references to the column with the result
+              of the action, presenting the result under the original field-id. For
+              example, if the action for field-id `9` is mask-alphanum, the reader must
+              return the masked value as field-id `9` in the query output.
+
+            3. A server must not return more than one projection for the same field-id
+              in required-column-projections. If a duplicate field-id appears, the reader
+              must fail the query.
+
+            4. A projection must not target a map's key field-id. Applying an action
+              to keys can produce duplicate or null keys, which readers silently
+              coalesce or reject, causing data loss.
+
+            5. A reader must enforce projections on the columns it is actually reading.
+              Projections referencing columns that are not being read do not apply.
+          type: array
+          items:
+            $ref: '#/components/schemas/Action'
+        required-row-filter:
+          description: >
+            An expression that limits which rows the reader may return.
+
+            1. The expression must evaluate to a boolean (TRUE or FALSE; Iceberg predicates
+              never produce NULL). A reader must discard any row for which the filter
+              evaluates to FALSE, and no information derived from discarded rows may be
+              included in the query result.
+
+            2. If this property is absent, null, or always true then no mandatory filtering is required.
+
+            3. Column references within the expression must use field IDs (IdReference),
+              not column names. This ensures the filter remains valid across column renames,
+              consistent with required-column-projections which also reference columns by field-id.
+          allOf:
+            - $ref: '#/components/schemas/Predicate'
+
+    Action:
+      discriminator:
+        propertyName: action
+        mapping:
+          mask-alphanum: '#/components/schemas/MaskAlphanum'
+          mask-to-fixed-value: '#/components/schemas/MaskToFixedValue'
+          replace-with-null: '#/components/schemas/ReplaceWithNull'
+          show-first-4: '#/components/schemas/ShowFirst4'
+          show-last-4: '#/components/schemas/ShowLast4'
+          truncate-to-year: '#/components/schemas/TruncateToYear'
+          truncate-to-month: '#/components/schemas/TruncateToMonth'
+          sha-256-global: '#/components/schemas/Sha256Global'
+          sha-256-query-local: '#/components/schemas/Sha256QueryLocal'
+      type: object
+      required:
+        - action
+        - field-id
+      properties:
+        action:
+          type: string
+        field-id:
+          type: integer
+          description: Field ID of the column being projected.
+
+    MaskAlphanum:
+      description: >
+        Redacts the column value using the following rules to transform Unicode code points:
+
+        - Digits (U+0030–U+0039, 0-9) are replaced with 'n'
+        - The following punctuation characters are kept as-is:
+            U+0028 '('  LEFT PARENTHESIS
+            U+0029 ')'  RIGHT PARENTHESIS
+            U+002C ','  COMMA
+            U+002E '.'  FULL STOP
+            U+002D '-'  HYPHEN-MINUS
+            U+0040 '@'  COMMERCIAL AT
+        - All other Unicode characters (including letters, whitespace, and any punctuation
+          not listed above) are replaced with 'x'
+
+        For example: "iceberg16112018@apache.org" -> "xxxxxxxnnnnnnnn@xxxxxx.xxx"
+
+        NULL input is preserved (NULL -> NULL).
+
+        Applicable to: string
+      allOf:
+        - $ref: '#/components/schemas/Action'
+      properties:
+        action:
+          type: string
+          const: "mask-alphanum"
+
+    MaskToFixedValue:
+      description: >
+        Replaces the column value with a type-specific fixed value.
+        Readers must use exactly the values listed below to ensure consistency

Review Comment:
   Just a nit: consider to use `|` instead of `>`
   Two different tokens with different meaning:
   `|`  newlines are preserved    
   `>`  newlines are folded into spaces, so the bullet points get merged into one line (visible in the generated .py) .
   
   Fix only if you have bandwith :-); 
   Please repeat for all description with a list. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org