You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "fallintoplace (via GitHub)" <gi...@apache.org> on 2026/07/28 17:04:07 UTC

[PR] fix(sql): validate view identifiers at API boundaries [iceberg-go]

fallintoplace opened a new pull request, #1571:
URL: https://github.com/apache/iceberg-go/pull/1571

   ## What changed
   
   Validate view identifiers before SQL catalog view operations inspect namespaces, names, or storage locations.
   
   ## Why
   
   SQL view operations skipped the shared view identifier validation used by other catalogs. Invalid names such as `..` or `nested/view` could reach location resolution, where path normalization changes their meaning.
   
   Validation now happens before database or filesystem work for create, load, drop, and existence checks.
   
   ## Testing
   
   - `go test ./catalog/sql`
   - `go vet ./catalog/sql`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] fix(sql): validate view identifiers at API boundaries [iceberg-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #1571:
URL: https://github.com/apache/iceberg-go/pull/1571


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] fix(sql): validate view identifiers at API boundaries [iceberg-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #1571:
URL: https://github.com/apache/iceberg-go/pull/1571#discussion_r3732123232


##########
catalog/sql/sql.go:
##########
@@ -1717,6 +1725,10 @@ func (c *Catalog) DropView(ctx context.Context, identifier table.Identifier) err
 		return errViewsUnsupportedOnV0
 	}
 
+	if err := catalog.ValidateViewIdentifier(identifier); err != nil {

Review Comment:
   **Blocking — this can make existing rows permanently unremovable through the API.**
   
   `CreateView` previously had no validation at all, so views with identifiers like `a/b`, `.`, or embedded control characters can already exist in `iceberg_tables` in deployed catalogs. Adding validation to `DropView` means those rows now fail validation on the way *out* — the only API path that could remove them. The user is left with a view they can neither load nor drop, recoverable only by direct SQL against the catalog tables.
   
   This is also asymmetric with tables, which have always been validated on create via `createStagedTable` (`catalog/internal/utils.go:128`), so no comparable pool of invalid table rows can exist.
   
   **Suggested fix:** exempt `DropView` from component validation — keeping only a length/shape check sufficient to build the query — so a bad row can always be cleaned up. If you'd rather keep the guard for consistency, the PR needs a documented migration note telling operators how to remove pre-existing invalid views.



##########
catalog/sql/sql.go:
##########
@@ -1672,6 +1676,10 @@ func (c *Catalog) listViewsAll(ctx context.Context, namespace table.Identifier)
 		return nil, nil
 	}
 
+	if err := checkValidNamespace(namespace); err != nil {

Review Comment:
   **Blocking — this guard is not the validator the PR is about, and the test covering it passes for an unrelated reason.**
   
   `checkValidNamespace` here is the *local* helper at `sql.go:736`, which only checks `len(ident) < 1`. It is not `catalog.ValidateNamespaceIdentifier`, so no component rules — no empty levels, no `.`/`..`, no control characters — are applied on this path.
   
   The `{".."}` test case passes off the pre-existing not-found path, not off this guard: `..` has length 1, so `checkValidNamespace` returns nil, and the error comes from `resolveNamespaceKey` below failing to find the namespace. **The test stays green if you delete this guard entirely** — worth confirming locally, it's a quick check.
   
   What the line *does* change is real but unintended: `ListViews(ctx, nil)` now errors where it previously listed root-namespace views. That also diverges from `listTablesAll` (`sql.go:1387`), which has no such guard, so listing tables and listing views in the same catalog now behave differently for a nil namespace.
   
   **Suggested fix:** either use `catalog.ValidateNamespaceIdentifier` here *and* in `listTablesAll` so the two agree and the guard actually validates something, or drop this line and let the not-found path handle it as before. Either way the `{".."}` test needs an assertion that fails when the guard is removed.



##########
catalog/catalog.go:
##########
@@ -249,12 +249,7 @@ func NamespaceFromIdent(ident table.Identifier) table.Identifier {
 	return ident[:len(ident)-1]
 }
 
-func validateIdentifier(ident table.Identifier, notFoundErr error) error {
-	if len(ident) < 2 {
-		return fmt.Errorf("%w: missing namespace or invalid identifier %v",
-			notFoundErr, strings.Join(ident, "."))
-	}
-
+func validateIdentifierComponents(ident table.Identifier, notFoundErr error) error {

Review Comment:
   This extraction leaves `validateIdentifierComponents` with exactly one caller — `validateIdentifier` on line 276 — and it was previously inline there. The new `ValidateNamespaceIdentifier` deliberately does *not* use it, since namespaces are allowed component shapes that tables and views are not.
   
   So the extraction is scaffolding from an earlier design in which namespaces reused the component rules, and nothing in the shipped code depends on it. #1614 carries the same leftover, and reusing these rules for namespaces is specifically what the maintainer pushed back on there.
   
   **Suggested fix:** drop the extraction and inline the loop back into `validateIdentifier`. It removes a package-level function that reads as shared infrastructure but has no second consumer, and shrinks the conflict surface with #1614.



##########
catalog/sql/sql.go:
##########
@@ -1233,7 +1233,7 @@ func (c *Catalog) CheckTableExists(ctx context.Context, identifier table.Identif
 }
 
 func (c *Catalog) CreateNamespace(ctx context.Context, namespace table.Identifier, props iceberg.Properties) error {
-	if err := checkValidNamespace(namespace); err != nil {
+	if err := catalog.ValidateNamespaceIdentifier(namespace); err != nil {

Review Comment:
   This rewire is out of scope for a "validate view identifiers" PR — `CreateNamespace` is neither a view API nor a boundary this PR's title covers — and #1614 makes the same change to the REST catalog with a broader contract.
   
   It's also only half the story on its own: `DropNamespace` (`sql.go:1288`), and the load/update-properties paths stay on the length-only `checkValidNamespace` (`sql.go:736`), so after this the SQL catalog validates namespace identifiers on create but not on any other namespace operation.
   
   **Suggested fix:** drop this hunk and let the namespace contract be settled in #1614 in one place, for all operations. If you'd rather keep it here, apply it to every namespace entry point in this file rather than just `CreateNamespace`, so the catalog has one consistent rule.



##########
catalog/catalog.go:
##########
@@ -272,6 +267,31 @@ func validateIdentifier(ident table.Identifier, notFoundErr error) error {
 	return nil
 }
 
+func validateIdentifier(ident table.Identifier, notFoundErr error) error {
+	if len(ident) < 2 {
+		return fmt.Errorf("%w: missing namespace or invalid identifier %v",
+			notFoundErr, strings.Join(ident, "."))
+	}
+
+	return validateIdentifierComponents(ident, notFoundErr)
+}
+
+// ValidateNamespaceIdentifier checks that an identifier contains at least one valid namespace level.
+func ValidateNamespaceIdentifier(ident table.Identifier) error {

Review Comment:
   **Blocking as a coordination issue rather than a code defect:** this exact function is also added by #1614, at this same location, with an identical body.
   
   The godoc differs, and #1614's is the accurate one. This comment says the function "checks that an identifier contains at least one valid namespace level," but the implementation never defines or checks what makes a level *valid* — it rejects only an empty identifier and NUL bytes. #1614 says so directly ("no null characters ... other rules intentionally left to the implementation"), which matches what the code does.
   
   **Suggested fix:** rebase onto #1614 and drop this copy. It isn't view-related, so removing it makes this PR match its own title, and it avoids a merge conflict where the two godocs would have to be reconciled by hand.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] fix(sql): validate view identifiers at API boundaries [iceberg-go]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #1571:
URL: https://github.com/apache/iceberg-go/pull/1571#discussion_r3719185600


##########
catalog/sql/sql.go:
##########
@@ -1668,6 +1672,12 @@ func (c *Catalog) ListViews(ctx context.Context, namespace table.Identifier) ite
 }
 
 func (c *Catalog) listViewsAll(ctx context.Context, namespace table.Identifier) ([]table.Identifier, error) {
+	if len(namespace) > 0 {
+		if err := catalog.ValidateNamespaceIdentifier(namespace); err != nil {

Review Comment:
   I think this makes `ListViews` stricter than the catalog it's reading from.
   
   `checkValidNamespace` is what `CreateNamespace`, `DropNamespace`, and `LoadNamespaceProperties` all use, and it only checks `len(ident) >= 1`, never the components. So `CreateNamespace(ctx, []string{".."}, nil)` succeeds today, the row lands in the DB, and `ListTables` on it still works since `listTablesAll` goes straight to `resolveNamespaceKey`. After this change `ListViews` rejects that same namespace with `ErrNoSuchNamespace` before it ever reaches the lookup. Same namespace, two different answers depending on which list call we make.
   
   I'd rather push the component check down into `checkValidNamespace` so the write path refuses to create those names in the first place and both list paths see the same universe. If tightening create feels out of scope here, I'd drop this block and let `resolveNamespaceKey`'s not-found path handle it, matching `listTablesAll`. Either is defensible, but I don't think we want the split. wdyt?



##########
catalog/sql/sql_test.go:
##########
@@ -2218,6 +2218,43 @@ func (s *SqliteCatalogTestSuite) TestCreateView() {
 	s.True(exists)
 }
 
+func TestViewOperationsRejectInvalidIdentifiers(t *testing.T) {
+	t.Parallel()
+
+	invalid := []struct {
+		name       string
+		identifier table.Identifier
+	}{
+		{name: "nil", identifier: nil},
+		{name: "empty", identifier: table.Identifier{}},
+		{name: "missing namespace", identifier: table.Identifier{"view"}},
+		{name: "empty name", identifier: table.Identifier{"ns", ""}},
+		{name: "dot name", identifier: table.Identifier{"ns", "."}},
+		{name: "parent name", identifier: table.Identifier{"ns", ".."}},
+		{name: "path separator", identifier: table.Identifier{"ns", "nested/view"}},
+		{name: "control character", identifier: table.Identifier{"ns", "view\nname"}},
+	}
+
+	cat := &sqlcat.Catalog{}
+	for _, test := range invalid {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+
+			err := cat.CreateView(context.Background(), test.identifier, nil, "", nil)
+			assert.ErrorIs(t, err, catalog.ErrNoSuchView)
+
+			err = cat.DropView(context.Background(), test.identifier)
+			assert.ErrorIs(t, err, catalog.ErrNoSuchView)
+
+			_, err = cat.CheckViewExists(context.Background(), test.identifier)
+			assert.ErrorIs(t, err, catalog.ErrNoSuchView)
+
+			_, err = cat.LoadView(context.Background(), test.identifier)
+			assert.ErrorIs(t, err, catalog.ErrNoSuchView)

Review Comment:
   `ListViews` is the one view op missing from this table, and it's also the one with its own validation call (`ValidateNamespaceIdentifier` rather than `ValidateViewIdentifier`). Its only coverage right now is the integration assertion down in `TestListViews`. I'd add a case here so a regression in `listViewsAll` shows up in the fast test too.



##########
catalog/sql/sql_test.go:
##########
@@ -2469,6 +2506,13 @@ func (s *SqliteCatalogTestSuite) TestListViews() {
 
 		break
 	}
+
+	viewsIter = db.ListViews(context.Background(), []string{".."})
+	for _, err := range viewsIter {

Review Comment:
   This can pass without asserting anything. If `ListViews` ever returns an empty iterator instead of one that yields the error, the loop body never runs, `s.ErrorIs` never fires, and the suite still goes green. It works today because the error path yields exactly one element, but that's an implicit contract we aren't checking here.
   
   I'd pull the error out of the loop and assert after it:
   
   ```go
   var gotErr error
   yielded := false
   for _, err := range db.ListViews(context.Background(), []string{".."}) {
   	yielded, gotErr = true, err
   
   	break
   }
   s.True(yielded, "invalid namespace must yield at least one element")
   s.ErrorIs(gotErr, catalog.ErrNoSuchNamespace)
   ```



##########
catalog/sql/sql.go:
##########
@@ -1788,6 +1802,10 @@ func (c *Catalog) DropView(ctx context.Context, identifier table.Identifier) err
 
 // CheckViewExists returns true if a view exists in the catalog.
 func (c *Catalog) CheckViewExists(ctx context.Context, identifier table.Identifier) (bool, error) {
+	if err := catalog.ValidateViewIdentifier(identifier); err != nil {

Review Comment:
   This one shifts V0 behavior. `CheckViewExists` used to return `(false, nil)` for any identifier on a V0 catalog, and now an invalid one gets an error instead. `ListViews` has the same shape, where `(nil, nil)` becomes `ErrNoSuchNamespace`. For `CreateView`, `DropView`, and `LoadView` it's only a swap of which error comes back, but for these two we're turning a clean negative into a failure.
   
   I'd move the validation below the `isV0()` guard in all five. That also matches how `ValidateTableIdentifier` sits after the early exits in `LoadTable` and `CommitTable`.
   
   One thing to flag: if we do that, `TestViewOperationsRejectInvalidIdentifiers` stops passing, because the zero-value catalog it uses is V0. Separate note on that below.



##########
catalog/sql/sql_test.go:
##########
@@ -2218,6 +2218,43 @@ func (s *SqliteCatalogTestSuite) TestCreateView() {
 	s.True(exists)
 }
 
+func TestViewOperationsRejectInvalidIdentifiers(t *testing.T) {
+	t.Parallel()
+
+	invalid := []struct {
+		name       string
+		identifier table.Identifier
+	}{
+		{name: "nil", identifier: nil},
+		{name: "empty", identifier: table.Identifier{}},
+		{name: "missing namespace", identifier: table.Identifier{"view"}},
+		{name: "empty name", identifier: table.Identifier{"ns", ""}},
+		{name: "dot name", identifier: table.Identifier{"ns", "."}},
+		{name: "parent name", identifier: table.Identifier{"ns", ".."}},
+		{name: "path separator", identifier: table.Identifier{"ns", "nested/view"}},
+		{name: "control character", identifier: table.Identifier{"ns", "view\nname"}},
+	}
+
+	cat := &sqlcat.Catalog{}

Review Comment:
   `&sqlcat.Catalog{}` leaves `schemaVersion` at its zero value, which is V0, so this test only passes because the new validation sits above the `isV0()` guard. It's asserting guard ordering rather than the behavior we actually care about.
   
   I'd stand up a V1 in-memory SQLite catalog here instead, so the assertions hold whichever side of `isV0()` the validation ends up on.



##########
catalog/catalog.go:
##########
@@ -272,6 +267,24 @@ func validateIdentifier(ident table.Identifier, notFoundErr error) error {
 	return nil
 }
 
+func validateIdentifier(ident table.Identifier, notFoundErr error) error {
+	if len(ident) < 2 {
+		return fmt.Errorf("%w: missing namespace or invalid identifier %v",
+			notFoundErr, strings.Join(ident, "."))
+	}
+
+	return validateIdentifierComponents(ident, notFoundErr)
+}
+
+// ValidateNamespaceIdentifier checks that an identifier contains at least one valid namespace level.
+func ValidateNamespaceIdentifier(ident table.Identifier) error {
+	if len(ident) < 1 {
+		return fmt.Errorf("%w: empty namespace identifier", ErrNoSuchNamespace)
+	}
+

Review Comment:
   Since this is exported now, I'd note in the doc comment that it's stricter than the length-only `checkValidNamespace` the SQL and REST catalogs use internally. We end up with two namespace validators in tree with different semantics and nothing pointing that out, so the next catalog implementation just picks whichever one it happens to find first.
   
   Not a blocker on its own, though it matters more if we keep the asymmetry (see my note on `listViewsAll`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org