You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "laskoviymishka (via GitHub)" <gi...@apache.org> on 2026/05/10 21:03:36 UTC

[PR] feat(puffin): enforce required DV blob properties on write [iceberg-go]

laskoviymishka opened a new pull request, #1060:
URL: https://github.com/apache/iceberg-go/pull/1060

   Closes #1057. Puffin spec requires `deletion-vector-v1` blobs to carry both `cardinality` and `referenced-data-file` properties. AddBlob already enforces the snapshot-id/sequence-number invariants for DV blobs; extend the same branch to reject blobs missing either property.
   
   Closes the gap that prompted the reader-side slog.Warn fallback in #1049 — once writers can't omit the property, the warning path stays unreachable from in-tree writes and only fires on cross-impl reads of files written by third-party tools.
   
   Two existing test fixtures that wrote DV blobs without the cardinality property are updated: TestRoundTrip in puffin_test.go and writePuffinWithDVBlob in dv's deletion_vector_test.go.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(puffin): enforce required DV blob properties on write [iceberg-go]

Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on code in PR #1060:
URL: https://github.com/apache/iceberg-go/pull/1060#discussion_r3237293569


##########
puffin/puffin_writer.go:
##########
@@ -137,6 +138,28 @@ func (w *Writer) AddBlob(input BlobMetadataInput, data []byte) (BlobMetadata, er
 		if input.SequenceNumber != -1 {
 			return BlobMetadata{}, errors.New("puffin: deletion-vector-v1 requires sequence-number to be -1")
 		}
+		// Properties cardinality and referenced-data-file are spec-mandated
+		// for deletion-vector-v1; the reader in table/dv does not enforce
+		// them today, so we hold the line here to keep Go-emitted files
+		// always conformant.
+		if input.Properties["cardinality"] == "" {
+			return BlobMetadata{}, errors.New("puffin: deletion-vector-v1 requires a cardinality property")
+		}
+		// Reject non-numeric or negative values at write time too —
+		// otherwise a writer could emit "cardinality": "abc" that the
+		// reader hard-rejects when it parses the property. Symmetric
+		// with strconv.ParseInt + non-negative check on the read side.
+		card, err := strconv.ParseInt(input.Properties["cardinality"], 10, 64)

Review Comment:
   Good catch, refactored this to `ParseUint`.
   
   For `"-1"`, the leading `-` now fails as invalid syntax before any value is parsed, so non-numeric and negative values go through the same error path. No separate negative check needed.
   
   I also cleaned up `dv/deletion_vector_test.go`: the old reader-side “negative cardinality rejected” and “malformed property rejected” subtests can’t build their fixtures anymore, because the writer now rejects them at `AddBlob` time. I removed those and left a pointer to the writer-side coverage in `puffin_test.go`.
   
   The reader-side checks in `blobCardinality` still stay, since we can still hit malformed files written by third-party writers.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(puffin): enforce required DV blob properties on write [iceberg-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #1060:
URL: https://github.com/apache/iceberg-go/pull/1060


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] feat(puffin): enforce required DV blob properties on write [iceberg-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #1060:
URL: https://github.com/apache/iceberg-go/pull/1060#discussion_r3228271081


##########
puffin/puffin_writer.go:
##########
@@ -137,6 +138,28 @@ func (w *Writer) AddBlob(input BlobMetadataInput, data []byte) (BlobMetadata, er
 		if input.SequenceNumber != -1 {
 			return BlobMetadata{}, errors.New("puffin: deletion-vector-v1 requires sequence-number to be -1")
 		}
+		// Properties cardinality and referenced-data-file are spec-mandated
+		// for deletion-vector-v1; the reader in table/dv does not enforce
+		// them today, so we hold the line here to keep Go-emitted files
+		// always conformant.
+		if input.Properties["cardinality"] == "" {
+			return BlobMetadata{}, errors.New("puffin: deletion-vector-v1 requires a cardinality property")
+		}
+		// Reject non-numeric or negative values at write time too —
+		// otherwise a writer could emit "cardinality": "abc" that the
+		// reader hard-rejects when it parses the property. Symmetric
+		// with strconv.ParseInt + non-negative check on the read side.
+		card, err := strconv.ParseInt(input.Properties["cardinality"], 10, 64)

Review Comment:
   Could get two birds with one stone by using `ParseUint` instead which would error on a negative value. So you could just do something like
   
   ```go
   if _, err := strconv.ParseUint(input.Properties["cardinality"], 10, 64); err != nil {
           // got something other than a non-negative integer
   }
   ```
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org