[GitHub] [iceberg] rdblue commented on a diff in pull request #6582: Add a Spark procedure to collect NDV

[GitBox <gi...@apache.org>]

rdblue commented on code in PR #6582:
URL: https://github.com/apache/iceberg/pull/6582#discussion_r1073829241


##########
core/src/main/java/org/apache/iceberg/puffin/StandardBlobTypes.java:
##########
@@ -26,4 +26,6 @@ private StandardBlobTypes() {}
    * href="https://datasketches.apache.org/">Apache DataSketches</a> library
    */
   public static final String APACHE_DATASKETCHES_THETA_V1 = "apache-datasketches-theta-v1";
+
+  public static final String NDV_BLOB = "ndv-blob";

Review Comment:
   Focusing on just the table level NDV case:
   * We don't want to introduce a new stats blob type if possible. We chose Theta because of the additional capabilities the sketch has.
   * Can you be more specific about what Spark supports? Is it that we want HLL to feed HLL buffers into Spark? I would expect Spark to want NDV values, not the buffer/blob data.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org