[GitHub] [iceberg] jackye1995 commented on issue #1887: Standardize the way to ship new catalogs

[GitBox <gi...@apache.org>]

jackye1995 commented on issue #1887:
URL: https://github.com/apache/iceberg/issues/1887#issuecomment-741077821


   > My vote would be to start with X == 1mb.
   
   > @jackye1995 is the Glue catalog small enough if included w/o dependencies?
   
   Yeah that's actually a good way to go, basically mark all the AWS dependencies as provided. In that case the added jar size will only be the `iceberg-aws` itself which is 53KB.
   
   So the user experience would be something like:
   
   ```
   spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:0.11.0,software.amazon.awssdk:bundle:2.15.40 \
       --conf spark.sql.catalog.test=org.apache.iceberg.spark.SparkCatalog \
       --conf spark.sql.catalog.test.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \
       --conf spark.sql.catalog.test.warehouse=s3://some-bucket
   ```
   
   which looks good to me.
   
   I will put up a PR for this after #1823 and #1844 are merged to change dependencies to `compileOnly`. 
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org