[GitHub] [druid] jihoonson opened a new issue #11125: Investigate how we can reduce the size of binary distribution

[GitBox <gi...@apache.org>]

jihoonson opened a new issue #11125:
URL: https://github.com/apache/druid/issues/11125


   There is a limit on the max file size that can be committed to the asf subversion server. The max size is currently set to 450MB for us. The binary size of 0.21.0 is 358MB. Looking more closely at what are occupying much space, the core extensions contribute more than half of the total binary size.
   
   ```
   ➜  apache-druid-0.21.0 git:(733697c25f) du -d 1 . | sort -rn  
   407196	.
   259932	./extensions
   111668	./lib
   31132	./hadoop-dependencies
   2488	./quickstart
   860	./licenses
   808	./conf
   148	./bin
   ```
   
   Here is the size of core extensions.
   
   ```
   ➜  extensions git:(733697c25f) du -d 1 . | sort -rn
   259932	.
   63412	./druid-ranger-security
   49192	./druid-kubernetes-extensions
   41260	./druid-parquet-extensions
   28168	./druid-hdfs-storage
   15616	./druid-orc-extensions
   11160	./druid-kafka-extraction-namespace
   9796	./druid-kafka-indexing-service
   6760	./druid-avro-extensions
   6428	./druid-aws-rds-extensions
   5288	./druid-kerberos
   3856	./druid-azure-extensions
   3848	./druid-pac4j
   2956	./druid-kinesis-indexing-service
   2396	./druid-datasketches
   2228	./druid-stats
   2000	./druid-protobuf-extensions
   1984	./druid-lookups-cached-single
   1408	./druid-lookups-cached-global
   936	./postgresql-metadata-storage
   576	./druid-bloom-filter
   228	./druid-google-extensions
   160	./druid-basic-security
   116	./druid-histogram
   80	./druid-s3-extensions
   36	./druid-ec2-extensions
   24	./mysql-metadata-storage
   16	./simple-client-sslcontext
   ```
   
   As we are going to keep adding new extensions in the future, 100MB might not be a lot of space left. We should investigate how we can reduce the size of these extensions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org