You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2023/03/05 19:38:14 UTC
[GitHub] [ozone] adoroszlai opened a new pull request, #4349: HDDS-8079. RocksDB included in client libs doubles shaded jar size
adoroszlai opened a new pull request, #4349:
URL: https://github.com/apache/ozone/pull/4349
## What changes were proposed in this pull request?
HDDS-7905 moved the Managed RocksDB classes from `hadoop-hdds/framework` to `hadoop-hdds/common` to be able to use them from `rocksdb-checkpoint-differ`.
`common` is shared between client and server components. Thus RocksDB library ends up in shaded FS jars. Since `rocksdbjni-7.7.3.jar` is 56MB, it almost doubles shaded jar sizes:
```
before: 60M share/ozone/lib/ozone-filesystem-hadoop3-1.4.0-SNAPSHOT.jar
after: 115M share/ozone/lib/ozone-filesystem-hadoop3-1.4.0-SNAPSHOT.jar
```
This change creates a new `hdds-managed-rocksdb` module for the managed RocksDB classes and the RocksDB library they depend on. This new module is used by both `hdds-framework` and `rocksdb-checkpoint-differ`. Dependency on RocksDB is removed from `hdds-common` module, as clients do not need it.
https://issues.apache.org/jira/browse/HDDS-8079
## How was this patch tested?
Verified tarball size is reduced to previous smaller one.
Before HDDS-7905:
```
Total size of all the files uploaded is 383031120 bytes
```
https://github.com/apache/ozone/actions/runs/4317076701/jobs/7533694758#step:7:78
Before patch:
```
Total size of all the files uploaded is 555056639 bytes
```
https://github.com/apache/ozone/actions/runs/4317355656/jobs/7534328633#step:7:109
With patch:
```
Total size of all the files uploaded is 383136131 bytes
```
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4337551095/jobs/7573634820#step:7:77
Full CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4337551095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4349: HDDS-8079. RocksDB included in client libs doubles shaded jar size
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4349:
URL: https://github.com/apache/ozone/pull/4349#issuecomment-1455543314
Thanks @kerneltime, @sadanand48 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime merged pull request #4349: HDDS-8079. RocksDB included in client libs doubles shaded jar size
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime merged PR #4349:
URL: https://github.com/apache/ozone/pull/4349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org