You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by si...@apache.org on 2018/01/26 19:02:06 UTC
[bookkeeper] branch master updated: BP-26: Move the development of
distributedlog library to bookkeeper
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push:
new d06fad2 BP-26: Move the development of distributedlog library to bookkeeper
d06fad2 is described below
commit d06fad25d82a504bc46c906b5c23b5634567d543
Author: Sijie Guo <si...@apache.org>
AuthorDate: Fri Jan 26 11:01:58 2018 -0800
BP-26: Move the development of distributedlog library to bookkeeper
Descriptions of the changes in this PR:
This is a BP proposing moving the development of distributedlog library to bookkeeper. See details in the BP file.
Master Issue: #1024
Author: Sijie Guo <si...@apache.org>
Reviewers: Henry Saputra <hs...@apache.org>, Matteo Merli <mm...@apache.org>, Dave Rusek <da...@gmail.com>, Ivan Kelly <iv...@apache.org>, Enrico Olivelli <eo...@gmail.com>, Jia Zhai <None>, Sijie Guo <si...@apache.org>, Yiming Zang <yz...@gmail.com>
This closes #1025 from sijie/bp_move_distributedlog
---
site/bps/BP-26-move-distributedlog-core-library.md | 62 ++++++++++++++++++++++
site/community/bookkeeper_proposals.md | 3 +-
2 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/site/bps/BP-26-move-distributedlog-core-library.md b/site/bps/BP-26-move-distributedlog-core-library.md
new file mode 100644
index 0000000..6cf646a
--- /dev/null
+++ b/site/bps/BP-26-move-distributedlog-core-library.md
@@ -0,0 +1,62 @@
+---
+title: "BP-26: Move distributedlog library as part of bookkeeper"
+issue: https://github.com/apache/bookkeeper/1024
+state: 'Accepted'
+release: "N/A"
+---
+
+### Motivation
+
+DistributedLog is an extension of Apache BookKeeper, which offers *reopenable* log streams as its storage primitives.
+It is tightly built over bookkeeper ledgers, and provides an easier-to-use abstraction and api to use. Applications
+can use *named* log streams rather than *numbered* ledgers to store their data. For example, users can use log streams
+as files to storge objects, checkpoints and other more general filesystem related use cases.
+
+Moving the distributedlog core library as part of bookkeeper would have following benefits:
+
+- It provides more generic "reopenable" log abstraction. It lowers the barrier for people to use bookkeeper to store
+ data, and bring in more use cases into bookkeeper ecosystem.
+- Using ledgers to build continous log stream has been a pattern that been reimplemented multiple times at multiple places,
+ from older projects like HDFS namenode log manager, Hedwig to the newer projects like DistributedLog and Pulsar.
+- Most of the distributedlog usage is using the distributedlog library which only depends Apache BookKeeper and there is no
+ additional components introduced. To simplify those usages, it is better to release distributedlog library along with
+ bookkeeper. It provides a better integration and release procedure.
+
+This proposal proposes "moving the distributedlog library code base as part of bookkeeper and continuing the library
+development in bookkeeper".
+
+### Public Interfaces
+
+This is a new library moved in bookkeeper. It will *NOT* touch any existing bookkeeper modules and ledger api.
+
+### Proposed Changes
+
+This proposal will *ONLY* move following library-only modules from distributedlog repo:
+
+- distributedlog-core: the log stream library that build over bookkeeper ledgr api. It doesn't introduce any service components. Library only.
+- distributedlog-io/dlfs: A hdfs filesystem api wrapper over the log stream api, to provide filesystem-like usage over bookkeeper.
+
+This proposal will *NOT* move other service components like "distributedlog-proxy".
+
+The steps to make this change are described as following:
+
+- the proposed modules (`distributedlog-core` and `distributedlog-io/dlfs`) will be moved under `stream/distributedlog` directory at apache bookkeeper repo.
+- a new "stream" profile will be added to the root `pom.xml` file. The distributedlog module will only be build when "-Pstream" is specified
+ in the maven build command. This allows users who only use ledger api skip building distributedlog module.
+- the distributedlog api, javadoc api and some tutorials will be integrated with current bookkeeper website to provide integrated experiences
+ when users browse bookkeeper website.
+
+### Compatibility, Deprecation, and Migration Plan
+
+This doesn't change existing modules or api. so no compatibility, deprecation and migration plan for bookkeeper users.
+
+For distributedlog users, the distributedlog library will begin release under groupId `org.apache.bookkeeper` instead of `org.apache.distributedlog`.
+API documentation will be updated to reflect this.
+
+### Test Plan
+
+N/A
+
+### Rejected Alternatives
+
+N/A
diff --git a/site/community/bookkeeper_proposals.md b/site/community/bookkeeper_proposals.md
index 672169c..608483a 100644
--- a/site/community/bookkeeper_proposals.md
+++ b/site/community/bookkeeper_proposals.md
@@ -85,7 +85,7 @@ using Google Doc.
This section lists all the _bookkeeper proposals_ made to BookKeeper.
-*Next Proposal Number: 26*
+*Next Proposal Number: 27*
### Inprogress
@@ -97,6 +97,7 @@ Proposal | State
[BP-14 Relax durability](https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-14+Relax+durability) | Accepted
[BP-16: Thin Client - Remove direct metadata storage access from clients](https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-16%3A+Thin+Client+-+Remove+direct+metadata+storage+access+from+clients) | Draft
[BP-18: LedgerType, Flags and StorageHints](https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-18%3A+LedgerType%2C+Flags+and+StorageHints) | Accepted
+[BP-26: Move distributedlog library as part of bookkeeper](../../bps/BP-26-move-distributedlog-core-library) | Accepted
### Adopted
--
To stop receiving notification emails like this one, please contact
sijie@apache.org.