You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pekko.apache.org by fa...@apache.org on 2023/07/29 14:22:21 UTC
[incubator-pekko-persistence-dynamodb] branch main updated: update doc links and reduce Akka links (#70)
This is an automated email from the ASF dual-hosted git repository.
fanningpj pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-pekko-persistence-dynamodb.git
The following commit(s) were added to refs/heads/main by this push:
new ec1091b update doc links and reduce Akka links (#70)
ec1091b is described below
commit ec1091b7b1fe0b0b140936a5b86817db58c482ed
Author: PJ Fanning <pj...@users.noreply.github.com>
AuthorDate: Sat Jul 29 15:22:15 2023 +0100
update doc links and reduce Akka links (#70)
* update doc links and reduce Akka links
* Update CONTRIBUTING.md
* uptate documentation contributing section
---
CONTRIBUTING.md | 54 ++++++---------
README.md | 77 ++++++++++++++++------
project/build.properties | 2 +-
src/main/resources/reference.conf | 12 ++--
.../dynamodb/journal/DynamoDBJournalRequests.scala | 3 +-
5 files changed, 87 insertions(+), 61 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 8eabb2a..beed650 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -41,23 +41,19 @@ This is the process for committing code into main branch. There are of course ex
For a Pull Request to be considered at all it has to meet these requirements:
1. Live up to the current code standard:
- - Not violate [DRY](http://programmer.97things.oreilly.com/wiki/index.php/Don%27t_Repeat_Yourself).
- - [Boy Scout Rule](http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule) needs to have been applied.
+ - Not violate [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).
+ - [Boy Scout Rule](https://www.oreilly.com/library/view/97-things-every/9780596809515/ch08.html) needs to have been applied.
2. Regardless if the code introduces new features or fixes bugs or regressions, it must have comprehensive tests.
3. The code must be well documented in the Typesafe's standard documentation format (see the ‘Documentation’ section below).
4. The commit messages must properly describe the changes, see further below.
-5. All Typesafe projects must include Typesafe copyright notices. Each project can choose between one of two approaches:
-
- 1. All source files in the project must have a Typesafe copyright notice in the file header.
- 2. The Notices file for the project includes the Typesafe copyright notice and no other files contain copyright notices. See http://www.apache.org/legal/src-headers.html for instructions for managing this approach for copyrights.
-
- Akka uses the first choice, having copyright notices in every file header.
-
- Other guidelines to follow for copyright notices:
-
- - Use a form of ``Copyright (C) 2011-2016 Typesafe Inc. <http://www.typesafe.com>``, where the start year is when the project or file was first created and the end year is the last time the project or file was modified.
- - Never delete or change existing copyright notices, just add additional info.
- - Do not use ``@author`` tags since it does not encourage [Collective Code Ownership](http://www.extremeprogramming.org/rules/collective.html). However, each project should make sure that the contributors gets the credit they deserve—in a text file or page on the project website and in the release notes etc.
+5. All ASF projects require the source code to have license headers.
+
+Other guidelines to follow for copyright notices:
+
+- If you copy code from another source then retain its source header(s).
+- New source files should have an [Apache Source License](https://www.apache.org/legal/src-headers.html).
+- Never delete or change existing source headers.
+- Do not use ``@author`` tags since it does not encourage [Collective Code Ownership](http://www.extremeprogramming.org/rules/collective.html).
If these requirements are not met then the code should **not** be merged into main branch, or even reviewed - regardless of how good or important it is. No exceptions.
@@ -69,24 +65,16 @@ Each project should be configured to use a continuous integration (CI) tool (i.e
## Documentation
-All documentation should be generated using the sbt-site-plugin, *or* publish artifacts to a repository that can be consumed by the Typesafe stack.
+Unlike most Apache Pekko projects, the Apache Pekko DynamoDB Persistence Plugin project does not yet have documentation
+that is published to [pekko.apache.org](https://pekko.apache.org). Until this happens, we will use the GitHub project
+README and wiki to document how this project works.
-All documentation must abide by the following maxims:
-
-- Example code should be run as part of an automated test suite.
-- Version should be **programmatically** specifiable to the build.
-- Generation should be **completely automated** and available for scripting.
-- Artifacts that must be included in the Typesafe stack should be published to a maven “documentation” repository as documentation artifacts.
-
-All documentation is preferred to be in Typesafe's standard documentation format [reStructuredText](http://doc.akka.io/docs/akka/snapshot/dev/documentation.html) compiled using Typesafe's customized [Sphinx](http://sphinx.pocoo.org/) based documentation generation system, which among other things allows all code in the documentation to be externalized into compiled files and imported into the documentation.
-
-For more info, or for a starting point for new projects, look at the [Typesafe Documentation Template project](https://github.com/typesafehub/doc-template).
-
-For larger projects that have invested a lot of time and resources into their current documentation and samples scheme (like for example Play), it is understandable that it will take some time to migrate to this new model. In these cases someone from the project needs to take the responsibility of manual QA and verifier for the documentation and samples.
+Any new features should be documented. If there are any mistakes in the documentation, please get in touch and we will
+try to get the documentation fixed.
## External Dependencies
-All the external runtime dependencies for the project, including transitive dependencies, must have an open source license that is equal to, or compatible with, [Apache 2](http://www.apache.org/licenses/LICENSE-2.0).
+All the external runtime dependencies for the project, including transitive dependencies, must have an open source license that is equal to, or compatible with, [Apache 2](https://www.apache.org/licenses/LICENSE-2.0).
This must be ensured by manually verifying the license for all the dependencies for the project:
@@ -94,9 +82,9 @@ This must be ensured by manually verifying the license for all the dependencies
2. Whenever a committer to the project adds a new dependency.
3. Whenever a new release is cut (public or private for a customer).
-Which licenses are compatible with Apache 2 are defined in [this doc](http://www.apache.org/legal/3party.html#category-a), where you can see that the licenses that are listed under ``Category A`` automatically compatible with Apache 2, while the ones listed under ``Category B`` needs additional action:
+Which licenses are compatible with Apache 2 are defined in [this doc](https://www.apache.org/legal/resolved.html#category-a), where you can see that the licenses that are listed under ``Category A`` automatically compatible with Apache 2, while the ones listed under ``Category B`` needs additional action:
-> Each license in this category requires some degree of [reciprocity](http://www.apache.org/legal/3party.html#define-reciprocal); therefore, additional action must be taken in order to minimize the chance that a user of an Apache product will create a derivative work of a reciprocally-licensed portion of an Apache product without being aware of the applicable requirements.
+> Each license in this category requires some degree of [reciprocity](https://www.apache.org/legal/resolved.html#weak-copyleft-licenses); therefore, additional action must be taken in order to minimize the chance that a user of an Apache product will create a derivative work of a reciprocally-licensed portion of an Apache product without being aware of the applicable requirements.
Each project must also create and maintain a list of all dependencies and their licenses, including all their transitive dependencies. This can be done in either in the documentation or in the build file next to each dependency.
@@ -110,9 +98,9 @@ Also, to facilitate both well-formed commits and working together, the ``wip`` a
Follow these guidelines when creating public commits and writing commit messages.
-1. If your work spans multiple local commits (for example; if you do safe point commits while working in a feature branch or work in a branch for long time doing merges/rebases etc.) then please do not commit it all but rewrite the history by squashing the commits into a single big commit which you write a good commit message for (like discussed in the following sections). For more info read this article: [Git Workflow](http://sandofsky.com/blog/git-workflow.html). Every commit should be [...]
+1. If your work spans multiple local commits (for example; if you do safe point commits while working in a feature branch or work in a branch for long time doing merges/rebases etc.) then please do not commit it all but rewrite the history by squashing the commits into a single big commit which you write a good commit message for (like discussed in the following sections). For more info read this article: [Git Workflow](https://sandofsky.com/workflow/git-workflow/). Every commit should b [...]
-2. First line should be a descriptive sentence what the commit is doing, including the ticket number. It should be possible to fully understand what the commit does—but not necessarily how it does it—by just reading this single line. We follow the “imperative present tense” style for commit messages ([more info here](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)).
+2. First line should be a descriptive sentence what the commit is doing, including the ticket number. It should be possible to fully understand what the commit does—but not necessarily how it does it—by just reading this single line. We follow the “imperative present tense” style for commit messages ([more info here](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)).
It is **not ok** to only list the ticket number, type "minor fix" or similar.
If the commit is a small fix, then you are done. If not, go to 3.
@@ -132,4 +120,4 @@ Example:
## Source style
-Akka uses [scalafmt](https://scalameta.org/scalafmt/) to enforce some of the code style rules.
+Apache Pekko uses [scalafmt](https://scalameta.org/scalafmt/) to enforce some of the code style rules.
diff --git a/README.md b/README.md
index b84eaf5..7f0a8e5 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,7 @@
-DynamoDBJournal for Apache Pekko Persistence
-============================================
+# DApache Pekko DynamoDB Persistence Plugin
A replicated Pekko Persistence journal backed by
-[Amazon DynamoDB](http://aws.amazon.com/dynamodb/).
+[Amazon DynamoDB](https://aws.amazon.com/dynamodb/).
- This plugin implements both a journal as well as a snapshot store,
- This includes a Pekko Persistence Query plugin. However, this requires an additional GSI for efficient usage.
@@ -14,8 +13,7 @@ Supported versions:
[](https://github.com/apache/incubator-pekko-persistence-dynamodb/actions)
-Installation
-------------
+## Installation
This plugin is not yet released. When it is released, it will be published to the Maven Central repository with the following names:
@@ -38,8 +36,7 @@ Snapshot versions are available.
- you will need to add a resolver set to `https://repository.apache.org/content/groups/snapshots`
- in sbt 1.9.0+, you can add `resolvers += Resolver.ApacheMavenSnapshotsRepo`
-Configuration
--------------
+## Configuration
### Journal
~~~
@@ -55,7 +52,7 @@ my-dynamodb-journal { # and add some overrides
}
~~~
-For details on the endpoint URL please refer to the [DynamoDB documentation](http://docs.aws.amazon.com/general/latest/gr/rande.html#ddb_region). There are many more settings that can be used for fine-tuning and adapting this journal plugin to your use-case, please refer to the [reference.conf](https://github.com/apache/incubator-pekko-persistence-dynamodb/blob/main/src/main/resources/reference.conf) file.
+For details on the endpoint URL please refer to the [DynamoDB documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html#ddb_region). There are many more settings that can be used for fine-tuning and adapting this journal plugin to your use-case, please refer to the [reference.conf](https://github.com/apache/incubator-pekko-persistence-dynamodb/blob/main/src/main/resources/reference.conf) file.
Before you can use these settings you will have to create a table, e.g. using the AWS console, with the following schema:
@@ -85,7 +82,7 @@ The table to create for snapshot storage has the schema:
* a sort key of type Number with name `ts`
* a local secondary index with name `ts-idx` that is an index on the combination of `par` and `ts`
-The DynamoDB item of a snapshot [can be 400 kB](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items). Using a binary serialisation format like ProtoBuf or Kryo will use that space most effectively.
+The DynamoDB item of a snapshot [can be 400 kB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items). Using a binary serialisation format like ProtoBuf or Kryo will use that space most effectively.
### Read journal (Pekko persistence query)
contributed by [@joost-de-vries](https://github.com/joost-de-vries))
@@ -98,8 +95,8 @@ dynamodb-read-journal {
# persistence-ids-index-name: "persistence-ids-idx"
}
~~~
-Storage Semantics
------------------
+
+## Storage Semantics
DynamoDB only offers consistency guarantees for a single storage item—which corresponds to one event in the case of this Pekko Persistence plugin. This means that any single event is either written to the journal (and thereby visible to later replays) or it is not. This plugin supports atomic multi-event batches nevertheless, by marking the contained events such that partial replay can be avoided (see the `idx` and `cnt` attributes in the storage format description below). Consider the f [...]
@@ -116,8 +113,7 @@ In the first case a recovery will only ever see all of the events or none of the
In the second case each event is treated in isolation and may or may not be replayed depending on whether it was persisted successfully or not.
-Performance Considerations
---------------------------
+## Performance Considerations
This plugin uses the AWS Java SDK which means that the number of requests that can be made concurrently is limited by the number of connections to DynamoDB and by the number of threads in the thread-pool that is used by the AWS HTTP client. The default setting is 50 connections which for a deployment that is used from the same EC2 region allows roughly 5000 requests per second (where every persisted event batch is roughly one request). If a single ActorSystem needs to persist more than t [...]
@@ -127,15 +123,13 @@ my-dynamodb-journal.aws-client-config.max-connections = <your value here>
Changing this number changes both the number of concurrent connections and the used thread-pool size.
-Compatibility with Akka versions
------------------------------------
+## Compatibility with Akka versions
pekko-persistence-dynamodb is derived from [akka-persistence-dynamodb](https://github.com/akka/akka-persistence-dynamodb) v1.3.0.
Anyone migrating from using akka-persistence-dynamodb should first upgrade to akka-persistence-dynamodb v1.3.0.
-Plugin Development
-------------------
+## Plugin Development
### Dev Setup
@@ -211,14 +205,59 @@ This is somewhat more difficult to code, but offers higher throughput possibilit
When writing an item we typically do not touch the high sequence number storage, only when writing an item with sort key `0` is this done. This implies that reading the highest sequence number will need to first query the sequence shards for the highest multiple of 100 and then send a `Query` for the corresponding P entry’s hash key to find the highest stored sort key number.
-Credits
--------
+## Building from Source
+
+### Prerequisites
+- Make sure you have installed a Java Development Kit (JDK) version 8 or later.
+- Make sure you have [sbt](https://www.scala-sbt.org/) installed and using this JDK.
+- [Graphviz](https://graphviz.gitlab.io/download/) is needed for the scaladoc generation build task, which is part of the release.
+
+### Running the Build
+- Open a command window and change directory to your preferred base directory
+- Use git to clone the [repo](https://github.com/apache/incubator-pekko-persistence-dynamodb) or download a source release from https://pekko.apache.org (and unzip or untar it, as appropriate)
+- Change directory to the directory where you installed the source (you should have a file called `build.sbt` in this directory)
+- `sbt compile` compiles the main source for project default version of Scala (2.13)
+ - `sbt +compile` will compile for all supported versions of Scala
+- `sbt test` will compile the code and run the unit tests
+- `sbt testQuick` similar to test but when repeated in shell mode will only run failing tests
+- `sbt package` will build the jar
+ - the jar will be built to `target` directory
+- `sbt publishLocal` will push the jars to your local Apache Ivy repository
+- `sbt publishM2` will push the jars to your local Apache Maven repository
+- `sbt doc` will build the Javadocs for all the modules and load them to one place (may require Graphviz, see Prerequisites above)
+ - the `index.html` file will appear in `target/api/`
+- `sbt sourceDistGenerate` will generate source release to `target/dist/`
+- The version number that appears in filenames and docs is derived, by default. The derived version contains the most git commit id or the date/time (if the directory is not under git control).
+ - You can set the version number explicitly when running sbt commands
+ - eg `sbt "set ThisBuild / version := \"1.0.0\"; sourceDistGenerate"`
+ - Or you can add a file called `version.sbt` to the same directory that has the `build.sbt` containing something like
+ - `ThisBuild / version := "1.0.0"`
+
+## Community
+
+There are several ways to interact with the Pekko community:
+
+- [GitHub discussions](https://github.com/apache/incubator-pekko-persistence-dynamodb/discussions): for questions and general discussion.
+- [Pekko dev mailing list](https://lists.apache.org/list.html?dev@pekko.apache.org): for Pekko development discussions.
+- [Pekko users mailing list](https://lists.apache.org/list.html?users@pekko.apache.org): for Pekko user discussions.
+- [GitHub issues](https://github.com/apache/incubator-pekko-persistence-dynamodb/issues): for bug reports and feature requests. Please search the existing issues before creating new ones. If you are unsure whether you have found a bug, consider asking in GitHub discussions or the mailing list first.
+
+## Credits
- Initial development was done by [Scott Clasen](https://github.com/sclasen/akka-persistence-dynamodb).
- Update to Akka 2.4 and further development up to version 1.0 was kindly sponsored by [Zynga Inc.](https://www.zynga.com/).
- The snapshot store and readjournal were contributed by [Joost de Vries](https://github.com/joost-de-vries)
- [Corey O'Connor](https://dogheadbone.com/)
- Lightbend team
+- Apache Pekko Community
- Ryan Means
- Jean-Luc Deprez
- Michal Janousek
+
+## Code of Conduct
+
+Apache Pekko is governed by the [Apache code of conduct](https://www.apache.org/foundation/policies/conduct.html). By participating in this project you agree to abide by its terms.
+
+## License
+
+Apache Pekko DynamoDB Persistence Plugin is available under the Apache License, version 2.0. See [LICENSE](LICENSE) file for details.
diff --git a/project/build.properties b/project/build.properties
index 875b706..52413ab 100644
--- a/project/build.properties
+++ b/project/build.properties
@@ -1 +1 @@
-sbt.version=1.9.2
+sbt.version=1.9.3
diff --git a/src/main/resources/reference.conf b/src/main/resources/reference.conf
index aa395d3..939f7e8 100644
--- a/src/main/resources/reference.conf
+++ b/src/main/resources/reference.conf
@@ -50,15 +50,15 @@ dynamodb-journal {
#
# It is recommended to leave this (and the aws-secret-access-key) setting
# empty in order to use the default credentials provider chain, see
- # http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
+ # https://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
aws-access-key-id = ""
- # The AWS secret to use in conjuction with the AWS key ID.
+ # The AWS secret to use in conjunction with the AWS key ID.
# Will only be used if aws-access-key-id is also set.
#
# It is recommended to leave this (and the aws-access-key-id) setting
# empty in order to use the default credentials provider chain, see
- # http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
+ # https://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
aws-secret-access-key = ""
# number of concurrently running replay prefetch operations for a
@@ -82,7 +82,7 @@ dynamodb-journal {
}
# AWS client configuration settings, see
- # http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/?com/amazonaws/ClientConfiguration.html
+ # https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/?com/amazonaws/ClientConfiguration.html
#
# (setting any of these to the string "default" means that the corresponding
# setter method on the ClientConfiguration will not be invoked)
@@ -169,7 +169,7 @@ dynamodb-snapshot-store {
#
# It is recommended to leave this (and the aws-secret-access-key) setting
# empty in order to use the default credentials provider chain, see
- # http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
+ # https://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
aws-access-key-id = ${dynamodb-journal.aws-access-key-id}
# The AWS secret to use in conjuction with the AWS key ID.
@@ -177,7 +177,7 @@ dynamodb-snapshot-store {
#
# It is recommended to leave this (and the aws-access-key-id) setting
# empty in order to use the default credentials provider chain, see
- # http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
+ # https://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html#using-the-default-credential-provider-chain
aws-secret-access-key = ${dynamodb-journal.aws-secret-access-key}
# If this is set to `on` then every DynamoDB request will be logged
diff --git a/src/main/scala/org/apache/pekko/persistence/dynamodb/journal/DynamoDBJournalRequests.scala b/src/main/scala/org/apache/pekko/persistence/dynamodb/journal/DynamoDBJournalRequests.scala
index c03c815..137a89b 100644
--- a/src/main/scala/org/apache/pekko/persistence/dynamodb/journal/DynamoDBJournalRequests.scala
+++ b/src/main/scala/org/apache/pekko/persistence/dynamodb/journal/DynamoDBJournalRequests.scala
@@ -63,8 +63,7 @@ trait DynamoDBJournalRequests extends DynamoDBRequests {
* Write all PersistentRepr in the AtomicWrite provided
*
* If there are any errors serializing (preparing the batch writes), then we must return
- * a Future.success(Failure) as the result. This is needed to be compliant with
- * Akka Persistence 2.4
+ * a Future.success(Failure) as the result.
*
* @param atomicWrite Contains a list of persistentRepr that need to be persisted atomically
* @return a successfully completed Future that contains either a Success or Failure
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pekko.apache.org
For additional commands, e-mail: commits-help@pekko.apache.org