You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by pe...@apache.org on 2021/04/01 09:55:59 UTC
[pulsar] branch branch-2.7 updated (aa3969b -> 925fb1d)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a change to branch branch-2.7
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from aa3969b Reduce log level for ack-grouping tracker (#10094)
new cf76da3 [Issue 10058]:apply-config-from-env.py to commented default values (#10060)
new 925fb1d [Broker] Fix NonDurableCursorImpl initialPosition by startCursorPosition greater than lastConfirmedEntry problem. (#10095)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docker/pulsar/scripts/apply-config-from-env.py | 6 ++++--
.../org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java | 2 +-
.../org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java | 7 +++++++
3 files changed, 12 insertions(+), 3 deletions(-)
[pulsar] 01/02: [Issue 10058]:apply-config-from-env.py to commented
default values (#10060)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.7
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit cf76da37226308bc3e3f627fdeae161d72076ee4
Author: Kevin Wilson <kl...@comcast.net>
AuthorDate: Wed Mar 31 21:49:38 2021 -0600
[Issue 10058]:apply-config-from-env.py to commented default values (#10060)
Add handling of commented name=value so that apply-config-from-env.py may un-comment the values if needed.
It is common practice to leave default values commented out in the configuration files. By not handling the commented case, the end customer may need to jump through hoops(such as adding un-commented values via script) to make changes to these values in a production environment. This enhancement will ease the process by allowing the script to catalog the commented as well as un-commented values, and make replacements to both as needed, leading to a smoother customer experience when a [...]
If a value is not supplied to a commented variable, then the line remains commented out, and the default value should be assumed by the application.
Any lines that do not contain a = or result in a parsing error should be ignored and the original line left in the configuration file.
(cherry picked from commit 4b7facf49b1ef166f21ccd1b3aea37d482900313)
---
docker/pulsar/scripts/apply-config-from-env.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/docker/pulsar/scripts/apply-config-from-env.py b/docker/pulsar/scripts/apply-config-from-env.py
index 992b8f7..803c4f1 100755
--- a/docker/pulsar/scripts/apply-config-from-env.py
+++ b/docker/pulsar/scripts/apply-config-from-env.py
@@ -44,12 +44,14 @@ for conf_filename in conf_files:
for line in open(conf_filename):
lines.append(line)
line = line.strip()
- if not line or line.startswith('#'):
+ if not line:
continue
try:
k,v = line.split('=', 1)
- keys[k] = len(lines) - 1
+ if k.startswith('#'):
+ k = k[1:]
+ keys[k.strip()] = len(lines) - 1
except:
print("[%s] skip Processing %s" % (conf_filename, line))
[pulsar] 02/02: [Broker] Fix NonDurableCursorImpl initialPosition
by startCursorPosition greater than lastConfirmedEntry problem. (#10095)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.7
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 925fb1da99a0a9daea894b9c2b1d722d54dabcab
Author: congbo <39...@users.noreply.github.com>
AuthorDate: Thu Apr 1 16:18:00 2021 +0800
[Broker] Fix NonDurableCursorImpl initialPosition by startCursorPosition greater than lastConfirmedEntry problem. (#10095)
## Motivation
In order to fix reader set ```startMessageId``` greater than ```lastConfirmedEntry``` probelm.
When the reader set ```startMessageId``` greater than ```lastConfirmedEntry```, generate ```NonDurableCursorImpl``` will carry a ```MarkdeletePosition``` greater than lastConfirmedEntry.
When use ```getNumberOfEntriesInBacklog``` will throw
## implement
When ```startMessageId``` greater than ```lastConfirmedEntry``` use ```lastConfirmedEntry``` as ```markdeletePosition```
### Verifying this change
Add the tests for it
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes
Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)
(cherry picked from commit 7d84e72a08e3baca8cb5fdbfc4bbfc3a1ae3ec92)
---
.../org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java | 2 +-
.../org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java | 7 +++++++
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java
index 6d1bdfb..167bcec 100644
--- a/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java
+++ b/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorImpl.java
@@ -40,7 +40,7 @@ public class NonDurableCursorImpl extends ManagedCursorImpl {
// Compare with "latest" position marker by using only the ledger id. Since the C++ client is using 48bits to
// store the entryId, it's not able to pass a Long.max() as entryId. In this case there's no point to require
// both ledgerId and entryId to be Long.max()
- if (startCursorPosition == null || startCursorPosition.getLedgerId() == PositionImpl.latest.getLedgerId()) {
+ if (startCursorPosition == null || startCursorPosition.compareTo(ledger.lastConfirmedEntry) > 0) {
// Start from last entry
switch (initialPosition) {
case Latest:
diff --git a/managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java b/managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java
index 7a15035..3d806c9 100644
--- a/managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java
+++ b/managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/NonDurableCursorTest.java
@@ -323,6 +323,13 @@ public class NonDurableCursorTest extends MockedBookKeeperTestCase {
}
@Test(timeOut = 20000)
+ void markDeleteGreaterThanLastConfirmedEntry() throws Exception {
+ ManagedLedger ml1 = factory.open("my_test_ledger");
+ ManagedCursor mc1 = ml1.newNonDurableCursor(PositionImpl.get(Long.MAX_VALUE - 1, Long.MAX_VALUE - 1));
+ assertEquals(mc1.getMarkDeletedPosition(), ml1.getLastConfirmedEntry());
+ }
+
+ @Test(timeOut = 20000)
void testResetCursor() throws Exception {
ManagedLedger ledger = factory.open("my_test_move_cursor_ledger",
new ManagedLedgerConfig().setMaxEntriesPerLedger(10));