You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by ch...@apache.org on 2023/03/29 11:18:24 UTC
[bookkeeper] branch branch-4.16 updated (f12fb0f97e -> c14d5523a3)
This is an automated email from the ASF dual-hosted git repository.
chenhang pushed a change to branch branch-4.16
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
from f12fb0f97e [maven-release-plugin] rollback changes from release preparation of v4.16.0-rc1
new 378eb07b5c Optimize log4j dependency in Bookkeeper (#3892)
new 9b5a8106b5 release ledgerHandler when no missing fragments in checkAllLedgers (#3888)
new c14d5523a3 upgrade hadoop version to 3.3.5 to resolve CVE-2019-10202 (#3896)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
bookkeeper-server/pom.xml | 8 ++++++++
.../replication/AuditorCheckAllLedgersTask.java | 24 +++++++++++-----------
pom.xml | 10 +--------
testtools/pom.xml | 11 ++++++++++
4 files changed, 32 insertions(+), 21 deletions(-)
[bookkeeper] 03/03: upgrade hadoop version to 3.3.5 to resolve CVE-2019-10202 (#3896)
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chenhang pushed a commit to branch branch-4.16
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
commit c14d5523a33743e788a310f78df5f3ff22d8875f
Author: Hang Chen <ch...@apache.org>
AuthorDate: Wed Mar 29 17:42:05 2023 +0800
upgrade hadoop version to 3.3.5 to resolve CVE-2019-10202 (#3896)
### Motivation
There is a critical CVE-2019-10202 in `org.codehaus.jackson:jackson-mapper-asl`
Detailed paths
Introduced through: org.apache.distributedlog:dlfs@4.16.0-SNAPSHOT › org.apache.hadoop:hadoop-common@3.3.4 › org.apache.avro:avro@1.7.7 › org.codehaus.jackson:jackson-mapper-asl@1.9.2
Fix: No remediation path available.
Introduced through: org.apache.distributedlog:dlfs@4.16.0-SNAPSHOT › org.apache.hadoop:hadoop-common@3.3.4 › com.sun.jersey:jersey-json@1.19 › org.codehaus.jackson:jackson-mapper-asl@1.9.2
Fix: No remediation path available.
Introduced through: org.apache.distributedlog:dlfs@4.16.0-SNAPSHOT › org.apache.hadoop:hadoop-common@3.3.4 › com.sun.jersey:jersey-json@1.19 › org.codehaus.jackson:jackson-jaxrs@1.9.2 › org.codehaus.jackson:jackson-mapper-asl@1.9.2
Fix: No remediation path available.
Introduced through: org.apache.distributedlog:dlfs@4.16.0-SNAPSHOT › org.apache.hadoop:hadoop-common@3.3.4 › com.sun.jersey:jersey-json@1.19 › org.codehaus.jackson:jackson-xc@1.9.2 › org.codehaus.jackson:jackson-mapper-asl@1.9.2
Fix: No remediation path available.
### Changes
Upgrade hadoop-common version from 3.3.4 to 3.3.5 to resolve this CVE
(cherry picked from commit 0171a408e21a51eb74e18a07df1b0ea71b7638ff)
---
pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 89918cce5c..b43f99371d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -135,7 +135,7 @@
<grpc.version>1.47.0</grpc.version>
<guava.version>31.0.1-jre</guava.version>
<kerby.version>1.1.1</kerby.version>
- <hadoop.version>3.3.4</hadoop.version>
+ <hadoop.version>3.3.5</hadoop.version>
<hamcrest.version>1.3</hamcrest.version>
<hdrhistogram.version>2.1.10</hdrhistogram.version>
<jackson.version>2.13.4.20221013</jackson.version>
[bookkeeper] 02/03: release ledgerHandler when no missing fragments in checkAllLedgers (#3888)
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chenhang pushed a commit to branch branch-4.16
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
commit 9b5a8106b5129aab1f47be70a8ed344f871c8466
Author: lixinyang <84...@users.noreply.github.com>
AuthorDate: Wed Mar 29 09:44:45 2023 +0800
release ledgerHandler when no missing fragments in checkAllLedgers (#3888)
### Motivation
In checkAllLedgers when the ledger no missing fragments, will miss invoke` lh.closeAsync()` to close ledgerHandler, which cause autorecovery not invoke `unregisterLedgerMetadataListener` to release ledger metadata listeners. Heap memory be used too much and maybe will cause OOM;
<img width="1567" alt="image" src="https://user-images.githubusercontent.com/84127069/227937422-1113af68-9bf3-4466-97fa-d9b7cc5d72be.png">
### Changes
1. Invoke` lh.closeAsync()` when no missing fragments;
(cherry picked from commit 164417afc2474cbf4ba51e7cb57206b0eeb7d424)
---
.../replication/AuditorCheckAllLedgersTask.java | 24 +++++++++++-----------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorCheckAllLedgersTask.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorCheckAllLedgersTask.java
index 73ca36cd75..f6b7a6f36e 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorCheckAllLedgersTask.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorCheckAllLedgersTask.java
@@ -267,19 +267,19 @@ public class AuditorCheckAllLedgersTask extends AuditorTask {
if (bookies.isEmpty()) {
// no missing fragments
callback.processResult(BKException.Code.OK, null, null);
- return;
+ } else {
+ publishSuspectedLedgersAsync(bookies.stream().map(BookieId::toString).collect(Collectors.toList()),
+ Sets.newHashSet(lh.getId())
+ ).whenComplete((result, cause) -> {
+ if (null != cause) {
+ LOG.error("Auditor exception publishing suspected ledger {} with lost bookies {}",
+ lh.getId(), bookies, cause);
+ callback.processResult(BKException.Code.ReplicationException, null, null);
+ } else {
+ callback.processResult(BKException.Code.OK, null, null);
+ }
+ });
}
- publishSuspectedLedgersAsync(bookies.stream().map(BookieId::toString).collect(Collectors.toList()),
- Sets.newHashSet(lh.getId())
- ).whenComplete((result, cause) -> {
- if (null != cause) {
- LOG.error("Auditor exception publishing suspected ledger {} with lost bookies {}",
- lh.getId(), bookies, cause);
- callback.processResult(BKException.Code.ReplicationException, null, null);
- } else {
- callback.processResult(BKException.Code.OK, null, null);
- }
- });
} else {
callback.processResult(rc, null, null);
}
[bookkeeper] 01/03: Optimize log4j dependency in Bookkeeper (#3892)
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chenhang pushed a commit to branch branch-4.16
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
commit 378eb07b5c85eb499307f107aafa065154e0137f
Author: ZhangJian He <sh...@gmail.com>
AuthorDate: Tue Mar 28 17:02:33 2023 +0800
Optimize log4j dependency in Bookkeeper (#3892)
(cherry picked from commit e440132802ad883216453642820e407db00baed3)
---
bookkeeper-server/pom.xml | 8 ++++++++
pom.xml | 8 --------
testtools/pom.xml | 11 +++++++++++
3 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/bookkeeper-server/pom.xml b/bookkeeper-server/pom.xml
index a46b20c23d..ca7834ad41 100644
--- a/bookkeeper-server/pom.xml
+++ b/bookkeeper-server/pom.xml
@@ -153,6 +153,14 @@
<groupId>com.carrotsearch</groupId>
<artifactId>hppc</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </dependency>
<!-- testing dependencies -->
<dependency>
<groupId>org.apache.bookkeeper</groupId>
diff --git a/pom.xml b/pom.xml
index b154986600..89918cce5c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -830,14 +830,6 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
- <dependency>
- <groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-core</artifactId>
- </dependency>
- <dependency>
- <groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
- </dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
diff --git a/testtools/pom.xml b/testtools/pom.xml
index 600a4830e4..5ad6441ebb 100644
--- a/testtools/pom.xml
+++ b/testtools/pom.xml
@@ -25,4 +25,15 @@
<artifactId>testtools</artifactId>
<name>Apache BookKeeper :: Test Tools</name>
<version>4.16.0-SNAPSHOT</version>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </dependency>
+ </dependencies>
</project>