You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2021/10/06 17:08:53 UTC

[GitHub] [bookkeeper] RaulGracia opened a new pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

RaulGracia opened a new pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816


   ### Motivation
   Upgrades to log4j2 to get rid of CVE-2019-17571.  
   
   ### Changes
   
   TBD.
   
   Master Issue: #2815
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] RaulGracia commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
RaulGracia commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-945436041


   @zymap @eolivelli @pkumar-singh let me try to send a draft PR with the suggested change, so we can continue the discussion there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] RaulGracia commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
RaulGracia commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-937818625


   With current changes, log4j1.2 seems to be completely removed:
   [dependency-tree.txt](https://github.com/apache/bookkeeper/files/7303886/dependency-tree.txt)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] pkumar-singh commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
pkumar-singh commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r729987757



##########
File path: bookkeeper-dist/server/pom.xml
##########
@@ -85,8 +85,16 @@
 
     <!-- slf4j binding -->
     <dependency>
-      <groupId>org.slf4j</groupId>
-      <artifactId>slf4j-log4j12</artifactId>
+      <groupId>org.apache.logging.log4j</groupId>

Review comment:
       I know I am late at the PR, apologies. But I am not sure this is the right thing to do. Bookkeeper should work with SLF4J instead of log4j. Allow me to explain. There are three elements to it.
   1. SLF4J this is the facade for logging framework. It does not provide logging implementation. But just facade API. Bookkeeper should only be using that, test code could be a possible exception.  That way SLF4j can work with any type of logging infra, log4j2 or log4j1 being just few options out of many.
   2. SLF4j-LOG4J12: This is Binding for SLF4 facade with LOG4J1, "ideally" bookkeeper should not have any dependency on this. This should again be provided by application at runtime. Role of SLF4j-LOG4J12 is that If SLF4J sees SLF4j-LOG4J12 in the Runtime classpath, It will go on lookout for log4j12. And that will ensure that for actual logging log4j2 or log4j12 is used. In other words SLF4j-LOG4J12 works as bridge between facade and implementation. 
   3. log4j12 or log4j2 or simplelog4j or foolog4j: This is actual logging library. Which does the actual logging. And certainly should be provided at runtime by application. In ideal case Bookkeeper should not provide this as dependency. 
   
   To me looks like we have removed slf4j and started using log4j directly. Which means we have created a hard binding with log4j, which I believe is not desirable.
   
   @eolivelli @RaulGracia @merlimat @ivankelly @hsaputra  




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] RaulGracia commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
RaulGracia commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r724229964



##########
File path: bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/MdcUtils.java
##########
@@ -23,18 +23,18 @@
 
 import java.util.Map;
 
-import org.slf4j.MDC;
+import org.apache.logging.log4j.ThreadContext;

Review comment:
       Done, thanks for the input. Initially, I thought that the change was required, but it seems that it works using the slf4j API.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] pjfanning commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
pjfanning commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-1057971735


   This change does not seem to have been released officially yet and many people are looking at ASF and wondering why we still have releases that have insecure logging in them. Could a new release be done?
   
   Latest released pom:
   https://repo1.maven.org/maven2/org/apache/bookkeeper/bookkeeper-server/4.14.4/bookkeeper-server-4.14.4.pom


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r730376932



##########
File path: bookkeeper-dist/server/pom.xml
##########
@@ -85,8 +85,16 @@
 
     <!-- slf4j binding -->
     <dependency>
-      <groupId>org.slf4j</groupId>
-      <artifactId>slf4j-log4j12</artifactId>
+      <groupId>org.apache.logging.log4j</groupId>

Review comment:
       This is the dist package.
   
   It is important that in BK code we use only slf4j
   
   But this is the package, we have to provide an implementation 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] RaulGracia commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
RaulGracia commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r724229964



##########
File path: bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/MdcUtils.java
##########
@@ -23,18 +23,18 @@
 
 import java.util.Map;
 
-import org.slf4j.MDC;
+import org.apache.logging.log4j.ThreadContext;

Review comment:
       Done, thanks for the input. Initially, I thought that the change was required, but it seems that it works using the slf4j API.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] hangc0276 commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-1086538721


   I change the version of log4j from 2.14.1 to 2.17.1, and run bookie in local, it just print out few logs, and many logs are lost. 
   ```
   2022-04-02T11:55:49,537 - WARN  - [main:ServerCnxnFactory@309] - maxCnxns is not configured, using default value 0.
   2022-04-02T11:55:50,584 - ERROR - [main:DiskChecker@156] - Space left on device /tmp/bk-data/bookie0/current : 13121863680, Used space fraction: 0.97375435 > threshold 0.95.
   2022-04-02T11:55:50,585 - WARN  - [main:LedgerDirsManager@230] - /tmp/bk-data/bookie0/current is out of space. Adding it to filled dirs list
   2022-04-02T11:55:50,597 - ERROR - [main:Journal$LastLogMark@252] - Problems reading from /tmp/bk-data/bookie0/current/lastMark (this is okay if it is the first time starting this bookie
   2022-04-02T11:55:50,840 - WARN  - [BookieStateManagerService-0:ZKRegistrationManager@307] - No writable bookie registered node /ledgers/available/10.25.14.105:5000 when transitioning to readonly
   org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /ledgers/available/10.25.14.105:5000
           at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[org.apache.zookeeper-zookeeper-3.6.2.jar:3.6.2]
           at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[org.apache.zookeeper-zookeeper-3.6.2.jar:3.6.2]
           at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:2001) ~[org.apache.zookeeper-zookeeper-3.6.2.jar:3.6.2]
           at org.apache.bookkeeper.zookeeper.ZooKeeperClient.access$2101(ZooKeeperClient.java:70) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.zookeeper.ZooKeeperClient$11.call(ZooKeeperClient.java:773) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.zookeeper.ZooKeeperClient$11.call(ZooKeeperClient.java:767) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.zookeeper.ZooWorker.syncCallWithRetries(ZooWorker.java:140) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.zookeeper.ZooKeeperClient.delete(ZooKeeperClient.java:767) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.discover.ZKRegistrationManager.doRegisterReadOnlyBookie(ZKRegistrationManager.java:305) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.discover.ZKRegistrationManager.registerBookie(ZKRegistrationManager.java:226) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.bookie.BookieStateManager.doRegisterBookie(BookieStateManager.java:273) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.bookie.BookieStateManager.doRegisterBookie(BookieStateManager.java:261) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.bookie.BookieStateManager$2.call(BookieStateManager.java:224) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at org.apache.bookkeeper.bookie.BookieStateManager$2.call(BookieStateManager.java:220) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
           at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
           at java.lang.Thread.run(Thread.java:834) [?:?]
   WARNING: An illegal reflective access operation has occurred
   ```
   
   When I roll back the log4j version to 2.14.1, and run bookies in local, the log is normal
   ```
   2022-04-02T11:57:57,582 - INFO  - [main:LocalBookKeeper@458] - Using configuration file /Users/hangc/Workspace/sourcecode/CR/bookkeeper/bookkeeper-dist/all/target/bookkeeper-all-4.15.0-SNAPSHOT-bak/conf/bk_server.conf
   2022-04-02T11:57:57,593 - INFO  - [main:LocalBookKeeper@90] - Running 1 bookie(s) on zk ensemble = '127.0.0.1:2181'.
   2022-04-02T11:57:57,595 - INFO  - [main:LocalBookKeeper@120] - Starting ZK server
   2022-04-02T11:57:57,608 - INFO  - [main:ZookeeperBanner@42] - 
   2022-04-02T11:57:57,609 - INFO  - [main:ZookeeperBanner@42] -   ______                  _                                          
   2022-04-02T11:57:57,610 - INFO  - [main:ZookeeperBanner@42] -  |___  /                 | |                                         
   2022-04-02T11:57:57,610 - INFO  - [main:ZookeeperBanner@42] -     / /    ___     ___   | | __   ___    ___   _ __     ___   _ __   
   2022-04-02T11:57:57,611 - INFO  - [main:ZookeeperBanner@42] -    / /    / _ \   / _ \  | |/ /  / _ \  / _ \ | '_ \   / _ \ | '__|
   2022-04-02T11:57:57,611 - INFO  - [main:ZookeeperBanner@42] -   / /__  | (_) | | (_) | |   <  |  __/ |  __/ | |_) | |  __/ | |    
   2022-04-02T11:57:57,611 - INFO  - [main:ZookeeperBanner@42] -  /_____|  \___/   \___/  |_|\_\  \___|  \___| | .__/   \___| |_|
   2022-04-02T11:57:57,612 - INFO  - [main:ZookeeperBanner@42] -                                               | |                     
   2022-04-02T11:57:57,612 - INFO  - [main:ZookeeperBanner@42] -                                               |_|                     
   2022-04-02T11:57:57,612 - INFO  - [main:ZookeeperBanner@42] - 
   2022-04-02T11:57:57,630 - INFO  - [main:Environment@98] - Server environment:zookeeper.version=3.6.2--803c7f1a12f85978cb049af5e4ef23bd8b688715, built on 09/04/2020 12:44 GMT
   2022-04-02T11:57:57,630 - INFO  - [main:Environment@98] - Server environment:host.name=macbook-pro-3.lan
   2022-04-02T11:57:57,630 - INFO  - [main:Environment@98] - Server environment:java.version=11.0.12
   2022-04-02T11:57:57,630 - INFO  - [main:Environment@98] - Server environment:java.vendor=Oracle Corporation
   2022-04-02T11:57:57,630 - INFO  - [main:Environment@98] - Server environment:java.home=/Library/Java/JavaVirtualMachines/jdk-11.0.12.jdk/Contents/Home
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] RaulGracia commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
RaulGracia commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-937818625


   With current changes, log4j1.2 seems to be completely removed:
   [dependency-tree.txt](https://github.com/apache/bookkeeper/files/7303886/dependency-tree.txt)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] zymap commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
zymap commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-945306603


   @RaulGracia Would you like to address the new comment in another PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r730377059



##########
File path: bookkeeper-common/pom.xml
##########
@@ -74,6 +74,26 @@
       <artifactId>commons-lang3</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+      <version>${slf4j.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.logging.log4j</groupId>
+      <artifactId>log4j-1.2-api</artifactId>
+      <version>${log4j.version}</version>

Review comment:
       Here we should not add these deps in common.
   The common and server packages should not bring in any implementation 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-942444737


   @RaulGracia can you please resolve the conflicts ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r723718074



##########
File path: bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/MdcUtils.java
##########
@@ -23,18 +23,18 @@
 
 import java.util.Map;
 
-import org.slf4j.MDC;
+import org.apache.logging.log4j.ThreadContext;

Review comment:
       It looks like this is a breaking change.
   When you use slf4j you do not depend on the actual implementation.
   Now here you are forcing users to use log4j2




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on a change in pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#discussion_r723718074



##########
File path: bookkeeper-common/src/main/java/org/apache/bookkeeper/common/util/MdcUtils.java
##########
@@ -23,18 +23,18 @@
 
 import java.util.Map;
 
-import org.slf4j.MDC;
+import org.apache.logging.log4j.ThreadContext;

Review comment:
       It looks like this is a breaking change.
   When you use slf4j you do not depend on the actual implementation.
   Now here you are forcing users to use log4j2




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [bookkeeper] eolivelli commented on pull request #2816: Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571

Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2816:
URL: https://github.com/apache/bookkeeper/pull/2816#issuecomment-942444737


   @RaulGracia can you please resolve the conflicts ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org