You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/02/15 10:47:00 UTC
[jira] [Commented] (HADOOP-16113) Your project apache/hadoop is
using buggy third-party libraries [WARNING]
[ https://issues.apache.org/jira/browse/HADOOP-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769183#comment-16769183 ]
Steve Loughran commented on HADOOP-16113:
-----------------------------------------
thank you for this, it's always good to get a review of what issues other people know about
Upgrading dependencies is potentially a traumatic process. See [fear of dependencies|http://steveloughran.blogspot.com/2016/05/fear-of-dependencies.html] for a summary of my feelings there; HADOOP-9991 "upgrade to the latest version" for the eternal problem.
h3. Every update of every library breaks something, somewhere. Possibly transitively downstream.
That's a key problem we have. Java 9 modules promises better isolation for the transitive issue, but there's still our own code to worry about.
Looking at that list, apache httpclient is one we should be worrying about, because it is retrieving content from remote sites -if anything malicious can cause problems then we don't want that. Commons-io probably too. The others? I'm not sure.
FWIW, didn't know we were using Log4J2 at all; we'd stayed on 1.x for consistent configuration, through the commons-logging and SLF4J APIs. We'll have to see about getting the Ozone team to upgrade log4j 2 and then tell us how we went,
Anyway, regarding the other issues, it's the classic triage "is a bug worth fixing" problem as applied to upgrades. We tend to lag, just out of fear of change
BTW, within JIRA s short link "LANG-1397" is all we need. thanks
> Your project apache/hadoop is using buggy third-party libraries [WARNING]
> -------------------------------------------------------------------------
>
> Key: HADOOP-16113
> URL: https://issues.apache.org/jira/browse/HADOOP-16113
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Kaifeng Huang
> Priority: Major
>
> Hi, there!
> We are a research team working on third-party library analysis. We have found that some widely-used third-party libraries in your project have major/critical bugs, which will degrade the quality of your project. We highly recommend you to update those libraries to new versions.
> We have attached the buggy third-party libraries and corresponding jira issue links below for you to have more detailed information.
> 1. org.apache.logging.log4j log4j-core(hadoop-hdds/common/pom.xml)
> version: 2.11.0
> Jira issues:
> Log4j2 throws NoClassDefFoundError in Java 9
> affectsVersions:2.10.0,2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2129?filter=allopenissues
> Empty Automatic-Module-Name Header
> affectsVersions:2.10.0,2.11.0,3.0.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2254?filter=allopenissues
> gc-free mixed async loging loses parameter values after the first appender
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2301?filter=allopenissues
> Log4j 2.10+not working with SLF4J 1.8 in OSGI environment
> affectsVersions:2.10.0,2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2305?filter=allopenissues
> AsyncQueueFullMessageUtil causes unparsable message output
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2318?filter=allopenissues
> AbstractLogger NPE hides actual cause when getFormat returns null
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2320?filter=allopenissues
> AsyncLogger without specifying a level always uses ERROR
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2321?filter=allopenissues
> Errors thrown in formatting may stop background threads
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2333?filter=allopenissues
> JsonLayout not working with AsyncLoggerContextSelector in 2.11.0
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2341?filter=allopenissues
> Typo in log4j-api Activator
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2343?filter=allopenissues
> PropertiesUtil.reload() might throw NullPointerException
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2355?filter=allopenissues
> NameAbbreviator skips first fragments
> affectsVersions:2.11.0,2.11.1
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2365?filter=allopenissues
> Outputs wrong message when used within overridden Throwable method
> affectsVersions:2.8.1,2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2368?filter=allopenissues
> StringBuilder escapeJson performs unnecessary Memory Allocations
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2373?filter=allopenissues
> fix the CacheEntry map in ThrowableProxy#toExtendedStackTrace to be put and gotten with same key
> affectsVersions:2.6.2,2.7,2.8,2.8.1,2.8.2,2.9.0,2.9.1,2.10.0,2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2389?filter=allopenissues
> Fix incorrect links in Log4j web documentation.
> affectsVersions:2.11.0
> https://issues.apache.org/jira/projects/LOG4J2/issues/LOG4J2-2390?filter=allopenissues
> 2. org.apache.httpcomponents httpclient(hadoop-project/pom.xml)
> version: 4.5.2
> Jira issues:
> org.apache.http.impl.client.AbstractHttpClient#createClientConnectionManager Does not account for context class loader
> affectsVersions:4.4.1;4.5;4.5.1;4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1727?filter=allopenissues
> Memory Leak in OSGi support
> affectsVersions:4.4.1;4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1749?filter=allopenissues
> SystemDefaultRoutePlanner: Possible null pointer dereference
> affectsVersions:4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1766?filter=allopenissues
> Null pointer dereference in EofSensorInputStream and ResponseEntityProxy
> affectsVersions:4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1767?filter=allopenissues
> [OSGi] WeakList needs to support "clear" method
> affectsVersions:4.5.2;5.0 Alpha1
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1772?filter=allopenissues
> [OSGi] HttpProxyConfigurationActivator does not unregister HttpClientBuilderFactory
> affectsVersions:4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1773?filter=allopenissues
> Why is Retry around Redirect and not the other way round
> affectsVersions:4.5.2
> https://issues.apache.org/jira/projects/HTTPCLIENT/issues/HTTPCLIENT-1800?filter=allopenissues
> 3. commons-cli commons-cli(hadoop-project/pom.xml)
> version: 1.2
> Jira issues:
> Unable to select a pure long option in a group
> affectsVersions:1.0;1.1;1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-182?filter=allopenissues
> Clear the selection from the groups before parsing
> affectsVersions:1.0;1.1;1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-183?filter=allopenissues
> Commons CLI incorrectly stripping leading and trailing quotes
> affectsVersions:1.1;1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-185?filter=allopenissues
> Coding error: OptionGroup.setSelected causes java.lang.NullPointerException
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-191?filter=allopenissues
> StringIndexOutOfBoundsException in HelpFormatter.findWrapPos
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-193?filter=allopenissues
> HelpFormatter strips leading whitespaces in the footer
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-207?filter=allopenissues
> OptionBuilder only has static methods; yet many return an OptionBuilder instance
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-224?filter=allopenissues
> Unable to properly require options
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-230?filter=allopenissues
> OptionValidator Implementation Does Not Agree With JavaDoc
> affectsVersions:1.2
> https://issues.apache.org/jira/projects/CLI/issues/CLI-241?filter=allopenissues
> 4. commons-io commons-io(hadoop-project/pom.xml)
> version: 2.5
> Jira issues:
> ant test fails - resources missing from test classpath
> affectsVersions:2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-451?filter=allopenissues
> Exceptions are suppressed incorrectly when copying files.
> affectsVersions:2.4;2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-502?filter=allopenissues
> ThresholdingOutputStream.thresholdReached() results in FileNotFoundException
> affectsVersions:2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-512?filter=allopenissues
> Tailer.run race condition runaway logging
> affectsVersions:2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-528?filter=allopenissues
> Thread bug in FileAlterationMonitor#stop(int)
> affectsVersions:2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-535?filter=allopenissues
> 2.5 ExceptionInInitializerError
> affectsVersions:2.5
> https://issues.apache.org/jira/projects/IO/issues/IO-536?filter=allopenissues
> 5. commons-codec commons-codec(hadoop-project/pom.xml)
> version: 1.11
> Jira issues:
> InputStream not closed
> affectsVersions:1.10;1.11
> https://issues.apache.org/jira/projects/CODEC/issues/CODEC-225?filter=allopenissues
> 6. org.apache.commons commons-lang3(hadoop-project/pom.xml)
> version: 3.7
> Jira issues:
> NPE from SystemUtils.isJavaVersionAtLeast under Java 11 EA
> affectsVersions:3.7
> https://issues.apache.org/jira/projects/LANG/issues/LANG-1384?filter=allopenissues
> WordUtils.wrap throws StringIndexOutOfBoundsException when wrapLength is Integer.MAX_VALUE
> affectsVersions:3.7
> https://issues.apache.org/jira/projects/LANG/issues/LANG-1397?filter=allopenissues
> Sincerely~
> FDU Software Engineering Lab
> Feb 15th,2019
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org