You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/06/02 20:38:04 UTC
[jira] [Commented] (OOZIE-2651) Set
javax.xml.parsers.DocumentBuilderFactory sys prop to make XML handling
faster
[ https://issues.apache.org/jira/browse/OOZIE-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035379#comment-16035379 ]
Hadoop QA commented on OOZIE-2651:
----------------------------------
Testing JIRA OOZIE-2651
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:red}-1{color} the patch contains 1 line(s) with tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
. {color:red}WARNING{color}: the current HEAD has 6 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 1959
. Tests rerun: 79
. Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM,
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3858/
> Set javax.xml.parsers.DocumentBuilderFactory sys prop to make XML handling faster
> ---------------------------------------------------------------------------------
>
> Key: OOZIE-2651
> URL: https://issues.apache.org/jira/browse/OOZIE-2651
> Project: Oozie
> Issue Type: Improvement
> Components: core
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: OOZIE-2651.001.patch, OOZIE-2651.002.patch, OOZIE-2651.003.patch
>
>
> We noticed from some jstacks that Oozie was spending a lot of time looking for XML related jars (we do a lot of XML parsing in Oozie and Hadoop code). For instance, we'd see hundreds of threads blocked here
> {noformat}
> "pool-5-thread-57" prio=10 tid=0x00007f9579923000 nid=0x6b7f waiting for monitor entry [0x00007f94225e3000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
> - waiting to lock <0x0000000500026148> (a org.apache.catalina.loader.WebappClassLoader)
> at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
> at javax.xml.parsers.FactoryFinder.getProviderClass(FactoryFinder.java:117)
> at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:178)
> at javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:333)
> at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:255)
> at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:121)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2526)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
> - locked <0x00000007eac5ea28> (a org.apache.hadoop.conf.Configuration)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1144)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1116)
> at org.apache.oozie.util.XConfiguration.copy(XConfiguration.java:206)
> ...
> {noformat}
> There's nothing we can do about the lock at the {{WebappClassLoader}}, but by setting the {{javax.xml.parsers.DocumentBuilderFactory}} system property to {{org.apache.xerces.jaxp.DocumentBuilderFactoryImpl}}, we can speed this up a lot by getting rid of all of the searching that it's doing to find the implementation.
> See https://docs.oracle.com/javase/7/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#newInstance()
> and https://www.databasesandlife.com/java-always-explicitly-specify-which-xml-parser-to-use/
> In our testing, we had a metric around creating a new {{JobClient}}, which also calls {{DocumentBuilderFactory.newInstance}}, and it dropped from ~3min max to ~2sec max (not an exact test of the savings here). And we also didn't see so much blocking on this in the jstacks anymore.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)