You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Gary Gregory (JIRA)" <ji...@apache.org> on 2019/07/05 22:48:00 UTC
[jira] [Updated] (BCEL-317) Pluggable cache for ConstantUtf8
[ https://issues.apache.org/jira/browse/BCEL-317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Gregory updated BCEL-317:
------------------------------
Description:
Please see the comment https://issues.apache.org/jira/browse/BCEL-317?focusedCommentId=16879552&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16879552 for resolution.
----
Follow-up of BCEL-186. This enhancement is to provide an option to cache ConstantUtf8 instances that have the same value.
Email thread: [bcel] Idea to share ConstantUtf8 of same value among JavaClass instances
--------------------
We use BCEL library to inspect Java class. Thank you for the great library.
When our tool checks classes in ~200 jar files, it creates more than 2 million BCEL ConstantUtf8 instances. I suspect many of them share the same values such as "java.lang.String".
Without cache, my tool created 2,6 million ConstantUtf8 instances (before failing OutOfMemoryError: GC overhead limit exceeded) to check ~200 jar files.
!2644k_constantutf8_without_getCachedInstance.png!
With the cache, my tool created just 0.6 million ConstantUtf8 instances. It didn't throw the OutOfMemoryError.
!621k_constantutf8_with_getCachedInstance.png!
Old commit that made ConstantUtf8.getInstance [https://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/Constant.java?r1=1481383&r2=1481382&pathrev=1481383]
The ConstantUtf8.getInstance has been unused in BCEL since BCEL-186 [http://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/ConstantUtf8.java?r1=1652541&r2=1652540&pathrev=1652541]
h1. How our project uses BCEL
- Our tool (Linkage Checker) creates BCEL's ClassPathRepository with around 200 JAR files as its input class path in ClassDumper.createClassRepository([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L83])
{code:java}
private static Repository createClassRepository(List<Path> paths) {
ClassPath classPath = new LinkageCheckClassPath(paths);
return new ClassPathRepository(classPath); # This is BCEL API
}{code}
- reads JavaClasses one by one in ClassDumper.listClassesInJar ([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L360]) through the ClassPathRepository.
{code:java}
private ImmutableSet<JavaClass> listClassesInJar(Path jar) throws IOException {
ImmutableSet.Builder<JavaClass> javaClasses = ImmutableSet.builder();
for (String className : listClassNamesInJar(jar)) {
try {
JavaClass javaClass = classRepository.loadClass(className); # This is BCEL API. ConstantUtf8 is not cached.
javaClasses.add(javaClass);
...(omit)
return javaClasses.build();
}{code}
Stacktrace from the ClassDumper.listClassesInJar to BCEL's ConstantUtf8.getInstance would look like this:
{code:java}
ConstantUtf8.getInstance
Constant.readConstant
ConstantPool.<init>
ClassParser.readConstantPool
ClassParser.parse
ClassPathRepository.loadClass
ClassPathRepository.loadClass
ClassDumper.listClassesInJar {code}
- Use the JavaClass instances to check if any symbol references (entries in constant pool [Java Virtual Machine Specification 4.4. The Constant Pool|https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4]: CONSTANT_Class, CONSTANT_Fieldref, CONSTANT_Methodref, and CONSTANT_InterfaceMethodref) has invalid referent in the class path.
was:
Follow-up of BCEL-186. This enhancement is to provide an option to cache ConstantUtf8 instances that have the same value.
Email thread: [bcel] Idea to share ConstantUtf8 of same value among JavaClass instances
--------------------
We use BCEL library to inspect Java class. Thank you for the great library.
When our tool checks classes in ~200 jar files, it creates more than 2 million BCEL ConstantUtf8 instances. I suspect many of them share the same values such as "java.lang.String".
Without cache, my tool created 2,6 million ConstantUtf8 instances (before failing OutOfMemoryError: GC overhead limit exceeded) to check ~200 jar files.
!2644k_constantutf8_without_getCachedInstance.png!
With the cache, my tool created just 0.6 million ConstantUtf8 instances. It didn't throw the OutOfMemoryError.
!621k_constantutf8_with_getCachedInstance.png!
Old commit that made ConstantUtf8.getInstance [https://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/Constant.java?r1=1481383&r2=1481382&pathrev=1481383]
The ConstantUtf8.getInstance has been unused in BCEL since BCEL-186 [http://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/ConstantUtf8.java?r1=1652541&r2=1652540&pathrev=1652541]
h1. How our project uses BCEL
- Our tool (Linkage Checker) creates BCEL's ClassPathRepository with around 200 JAR files as its input class path in ClassDumper.createClassRepository([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L83])
{code:java}
private static Repository createClassRepository(List<Path> paths) {
ClassPath classPath = new LinkageCheckClassPath(paths);
return new ClassPathRepository(classPath); # This is BCEL API
}{code}
- reads JavaClasses one by one in ClassDumper.listClassesInJar ([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L360]) through the ClassPathRepository.
{code:java}
private ImmutableSet<JavaClass> listClassesInJar(Path jar) throws IOException {
ImmutableSet.Builder<JavaClass> javaClasses = ImmutableSet.builder();
for (String className : listClassNamesInJar(jar)) {
try {
JavaClass javaClass = classRepository.loadClass(className); # This is BCEL API. ConstantUtf8 is not cached.
javaClasses.add(javaClass);
...(omit)
return javaClasses.build();
}{code}
Stacktrace from the ClassDumper.listClassesInJar to BCEL's ConstantUtf8.getInstance would look like this:
{code:java}
ConstantUtf8.getInstance
Constant.readConstant
ConstantPool.<init>
ClassParser.readConstantPool
ClassParser.parse
ClassPathRepository.loadClass
ClassPathRepository.loadClass
ClassDumper.listClassesInJar {code}
- Use the JavaClass instances to check if any symbol references (entries in constant pool [Java Virtual Machine Specification 4.4. The Constant Pool|https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4]: CONSTANT_Class, CONSTANT_Fieldref, CONSTANT_Methodref, and CONSTANT_InterfaceMethodref) has invalid referent in the class path.
> Pluggable cache for ConstantUtf8
> --------------------------------
>
> Key: BCEL-317
> URL: https://issues.apache.org/jira/browse/BCEL-317
> Project: Commons BCEL
> Issue Type: Improvement
> Components: Main
> Affects Versions: 6.3.1
> Reporter: Tomo Suzuki
> Priority: Minor
> Attachments: 2644k_constantutf8_without_getCachedInstance.png, 621k_constantutf8_with_getCachedInstance.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Please see the comment https://issues.apache.org/jira/browse/BCEL-317?focusedCommentId=16879552&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16879552 for resolution.
> ----
> Follow-up of BCEL-186. This enhancement is to provide an option to cache ConstantUtf8 instances that have the same value.
>
> Email thread: [bcel] Idea to share ConstantUtf8 of same value among JavaClass instances
> --------------------
> We use BCEL library to inspect Java class. Thank you for the great library.
>
> When our tool checks classes in ~200 jar files, it creates more than 2 million BCEL ConstantUtf8 instances. I suspect many of them share the same values such as "java.lang.String".
>
> Without cache, my tool created 2,6 million ConstantUtf8 instances (before failing OutOfMemoryError: GC overhead limit exceeded) to check ~200 jar files.
> !2644k_constantutf8_without_getCachedInstance.png!
>
> With the cache, my tool created just 0.6 million ConstantUtf8 instances. It didn't throw the OutOfMemoryError.
> !621k_constantutf8_with_getCachedInstance.png!
>
> Old commit that made ConstantUtf8.getInstance [https://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/Constant.java?r1=1481383&r2=1481382&pathrev=1481383]
> The ConstantUtf8.getInstance has been unused in BCEL since BCEL-186 [http://svn.apache.org/viewvc/commons/proper/bcel/trunk/src/main/java/org/apache/bcel/classfile/ConstantUtf8.java?r1=1652541&r2=1652540&pathrev=1652541]
>
> h1. How our project uses BCEL
> - Our tool (Linkage Checker) creates BCEL's ClassPathRepository with around 200 JAR files as its input class path in ClassDumper.createClassRepository([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L83])
> {code:java}
> private static Repository createClassRepository(List<Path> paths) {
> ClassPath classPath = new LinkageCheckClassPath(paths);
> return new ClassPathRepository(classPath); # This is BCEL API
> }{code}
> - reads JavaClasses one by one in ClassDumper.listClassesInJar ([source code URL|https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/239fb82e368b2c181e358359f758b33c0a122787/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/ClassDumper.java#L360]) through the ClassPathRepository.
> {code:java}
> private ImmutableSet<JavaClass> listClassesInJar(Path jar) throws IOException {
> ImmutableSet.Builder<JavaClass> javaClasses = ImmutableSet.builder();
> for (String className : listClassNamesInJar(jar)) {
> try {
> JavaClass javaClass = classRepository.loadClass(className); # This is BCEL API. ConstantUtf8 is not cached.
> javaClasses.add(javaClass);
> ...(omit)
> return javaClasses.build();
> }{code}
> Stacktrace from the ClassDumper.listClassesInJar to BCEL's ConstantUtf8.getInstance would look like this:
> {code:java}
> ConstantUtf8.getInstance
> Constant.readConstant
> ConstantPool.<init>
> ClassParser.readConstantPool
> ClassParser.parse
> ClassPathRepository.loadClass
> ClassPathRepository.loadClass
> ClassDumper.listClassesInJar {code}
> - Use the JavaClass instances to check if any symbol references (entries in constant pool [Java Virtual Machine Specification 4.4. The Constant Pool|https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4]: CONSTANT_Class, CONSTANT_Fieldref, CONSTANT_Methodref, and CONSTANT_InterfaceMethodref) has invalid referent in the class path.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)