[jira] [Commented] (HADOOP-18500) Upgrade maven-shade-plugin to 3.3.0

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620550#comment-17620550 ] 

ASF GitHub Bot commented on HADOOP-18500:
-----------------------------------------

rshkv opened a new pull request, #5045:
URL: https://github.com/apache/hadoop/pull/5045

   ### Description of PR
   Upgrade maven-shade-plugin to 3.3.0. This avoids unnecessary rewrites of class files during relocation. 
   
   ### How was this patch tested?
   I built Hadoop locally and compared the byte code of the class files before and after the upgrade. Before, the byte code is different between classes inside relocated JARs and non-relocated JARs (e.g. in `hadoop-client-runtime` vs `hadoop-shaded-guava`). After, the byte code is the same.
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




> Upgrade maven-shade-plugin to 3.3.0
> -----------------------------------
>
>                 Key: HADOOP-18500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18500
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>            Reporter: Willi Raschkowski
>            Assignee: Ashutosh Gupta
>            Priority: Minor
>
> Maven-shade-plugin rewrites classes when moving them into {{hadoop-client}} JARs. That's true even when it doesn't actually need to modify the byte code of the classes, say for shading.
> We use a tool that checks for classpath duplicates that don't have equal byte codes. We noticed that it flags classes brought in via Hadoop, where one JAR containing them is {{hadoop-client-api}} and {{{}hadoop-client-runtime{}}}, and the other JAR is {{hadoop-common}} or {{{}hadoop-shaded-guava{}}}. The byte code for the same class is indeed different between the relocated and non-relocated JARs.
> This is because maven-shade-plugin, before 3.3.0, was rewriting class files even when the relocation was a "no-op". See MSHADE-391 and [apache/maven-shade-plugin#95|https://github.com/apache/maven-shade-plugin/pull/95].
> {quote}
> Maven Shade internally uses [ASM's {{ClassRemapper}}|https://asm.ow2.io/javadoc/org/objectweb/asm/commons/ClassRemapper.html] and defines a custom {{Remapper}} subclass, which takes care of relocation, partially doing the work by itself and partially delegating to the ASM parent class. An ASM {{ClassReader}} reads each class file from the original JAR and *unconditionally* writes it into a {{{}ClassWriter{}}}, plugging in the transformer.
> This transformation, even if not a single relocation (package name mapping) takes place, often leads to binary differences between original class and transformed class, because constant pool or stack map frames have been adjusted, not changing the functionality of the class, but making it look like something changed when comparing class files before and after the relocation process.
> {quote}
> Upgrading to maven-shade-plugin 3.3.0 fixes the unnecessary rewrite of classes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org