[jira] [Commented] (METRON-726) Clean up mvn site generation

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874102#comment-15874102 ] 

ASF GitHub Bot commented on METRON-726:
---------------------------------------

GitHub user justinleet opened a pull request:

    https://github.com/apache/incubator-metron/pull/459

    METRON-726: Clean up mvn site generation

    ### Summary
    An initial attempt to add checkstyle expanded a bit once it turned out that `mvn site` appears to not work right now (and probably hasn't for a long while). More background on some of the issues encountered (and error traces) is in the ticket itself.
    
    A small set of cleanups is included here, and is pretty easy to split out if we want it to be separate PRs (it mostly just got cleaned up while I was trying to fix things one by one, but for example, checkstyle could be included in a separate PR). It's small enough though, that I just left it together since it makes the generated site fairly nice and complete.
    
    ### Changes
    - The major change is to drop EMMA for JaCoCo. As noted in the ticket, it's unlikely to be worth the erffort to get EMMA working, given that it's not actively developed and it seems likely that EMMA just isn't compatible with Java 8 code (not terribly surprising given how old it is). I put in JaCoCo mostly to keep (make?) code coverage available, but I can easily remove it if we'd rather have a discussion on a different tool, or just a separate PR.
    - Added maven-site-plugin to top level pom
    - Refactored most reporting (maven-project-info-reports-plugin, maven-pmd-plugin, and code coverage) to go through top level pom, given that most of our reporting is exactly the same throughout the child poms.
    - Explicitly setup up maven-project-info-report-plugin reports (neither CLI param nor POM param skipped dependency reports correctly. Unsure why that is. If someone knows, I'd love to just change that)
    - Added maven-jxr-plugin to fix warnings in the site build
    - Added names to poms to make sure they show up nicely in site
    - Deploy set to /tmp/metron/site for now. Should eventually be adjusted to be somewhere useful (I believe other components like Hadoop do something nicer with it within Apache, but I'd need to look into it). I'll create a ticket for it, assuming we're good with this PR (mostly) as-is. Otherwise, I'll wait and see what spins out of discussion here.
    - Updated PMD to 3.7
    
    #### Testing
    To run everything, including code coverage up:
    `mvn clean install site:site site:deploy site:stage-deploy`
    
    Open up /tmp/metron/site/index.html in a browser. Everything (other than Javadoc) should be available, and clicking through modules should work because of the `site:stage-deploy`
    
    Skipping tests allows code coverage to be skipped.
    
    I also ran up quick-dev and made sure data passed through topologies. I didn't run up other components, because mostly the reporting section of the poms is touched, so if they built and the reporting built it seemed like enough. I can definitely spin up other components if anybody wants to be more thorough, though.
    
    #### Notes
    - Javadocs are currently broken outside of this ticket. Separate Jira/PR is open for that issue.
    - Reason for `site:stage-deploy` to is get the parent-child pom links working while testing. Otherwise, you can't navigate through the module structure.
    - Currently almost 25k(!) checkstyle issues.
    - It's possible to import a check style file in IntelliJ. I could export this for general use (modified it to use two spaces + possibly things like line length). I tested reformatting all Java files like this, and it clears out several thousand checkstyle issues. And makes the code more consistent. Obviously, that's a bit aggressive and touched things like autogenerated files, but it makes it clear we have more issues than trivially solved by code formatting.
    
    
    
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/justinleet/incubator-metron site_fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/459.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #459
    
----
commit 7028066c3e2cd3823aa15cf7972724e5f61ca05c
Author: justinjleet <ju...@gmail.com>
Date:   2017-02-20T02:00:13Z

    Moving away from emma to jacoco and fixing some other issues

commit f56e8bbbc0863a6a535e663594d049620d53c7d6
Author: justinjleet <ju...@gmail.com>
Date:   2017-02-20T02:44:51Z

    Fixing up and moving PMD upward in POM

----


> Clean up mvn site generation
> ----------------------------
>
>                 Key: METRON-726
>                 URL: https://issues.apache.org/jira/browse/METRON-726
>             Project: Metron
>          Issue Type: Bug
>            Reporter: Justin Leet
>            Assignee: Justin Leet
>            Priority: Minor
>
> Right now there's a couple issues with running mvn:site.  The most obvious is that EMMA appears to not work at all, but in attempting to fix that, several other issues came to light.
> Error seen:
> {code}
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on project metron-maas-common: failed to get report for org.codehaus.mojo:emma-maven-plugin: Failed to execute goal org.codehaus.mojo:emma-maven-plugin:1.0-alpha-3:instrument (report:emma) on project metron-maas-common: Execution report:emma of goal org.codehaus.mojo:emma-maven-plugin:1.0-alpha-3:instrument failed: CONSTANT_info: invalid tag value [18] -> [Help 1]
> {code}
> After commenting out everything EMMA, there are still some issues seen:
> {code}
> [WARNING] Unable to process class org/apache/metron/test/converters/BinaryConverters.class in JarAnalyzer File /Users/jleet/.m2/repository/org/apache/metron/metron-test-utilities/0.3.1/metron-test-utilities-0.3.1.jar
> org.apache.bcel.classfile.ClassFormatException: Invalid byte tag in constant pool: 18
> {code}
> This seems to be a Java 8 issue, which means that EMMA likely is impossible to make work.  I'm unsure that's the root cause, but given the age of EMMA plus (the outdated version of) BCEL thowing a very similar issue implies that the Java version is related. This also implies that our {{mvn site}} hasn't worked in a long time.
> Cleaning this up should include at least
> * Getting code coverage working again
> * Consolidating reporting in our poms.  A lot of it is repeated everywhere we have Java.
> * Ensure we can actually generate and look through the site.
> * METRON-725 fixes Javadoc, so reporting will still have issues until that is taken care of.
> * Apparently checkstyle got dropped at some point.  It's easy enough to add in, and can be taken care of here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)