[jira] [Comment Edited] (KAFKA-1722) static analysis code coverage for pci audit needs

["Ashish Kumar Singh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14281977#comment-14281977 ] 

Ashish Kumar Singh edited comment on KAFKA-1722 at 1/18/15 10:18 PM:
---------------------------------------------------------------------

After elaborate trials of various available tools and compatibility plugins, below is a brief summary.

*We need to measure code coverage of following modules*
# Core (in Scala, with a little Java code)
# Clients (in Java)
Other modules do not have tests.

*Lang specific coverage tools*
# Java, [JaCoCo|http://www.eclemma.org/jacoco/] appears to be a decent tool, which provides line and branch coverage.
# Scala, [Scoverage|http://scoverage.org/] provides line and branch coverage.

*Coverage summary*
[SonarQube|http://www.sonarqube.org/] is a widely used tool that provides the capability to merge compatibility reports form various modules and present an overall report. Sonar uses plugins to parse and understand coverage report of an underlying sub-module of a project. A project can have sub-modules with different coverage tools, i.e., in different languages. We need following plugins for Kafka.
# Sonar-Jacoco (v2.1)
# Sonar-scoverage-plugin

*Issues*
# Sonar-socverage-plugin depends on [scalac-scoverage-plugin|https://github.com/scoverage/scalac-scoverage-plugin]. scalac-scoverage-plugin can be used in a gradle project using [gradle-scoverage|https://github.com/scoverage/gradle-scoverage]. gradle-scoverage,as of now, only publishes html and cobertura report. However, scalac-scoverage-plugin needs scoverage report to be able to parse it.
In short, sonar can not report coverage for scala project as of now. A full coverage report does get generated for scala project, but it would not show up in overall report. I have discussed this with the collaborators of gradle-scoverage and they are working on it.
# Scala 2.10 is not supported by scalac-scoverage-plugin, [detailed discussion|https://github.com/scoverage/scalac-scoverage-plugin/blob/master/2.10.md].

*OK, so where do we stand*
We can generate coverage reports, with line and branch coverage included, for core and clients sub modules.
We can generate a sonar summary report for the project, but that will only include coverage of clients sub module.
Coverage report, web report, for core module will have to be browsed separately.
As soon as gradle-scoverage start publishing scoverage report, [tracked here|https://github.com/scoverage/scalac-scoverage-plugin/issues/81], we can see core's coverage as well in the sonar summary report.

If this sounds ok then I can provide a patch.


was (Author: singhashish):
After elaborate trials of various available tools and compatibility plugins, below is a brief summary.

*We need to measure code coverage of following modules*
# Core (in Scala, with a little Java code)
# Clients (in Java)
Other modules do not have tests.

*Lang specific coverage tools*
# Java, [JaCoCo|http://www.eclemma.org/jacoco/] appears to be a decent tool, which provides line and branch coverage.
# Scala, [Scoverage|http://scoverage.org/] provides line and branch coverage.

*Coverage summary*
[SonarQube|http://www.sonarqube.org/] is a widely used tool that provides the capability to merge compatibility reports form various modules and present an overall report. Sonar uses plugins to parse and understand coverage report of an underlying sub-module of a project. A project can have sub-modules with different coverage tools, i.e., in different languages. We need following plugins for Kafka.
# Sonar-Jacoco (v2.1)
# Sonar-scoverage-plugin

*Issues*
# Sonar-socverage-plugin depends on [scalac-scoverage-plugin|https://github.com/scoverage/scalac-scoverage-plugin]. scalac-scoverage-plugin can be used in a gradle project using [gradle-scoverage|https://github.com/scoverage/gradle-scoverage]. gradle-scoverage,as of now, only publishes html and cobertura report. However, scalac-scoverage-plugin needs scoverage report to be able to parse it.
In short, sonar can not report coverage for scala project as of now. A full coverage report does get generated for scala project, but it would not show up in overall report. I have discussed this with the collaborators of gradle-scoverage and they are working on it.
# Scala 2.10 is not supported by scalac-scoverage-plugin, [detailed discussion|https://github.com/scoverage/scalac-scoverage-plugin/blob/master/2.10.md].

*OK, so where do we stand*
We can generate coverage reports, with line and branch coverage included, for core and clients sub modules.
We can generate a sonar summary report for the project, but that will only include coverage of clients sub module.
Coverage report, web report, for core module will have to be browsed separately.
As soon as gradle-scoverage start publishing scoverage report, we can see core's coverage as well in the sonar summary report.

If this sounds ok then I can provide a patch.

> static analysis code coverage for pci audit needs
> -------------------------------------------------
>
>                 Key: KAFKA-1722
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1722
>             Project: Kafka
>          Issue Type: Bug
>          Components: security
>            Reporter: Joe Stein
>            Assignee: Ashish Kumar Singh
>             Fix For: 0.9.0
>
>
> Code coverage is a measure used to describe the degree to which the source code of a product is tested. A product with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a product with low code coverage. Apart from PCI audit needs, increasing user base of Kafka makes it important to increase code coverage of Kafka. Something just can not be improved without being measured.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)