[GitHub] JonZeolla opened a new pull request #24: METRON-2000: Fix bro plugin docker line counting for BRO_COUNT

[GitBox <gi...@apache.org>]

JonZeolla opened a new pull request #24: METRON-2000: Fix bro plugin docker line counting for BRO_COUNT
URL: https://github.com/apache/metron-bro-plugin-kafka/pull/24
 
 
   ## Contributor Comments
   This fixes an issue with the BRO_COUNT line counting in docker for x509 and http logs.
   
   The issue with x509 was that [`log-hostcerts-only.bro`](https://docs.zeek.org/en/stable/scripts/policy/protocols/ssl/log-hostcerts-only.bro.html) is loaded by default, which limits what is written to `x509.log` but does not affect what is sent to kafka.
   
   The issue with http was that it assumed that any line that contained `#` was a comment.  Because URIs are able to contain fragments (which contain `#`), it was miscounting.
   
   ### Testing
   Run `./run_end_to_end.sh` (with optional --skip-docker-build if you already have the containers built locally) and confirm all BRO_COUNT and KAFKA_COUNT numbers match.  Previously the http and x509 log counts did not match.
   
   ## Pull Request Checklist
   
   Thank you for submitting a contribution to Apache Metron's Bro kafka writer plugin.
   
   In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
   - [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
   
   ### For code changes:
   - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
   - [ ] Have you included steps or a guide to how the change may be verified and tested manually?
   - [ ] Have you ensured that the full suite of tests and checks have been executed via:
     ```
     bro-pkg test $GITHUB_USERNAME/metron-bro-plugin-kafka --version $BRANCH
     ```
   - [ ] Have you written or updated unit tests and or integration tests to verify your changes?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] Have you verified the basic functionality of the build by building and running locally with Apache Metron's [Vagrant full-dev environment](https://github.com/apache/metron/tree/master/metron-deployment/development/centos6) or the equivalent?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services