You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by dlyle65535 <gi...@git.apache.org> on 2016/07/26 21:52:55 UTC

[GitHub] incubator-metron pull request #202: METRON-242: Remove Squid Pattern

GitHub user dlyle65535 opened a pull request:

    https://github.com/apache/incubator-metron/pull/202

    METRON-242: Remove Squid Pattern

    Squid emits access log lines with and without a ip_dst_addr. I replaced the Squid grok pattern with a pattern that:
    
    1) Better handles the non-whitespace characters that appear after the timestamp
    2) Is tolerant to different characters between the url and the ip_dst_addr
    3) Makes ip_dst_addr optional
    
    In order to test both, I refactored the GrokParserTest to allow testing of different patterns with a single grok statement.
    
    This was tested on quick-dev-platform.
    
    Please don't let the JIRA title fool you, the Requester asked to remove it or add a better one. I chose to add a better one.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dlyle65535/incubator-metron METRON-242

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #202
    
----
commit 05f6738dcb5bc8340f321726570e32582c06c0a3
Author: David Lyle <dl...@gmail.com>
Date:   2016-07-26T15:22:08Z

    METRON-242: Remove Squid Pattern

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #202: METRON-242: Remove Squid Pattern

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/incubator-metron/pull/202#discussion_r72628148
  
    --- Diff: metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/SampleGrokParserTest.java ---
    @@ -55,12 +57,11 @@
       @Multiline
       public String expectedParsedString;
     
    -  public String getExpectedParsedString() {
    -    return expectedParsedString;
    -  }
     
    -  public String getRawMessage() {
    -    return "1453994987000|2016-01-28 15:29:48|   0.000|   0.000|  6|                          216.21.170.221|   80|                               10.0.2.15|39468|      AS|       0|       0|       0|22efa001|00000000|000|000|       1|      44|       0|       0|    0|idle";
    +  @Override
    +  public Map getTestData() {
    +    return new HashMap<String,String>();
    +    //return "1453994987000|2016-01-28 15:29:48|   0.000|   0.000|  6|                          216.21.170.221|   80|                               10.0.2.15|39468|      AS|       0|       0|       0|22efa001|00000000|000|000|       1|      44|       0|       0|    0|idle";
    --- End diff --
    
    Is the commented code still needed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #202: METRON-242: Remove Squid Pattern

Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:

    https://github.com/apache/incubator-metron/pull/202
  
    Easiest way to test this is to run the SquidParserTest unit test.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #202: METRON-242: Remove Squid Pattern

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-metron/pull/202


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #202: METRON-242: Remove Squid Pattern

Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on a diff in the pull request:

    https://github.com/apache/incubator-metron/pull/202#discussion_r72647884
  
    --- Diff: metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/SampleGrokParserTest.java ---
    @@ -55,12 +57,11 @@
       @Multiline
       public String expectedParsedString;
     
    -  public String getExpectedParsedString() {
    -    return expectedParsedString;
    -  }
     
    -  public String getRawMessage() {
    -    return "1453994987000|2016-01-28 15:29:48|   0.000|   0.000|  6|                          216.21.170.221|   80|                               10.0.2.15|39468|      AS|       0|       0|       0|22efa001|00000000|000|000|       1|      44|       0|       0|    0|idle";
    +  @Override
    +  public Map getTestData() {
    +    return new HashMap<String,String>();
    +    //return "1453994987000|2016-01-28 15:29:48|   0.000|   0.000|  6|                          216.21.170.221|   80|                               10.0.2.15|39468|      AS|       0|       0|       0|22efa001|00000000|000|000|       1|      44|       0|       0|    0|idle";
    --- End diff --
    
    No, it is not. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #202: METRON-242: Remove Squid Pattern

Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on the issue:

    https://github.com/apache/incubator-metron/pull/202
  
    +1 by inspection


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #202: METRON-242: Remove Squid Pattern

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/incubator-metron/pull/202
  
    +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---