You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2015/03/13 01:57:38 UTC

[jira] [Updated] (SAMZA-597) Support a JSON encoder for Log4J StreamAppender

     [ https://issues.apache.org/jira/browse/SAMZA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Riccomini updated SAMZA-597:
----------------------------------
    Attachment: SAMZA-597-0.patch

Attaching a patch. RB at:

https://reviews.apache.org/r/32006

Changes:

# Wrote a LoggingEventJsonSerde, and LoggingEventJsonSerdeFactory.
# Added a config to define whether the JSON encoding should include [LocationInfo|https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/spi/LocationInfo.html]. Defaulted to false.
# Updated configuration-table.html docs, and logging.md docs.
# Updated tests to make sure everything still works.
# Misc cleanup in Log4jSystemConfig to shrink lines of code.
# Changed default serializer from string to JSON. This matches the default CheckpointManager serialization format, and also makes it easier for Samza to integrate with ELK. This format is also less lossy than the LoggingEventStringSerde.
# Did not implement JSON-to-LoggingEvent decoding. Wasn't required for ELK integration.
# Throw an exception if the containerName SystemProperty is empty.

Also ran with hello-samza, and an internal job. Fully integrated the messages with ELK, and it worked.

> Support a JSON encoder for Log4J StreamAppender
> -----------------------------------------------
>
>                 Key: SAMZA-597
>                 URL: https://issues.apache.org/jira/browse/SAMZA-597
>             Project: Samza
>          Issue Type: Bug
>          Components: log4j
>    Affects Versions: 0.9.0
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>             Fix For: 0.9.0
>
>         Attachments: SAMZA-597-0.patch
>
>
> We are trying to hook a Samza job up to [ELK|https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-and-visualize-logs-on-ubuntu-14-04]. During this exercise, I found that the only format available is a String-encoded LoggingEvent. Logstash has a [well-defined|https://github.com/logstash/log4j-jsonevent-layout] LoggingEvent JSON encoding. It would be nice if Samza supported this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)