[jira] [Commented] (SAMZA-1460) StreamAppender does not explicitly create logging topic

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368047#comment-16368047 ] 

ASF GitHub Bot commented on SAMZA-1460:
---------------------------------------

GitHub user dnishimura opened a pull request:

    https://github.com/apache/samza/pull/423

    SAMZA-1460: StreamAppender does not explicitly create logging topic

    Creates the StreamAppender stream explicitly instead of relying on auto stream creation.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dnishimura/samza samza-1460-streamappender-create-logging-topic

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/423.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #423
    
----
commit 9c6437fc1b450c8b60ff6543a180d3b4ae929d4e
Author: Daniel Nishimura <dn...@...>
Date:   2018-02-17T00:27:12Z

    In StreamAppender, explicitly create the stream in the setup.

----


> StreamAppender does not explicitly create logging topic
> -------------------------------------------------------
>
>                 Key: SAMZA-1460
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1460
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Yi Pan (Data Infrastructure)
>            Priority: Major
>
> In StreamAppender, the logging topic is not explicitly created. We will be relying on Kafka cluster topic auto-creation feature to create the logging topic:
> {code}
>   protected void setupSystem() {
>     config = getConfig();
>     SystemFactory systemFactory = null;
>     Log4jSystemConfig log4jSystemConfig = new Log4jSystemConfig(config);
>     if (streamName == null) {
>       streamName = getStreamName(log4jSystemConfig.getJobName(), log4jSystemConfig.getJobId());
>     }
>     String systemName = log4jSystemConfig.getSystemName();
>     String systemFactoryName = log4jSystemConfig.getSystemFactory(systemName);
>     if (systemFactoryName != null) {
>       systemFactory = Util.<SystemFactory>getObj(systemFactoryName);
>     } else {
>       throw new SamzaException("Could not figure out \"" + systemName + "\" system factory for log4j StreamAppender to use");
>     }
>     setSerde(log4jSystemConfig, systemName, streamName);
>     systemProducer = systemFactory.getProducer(systemName, config, new MetricsRegistryMap());
>     systemStream = new SystemStream(systemName, streamName);
>     systemProducer.register(SOURCE);
>     systemProducer.start();
>     log.info(SOURCE + " has been registered in " + systemName + ". So all the logs will be sent to " + streamName
>         + " in " + systemName + ". Logs are partitioned by " + key);
>   }
> {code}
> It would be better to explicitly create the logging topic via SystemAdmin.createStream() to work w/ logging systems that do not support topic auto-creation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)