[jira] [Created] (SAMZA-1460) StreamAppender does not explicitly create logging topic

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

Yi Pan (Data Infrastructure) created SAMZA-1460:
---------------------------------------------------

             Summary: StreamAppender does not explicitly create logging topic
                 Key: SAMZA-1460
                 URL: https://issues.apache.org/jira/browse/SAMZA-1460
             Project: Samza
          Issue Type: Bug
            Reporter: Yi Pan (Data Infrastructure)


In StreamAppender, the logging topic is not explicitly created. We will be relying on Kafka cluster topic auto-creation feature to create the logging topic:
{code}
  protected void setupSystem() {
    config = getConfig();
    SystemFactory systemFactory = null;
    Log4jSystemConfig log4jSystemConfig = new Log4jSystemConfig(config);

    if (streamName == null) {
      streamName = getStreamName(log4jSystemConfig.getJobName(), log4jSystemConfig.getJobId());
    }

    String systemName = log4jSystemConfig.getSystemName();
    String systemFactoryName = log4jSystemConfig.getSystemFactory(systemName);
    if (systemFactoryName != null) {
      systemFactory = Util.<SystemFactory>getObj(systemFactoryName);
    } else {
      throw new SamzaException("Could not figure out \"" + systemName + "\" system factory for log4j StreamAppender to use");
    }

    setSerde(log4jSystemConfig, systemName, streamName);

    systemProducer = systemFactory.getProducer(systemName, config, new MetricsRegistryMap());
    systemStream = new SystemStream(systemName, streamName);
    systemProducer.register(SOURCE);
    systemProducer.start();

    log.info(SOURCE + " has been registered in " + systemName + ". So all the logs will be sent to " + streamName
        + " in " + systemName + ". Logs are partitioned by " + key);
  }
{code}

It would be better to explicitly create the logging topic via SystemAdmin.createStream() to work w/ logging systems that do not support topic auto-creation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)