[jira] [Resolved] (SAMZA-2071) Skip StartpointManager instantiation for PassThroughCoordinator.

["Shanthoosh Venkataraman (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shanthoosh Venkataraman resolved SAMZA-2071.
--------------------------------------------
    Resolution: Fixed

Resolved by the pull-request: https://github.com/apache/samza/pull/908 

> Skip StartpointManager instantiation for PassThroughCoordinator.
> ----------------------------------------------------------------
>
>                 Key: SAMZA-2071
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2071
>             Project: Samza
>          Issue Type: Improvement
>    Affects Versions: 1.1
>            Reporter: Shanthoosh Venkataraman
>            Assignee: Shanthoosh Venkataraman
>            Priority: Major
>
> Samza support PassThroughCoordinator where the coordination between processors and checkpoints are expected to be managed by the external service. Coordinator stream is used in samza to persist the metadata related to the coordiantion and a execution. JobCoordinator is not applicable for the PassthroughCoordinator case in samza.
> SAMZA-1985 added a startpoint manager implementation and by default used coordinator stream as the startpoint metadata store. After this, running PassthroughCoordinator locally require coordinator stream and the associated coordinator configurations. If it is not defined, it spams the log with the following exception.
> {noformat}
> org.apache.samza.config.ConfigException: Missing job.coordinator.system configuration. Cannot proceed with job execution. at org.apache.samza.config.JobConfig.getCoordinatorSystemName(JobConfig.scala:145)
> at org.apache.samza.util.CoordinatorStreamUtil$.getCoordinatorSystemStream(CoordinatorStreamUtil.scala:56)
> at org.apache.samza.util.CoordinatorStreamUtil.getCoordinatorSystemStream(CoordinatorStreamUtil.scala)
> at org.apache.samza.coordinator.metadatastore.CoordinatorStreamStore.<init>(CoordinatorStreamStore.java:81)
> at org.apache.samza.coordinator.metadatastore.CoordinatorStreamMetadataStoreFactory.getMetadataStore(CoordinatorStreamMetadataStoreFactory.java:34)
> at org.apache.samza.startpoint.StartpointManager.<init>(StartpointManager.java:77)
> at org.apache.samza.container.SamzaContainer$.apply(SamzaContainer.scala:428)
> at org.apache.samza.container.SamzaContainer.apply(SamzaContainer.scala)
> at org.apache.samza.processor.StreamProcessor.createSamzaContainer(StreamProcessor.java:333)
> at org.apache.samza.processor.StreamProcessor$1.onNewJobModel(StreamProcessor.java:405)
> at org.apache.samza.standalone.PassthroughJobCoordinator.start(PassthroughJobCoordinator.java:103)
> at org.apache.samza.processor.StreamProcessor.start(StreamProcessor.java:266)
> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
> at org.apache.samza.runtime.LocalApplicationRunner.run(LocalApplicationRunner.java:130)
> at org.apache.samza.runtime.ApplicationRunner.run(ApplicationRunner.java:40)
> at org.apache.beam.runners.samza.SamzaRunner.run(SamzaRunner.java:110)
> at org.apache.beam.runners.samza.TestSamzaRunner.run(TestSamzaRunner.java:75)
> at org.apache.beam.runners.samza.runtime.SamzaStoreStateInternalsTest.testMapStateIterator(SamzaStoreStateInternalsTest.java:132)
> {noformat}
> This hampers local mode development and testing of samza-beam jobs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)