You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by GitBox <gi...@apache.org> on 2020/02/21 00:28:05 UTC
[GitHub] [samza] alnzng opened a new pull request #1286: SAMZA-2465: Task
inputs information lost when enabled `RegExTopicGenerator ` or specified
'task.inputs' explicitly in non-legacy application
alnzng opened a new pull request #1286: SAMZA-2465: Task inputs information lost when enabled `RegExTopicGenerator ` or specified 'task.inputs' explicitly in non-legacy application
URL: https://github.com/apache/samza/pull/1286
# Symptom
If the user’s non-legacy application enabled `RegExTopicGenerator` or specified `task.inputs` explicitly and specified the input streams in its application descriptor, the expectation from the user side should be that the application can consume messages from specified input streams and Kafka topics that matched specified regex patterns.
However, in current logic seems the input information from the application descriptor will be overrided by the information from `RegExTopicGenerator` or `task.inputs` in the config file, which means the user’s application can only consume from matched Kafka topics or the inputs specified in `task.inputs`.
# Cause
The generated task inputs from the application descriptor are overrided by [JobNodeConfigurationGenerator.mergeConfig](https://github.com/alnzng/samza/blob/42df9090549c7a910d1ed16e1a787c36ae7b39d2/samza-core/src/main/java/org/apache/samza/execution/JobNodeConfigurationGenerator.java#L77) function.
# Changes
Merge generated inputs and original inputs before doing `JobNodeConfigurationGenerator.mergeConfig` function call.
# Tests
- [ ] All unit tests and integration tests are passed
# API Changes
None
# Upgrade Instructions
None
# Usage Instructions
Noe
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services