You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/03/09 14:34:00 UTC
[jira] [Work logged] (BEAM-14048) [CDAP] Design and implement ConfigWrapper for building CDAP PluginConfigs
[ https://issues.apache.org/jira/browse/BEAM-14048?focusedWorklogId=738794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-738794 ]
ASF GitHub Bot logged work on BEAM-14048:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Mar/22 14:33
Start Date: 09/Mar/22 14:33
Worklog Time Spent: 10m
Work Description: Amar3tto opened a new pull request #17051:
URL: https://github.com/apache/beam/pull/17051
CDAP plugins use their own configuration classes (ex. `SalesforceSourceConfig`, `SalesforceStreamingSourceConfig`) to set the required parameters to prepare and run pipeline in Source or Sink classes. All these config classes are extended from the base `io.cdap.cdap.api.plugin.PluginConfig` class.
As a first step in integration of **_CDAP plugins_** and _**Apache Beam**_, there should be implemented a mechanism to create an instance of the specific `PluginConfig` implementation.
For this purpose, suggested solution provides a `ConfigWrapper<T>` class. Apache Beam user creates an instance of this class by passing the `PluginConfig` specific class as a `T` parameter.
`ConfigWrapper<T>` object provides simple methods to instantiate the corresponding `T` object:
1. `fromJsonFile(File file)`
2. `withParams(Map<String, Object> params)`
3. `setParam(String paramName, Object value)`
4. `build()`
Apache Beam user could set required parameters for the CDAP plugin by passing the Map/JSON file or by manually setting each parameter.
After that, when the `build()` method was called, the `ConfigWrapper<T>` uses `InstantiationUtils`, which in turn uses Reflection API. `InstantiationUtils` class provides mechanisms for:
1. Creating an empty `PluginConfig` object bypassing the limitations of private constructors or the absence of no-args constructor.
2. Going over all `Field`s with the `cdap.Name` annotation and setting the appropriate parameter values for them.
As a result `ConfigWrapper<T>` returns `PluginConfig` object which can be further used when integrating with the corresponding plugin.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 738794)
Remaining Estimate: 0h
Time Spent: 10m
> [CDAP] Design and implement ConfigWrapper for building CDAP PluginConfigs
> -------------------------------------------------------------------------
>
> Key: BEAM-14048
> URL: https://issues.apache.org/jira/browse/BEAM-14048
> Project: Beam
> Issue Type: Task
> Components: io-java-cdap
> Reporter: Elizaveta Lomteva
> Assignee: Vitaly Terentyev
> Priority: P2
> Labels: cdap-io-sprint-1
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Context:
> CDAP plugins use their configuration classes (ex. {{{}SalesforceSinkConfig{}}}) to set the required parameters to prepare and run a pipeline in Source or Sink classes (ex. {{{}SalesforceBatchSink{}}}). The config instance is accepted as a Source/Sink constructor argument.
> The configuration classes are usually placed in the {{{}cdap/plugin/\{<plugin name>}/plugin/\{sink|source}/{batch|streaming{}}}} folder (ex. [Salesforce repo|https://github.com/data-integrations/salesforce/tree/d0d83fd82bbdbbb7ef07168f3b1698eebdc90c3e/src/main/java/io/cdap/plugin/salesforce/plugin]).
> h3. Task Description:
> Required to design wrapper class(es) for the CDAP plugin config class so that the CDAP plugin config instance can be created in the Apache Beam connector based on user input and used when creating CDAP Source or Sink.
> h3. Acceptance criteria:
> Design and source code of CDAP Config wrapper that will
> # accept custom parameters as constructor arguments (JSON/string or something else)
> # return an instance of plugins configuration class (ex. {{{}SalesforceSinkConfig{}}})
--
This message was sent by Atlassian Jira
(v8.20.1#820001)