You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/31 09:36:32 UTC
[GitHub] [pulsar] freeznet opened a new pull request, #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
freeznet opened a new pull request, #16891:
URL: https://github.com/apache/pulsar/pull/16891
Fixes https://github.com/apache/pulsar/issues/16674
### Motivation
Add doc for https://github.com/apache/pulsar/pull/16675
### Modifications
*Describe the modifications you've done.*
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [ ] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#issuecomment-1256835916
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
freeznet commented on PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#issuecomment-1208819550
> @freeznet thanks for adding the docs. I left several minor suggestions. Another concern is the historical doc versions you changed is 2.9.3 and 2.10.1 while the code PR seems to serve 2.9.4 and 2.10.2?
@momo-jun you are right, could you please tell me how can I add the doc to 2.9.4 and 2.10.2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun merged pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
momo-jun merged PR #16891:
URL: https://github.com/apache/pulsar/pull/16891
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#issuecomment-1208830558
> > @freeznet thanks for adding the docs. I left several minor suggestions. Another concern is the historical doc versions you changed is 2.9.3 and 2.10.1 while the code PR seems to serve 2.9.4 and 2.10.2?
>
> @momo-jun you are right, could you please tell me how can I add the doc to 2.9.4 and 2.10.2?
In the current practice of Pulsar doc release and maintenance, the docs for the upcoming 2.9.4 and 2.10.2 releases aren't in place until the release is ready to announce. Since [PIP-190](https://github.com/apache/pulsar/issues/16637) has been adopted, I'm generating 2.9.x and 2.10.x docs in #16925. You can leave them as they are in this PR and I will sync your changes there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#discussion_r934483171
##########
site2/docs/io-debezium-source.md:
##########
@@ -27,7 +27,9 @@ The configuration of Debezium source connector has the following properties.
| `database.history.pulsar.topic` | true | null | The name of the database history topic where the connector writes and recovers DDL statements. <br /><br />**Note: this topic is for internal use only and should not be used by consumers.** |
| `database.history.pulsar.service.url` | true | null | Pulsar cluster service URL for history topic. |
| `offset.storage.topic` | true | null | Record the last committed offsets that the connector successfully completes. |
-| `json-with-envelope` | false | false | Present the message only consist of payload.
+| `json-with-envelope` | false | false | Present the message only consist of payload. |
Review Comment:
```suggestion
| `json-with-envelope` | false | false | Present the message that consists of payload only. |
```
Is this what it means? The original one has a grammar issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#discussion_r934480070
##########
site2/docs/io-debezium-source.md:
##########
@@ -54,7 +56,32 @@ Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message consist
| `mongodb.password` | true | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |
+### Customize the Reader config for the metadata topics
+The Debezium Connector exposes `database.history.pulsar.reader.config` and `offset.storage.reader.config` to configure the reader of database schema history topic and the kafka connector offsets topic. For example, it can be used to configure the subscription name and other reader configurations. The available configuration can be found at [ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigurationData.java).
Review Comment:
```suggestion
The Debezium Connector exposes `database.history.pulsar.reader.config` and `offset.storage.reader.config` to configure the reader of database schema history topic and the Kafka connector offsets topic. For example, it can be used to configure the subscription name and other reader configurations. You can find the available configurations at [ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigurationData.java).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #16891: [improve][doc] Add doc of reader config to `pulsar-io-debezium`
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #16891:
URL: https://github.com/apache/pulsar/pull/16891#discussion_r934478862
##########
site2/docs/io-debezium-source.md:
##########
@@ -54,7 +56,32 @@ Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and the message consist
| `mongodb.password` | true | null | Password to be used when connecting to MongoDB. This is required only when MongoDB is configured to use authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that attempts to use a separate task for each replica set. |
+### Customize the Reader config for the metadata topics
+The Debezium Connector exposes `database.history.pulsar.reader.config` and `offset.storage.reader.config` to configure the reader of database schema history topic and the kafka connector offsets topic. For example, it can be used to configure the subscription name and other reader configurations. The available configuration can be found at [ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigurationData.java).
+
+For example, to configure the subscription name for both Reader, you can add the following configuration:
Review Comment:
```suggestion
For example, to configure the subscription name for both Readers, you can add the following configuration:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org