You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2023/05/12 05:48:40 UTC
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1138: [doc] Implove doc of cdc ingestion
JingsongLi commented on code in PR #1138:
URL: https://github.com/apache/incubator-paimon/pull/1138#discussion_r1191929167
##########
docs/content/how-to/cdc-ingestion.md:
##########
@@ -52,16 +58,53 @@ To use this feature through `flink run`, run the following shell command.
[--table-conf <paimon-table-sink-conf> [--table-conf <paimon-table-sink-conf> ...]]
```
-* `--warehouse` is the path to Paimon warehouse.
-* `--database` is the database name in Paimon catalog.
-* `--table` is the Paimon table name.
-* `--partition-keys` are the partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.
-* `--primary-keys` are the primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.
-* `--computed-column` are the definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are:
- * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.
-* `--mysql-conf` is the configuration for Flink CDC MySQL table sources. Each configuration should be specified in the format `key=value`. `hostname`, `username`, `password`, `database-name` and `table-name` are required configurations, others are optional. See its [document](https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mysql-cdc.html#connector-options) for a complete list of configurations.
-* `--catalog-conf` is the configuration for Paimon catalog. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of catalog configurations.
-* `--table-conf` is the configuration for Paimon table sink. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of table configurations.
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 3%">Configuration</th>
+ <th class="text-left" style="width: 10%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>`--warehouse`</code></td>
Review Comment:
remove all "`" in table
##########
docs/content/how-to/cdc-ingestion.md:
##########
@@ -30,6 +30,12 @@ Paimon supports synchronizing changes from different databases using change data
## MySQL
+### Prepare Flink SQL Connector MySQL CDC Bundled Jar
Review Comment:
Prepare CDC Bundled Jar
##########
docs/content/how-to/cdc-ingestion.md:
##########
@@ -52,16 +58,53 @@ To use this feature through `flink run`, run the following shell command.
[--table-conf <paimon-table-sink-conf> [--table-conf <paimon-table-sink-conf> ...]]
```
-* `--warehouse` is the path to Paimon warehouse.
-* `--database` is the database name in Paimon catalog.
-* `--table` is the Paimon table name.
-* `--partition-keys` are the partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.
-* `--primary-keys` are the primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.
-* `--computed-column` are the definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are:
- * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.
-* `--mysql-conf` is the configuration for Flink CDC MySQL table sources. Each configuration should be specified in the format `key=value`. `hostname`, `username`, `password`, `database-name` and `table-name` are required configurations, others are optional. See its [document](https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mysql-cdc.html#connector-options) for a complete list of configurations.
-* `--catalog-conf` is the configuration for Paimon catalog. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of catalog configurations.
-* `--table-conf` is the configuration for Paimon table sink. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of table configurations.
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 3%">Configuration</th>
+ <th class="text-left" style="width: 10%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>`--warehouse`</code></td>
+ <td><code>The path to Paimon warehouse.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--database`</code></td>
+ <td><code>The database name in Paimon catalog.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--table`</code></td>
+ <td><code>The Paimon table name.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--partition-keys`</code></td>
+ <td><code>The partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.</code></td>
Review Comment:
Use "dt,hh,mm" instead "`".
##########
docs/content/how-to/cdc-ingestion.md:
##########
@@ -52,16 +58,53 @@ To use this feature through `flink run`, run the following shell command.
[--table-conf <paimon-table-sink-conf> [--table-conf <paimon-table-sink-conf> ...]]
```
-* `--warehouse` is the path to Paimon warehouse.
-* `--database` is the database name in Paimon catalog.
-* `--table` is the Paimon table name.
-* `--partition-keys` are the partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.
-* `--primary-keys` are the primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.
-* `--computed-column` are the definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are:
- * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.
-* `--mysql-conf` is the configuration for Flink CDC MySQL table sources. Each configuration should be specified in the format `key=value`. `hostname`, `username`, `password`, `database-name` and `table-name` are required configurations, others are optional. See its [document](https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mysql-cdc.html#connector-options) for a complete list of configurations.
-* `--catalog-conf` is the configuration for Paimon catalog. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of catalog configurations.
-* `--table-conf` is the configuration for Paimon table sink. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of table configurations.
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 3%">Configuration</th>
+ <th class="text-left" style="width: 10%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>`--warehouse`</code></td>
+ <td><code>The path to Paimon warehouse.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--database`</code></td>
+ <td><code>The database name in Paimon catalog.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--table`</code></td>
+ <td><code>The Paimon table name.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--partition-keys`</code></td>
+ <td><code>The partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--primary-keys`</code></td>
+ <td><code>The primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--computed-column`</code></td>
+ <td><code>The definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are: * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.</code></td>
Review Comment:
Remove *, Use something like:
```
<td>Whether to double write to a changelog file. This changelog file keeps the details of data changes, it can be read directly during stream reads.<br /><br />Possible values:<ul><li>"none": No changelog file.</li><li>"input": Double write to a changelog file when flushing memory table, the changelog is from input.</li><li>"full-compaction": Generate changelog files with each full compaction.</li><li>"lookup": Generate changelog files through 'lookup' before committing the data writing.</li></ul></td>
```
##########
docs/content/how-to/cdc-ingestion.md:
##########
@@ -52,16 +58,53 @@ To use this feature through `flink run`, run the following shell command.
[--table-conf <paimon-table-sink-conf> [--table-conf <paimon-table-sink-conf> ...]]
```
-* `--warehouse` is the path to Paimon warehouse.
-* `--database` is the database name in Paimon catalog.
-* `--table` is the Paimon table name.
-* `--partition-keys` are the partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.
-* `--primary-keys` are the primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.
-* `--computed-column` are the definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are:
- * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.
-* `--mysql-conf` is the configuration for Flink CDC MySQL table sources. Each configuration should be specified in the format `key=value`. `hostname`, `username`, `password`, `database-name` and `table-name` are required configurations, others are optional. See its [document](https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mysql-cdc.html#connector-options) for a complete list of configurations.
-* `--catalog-conf` is the configuration for Paimon catalog. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of catalog configurations.
-* `--table-conf` is the configuration for Paimon table sink. Each configuration should be specified in the format `key=value`. See [here]({{< ref "maintenance/configurations" >}}) for a complete list of table configurations.
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 3%">Configuration</th>
+ <th class="text-left" style="width: 10%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>`--warehouse`</code></td>
+ <td><code>The path to Paimon warehouse.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--database`</code></td>
+ <td><code>The database name in Paimon catalog.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--table`</code></td>
+ <td><code>The Paimon table name.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--partition-keys`</code></td>
+ <td><code>The partition keys for Paimon table. If there are multiple partition keys, connect them with comma, for example `dt,hh,mm`.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--primary-keys`</code></td>
+ <td><code>The primary keys for Paimon table. If there are multiple primary keys, connect them with comma, for example `buyer_id,seller_id`.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--computed-column`</code></td>
+ <td><code>The definitions of computed columns. The argument field is from MySQL table field name. Supported expressions are: * year(date-column): Extract year from a DATE, DATETIME or TIMESTAMP. Output is an INT value represent the year.</code></td>
+ </tr>
+ <tr>
+ <td><code>`--mysql-conf`</code></td>
+ <td><code>The configuration for Flink CDC MySQL table sources. Each configuration should be specified in the format `key=value`. `hostname`, `username`, `password`, `database-name` and `table-name` are required configurations, others are optional. See its [document](https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mysql-cdc.html#connector-options) for a complete list of configurations.</code></td>
Review Comment:
[document] is not a link
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org