You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/06/19 07:11:03 UTC

[GitHub] [flink] danny0405 opened a new pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

danny0405 opened a new pull request #12713:
URL: https://github.com/apache/flink/pull/12713


   ## What is the purpose of the change
   
   Add document for primary syntax of CREATE TABLE statement.
   
   
   ## Brief change log
   
     - Add modifications to create.md
   
   
   ## Verifying this change
   
   ![image](https://user-images.githubusercontent.com/7644508/85106589-07b1d480-b23f-11ea-9385-50b158be95f0.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646493174


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805",
       "triggerID" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "297f5423c8de573e9a067b75b75fa77eef0c3e1d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "297f5423c8de573e9a067b75b75fa77eef0c3e1d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805) 
   * 297f5423c8de573e9a067b75b75fa77eef0c3e1d UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] Huanli-Meng commented on a change in pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on a change in pull request #12713:
URL: https://github.com/apache/flink/pull/12713#discussion_r442794942



##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),

Review comment:
       ```suggestion
   Primary key constraint can be either declared along with a column definition (a column constraint) or as a single line(a table constraint),
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] Huanli-Meng commented on a change in pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on a change in pull request #12713:
URL: https://github.com/apache/flink/pull/12713#discussion_r442795340



##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),
+for both cases, it should only be declared as a singleton, if you define multiple primary key constraints at the same time, an exception would be thrown.

Review comment:
       ```suggestion
   For both cases, it should only be declared as a singleton. If you define multiple primary key constraints at the same time, an exception would be thrown.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] Huanli-Meng commented on a change in pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on a change in pull request #12713:
URL: https://github.com/apache/flink/pull/12713#discussion_r442795591



##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),
+for both cases, it should only be declared as a singleton, if you define multiple primary key constraints at the same time, an exception would be thrown.
+
+##### Validity Check
+
+SQL standard specifies that a constraint can either be `ENFORCED` or `NOT ENFORCED`. This controls if the constraint checks are performed on the incoming/outgoing data.
+Flink does not own the data therefore the only mode we want to support is the `NOT ENFORCED` mode.
+Its up to the user to ensure that the query enforces key integrity.

Review comment:
       ```suggestion
   It is up to the user to ensure that the query enforces key integrity.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] Huanli-Meng commented on a change in pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on a change in pull request #12713:
URL: https://github.com/apache/flink/pull/12713#discussion_r442794942



##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),

Review comment:
       ```suggestion
   Primary key constraint can be either declared along with a column definition (a column constraint) or as a single line(a table constraint).
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646485710


   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 (Fri Jun 19 07:30:33 UTC 2020)
   
   **Warnings:**
    * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-18254).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work.
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] wuchong closed pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
wuchong closed pull request #12713:
URL: https://github.com/apache/flink/pull/12713


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646493174


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805",
       "triggerID" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] wuchong commented on a change in pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
wuchong commented on a change in pull request #12713:
URL: https://github.com/apache/flink/pull/12713#discussion_r442809974



##########
File path: docs/dev/table/sql/create.md
##########
@@ -127,14 +127,21 @@ CREATE TABLE [catalog_name.][db_name.]table_name
   (
     { <column_definition> | <computed_column_definition> }[ , ...n]
     [ <watermark_definition> ]
+    [ <out_of_line_constraint> ][ , ...n]
   )
   [COMMENT table_comment]
   [PARTITIONED BY (partition_column_name1, partition_column_name2, ...)]
   WITH (key1=val1, key2=val2, ...)
   [ LIKE source_table [( <like_options> )] ]
    
 <column_definition>:
-  column_name column_type [COMMENT column_comment]
+  column_name column_type [ <in_line_constraint> ] [COMMENT column_comment]
+  
+<in_line_constraint>:

Review comment:
       `column_constraint`?

##########
File path: docs/dev/table/sql/create.md
##########
@@ -127,14 +127,21 @@ CREATE TABLE [catalog_name.][db_name.]table_name
   (
     { <column_definition> | <computed_column_definition> }[ , ...n]
     [ <watermark_definition> ]
+    [ <out_of_line_constraint> ][ , ...n]
   )
   [COMMENT table_comment]
   [PARTITIONED BY (partition_column_name1, partition_column_name2, ...)]
   WITH (key1=val1, key2=val2, ...)
   [ LIKE source_table [( <like_options> )] ]
    
 <column_definition>:
-  column_name column_type [COMMENT column_comment]
+  column_name column_type [ <in_line_constraint> ] [COMMENT column_comment]
+  
+<in_line_constraint>:
+  [CONSTRAINT constraint_name] PRIMARY KEY [[NOT] ENFORCED]

Review comment:
       Let's force to use `NOT ENFORCED` here, because this is the only mode we support. Otherwise, the syntax will confuse users. 

##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),
+for both cases, it should only be declared as a singleton, if you define multiple primary key constraints at the same time, an exception would be thrown.
+
+##### Validity Check
+
+SQL standard specifies that a constraint can either be `ENFORCED` or `NOT ENFORCED`. This controls if the constraint checks are performed on the incoming/outgoing data.
+Flink does not own the data therefore the only mode we want to support is the `NOT ENFORCED` mode.
+Its up to the user to ensure that the query enforces key integrity.
+
+Flink will assume correctness of the primary key by assuming that the columns nullability is aligned with the columns in primary key. Connectors should ensure those are aligned.
+
+**Notes:** In a CREATE TABLE statement, creating a primary key constraint will alter the columns nullability, that means, a column with primary key constraint is not nullable.
+
+##### Upsert Sink
+
+For an upsert table sink to receive update message by keys, the primary key constraint is a **must** for the updating keys.
+For example, for a table with definition:
+
+{% highlight sql %}
+CREATE TABLE Orders (
+    user BIGINT PRIMARY KEY NOT ENFORCED,
+    product STRING,
+    order_time TIMESTAMP(3)
+) WITH ( . . . );
+{% endhighlight %}
+
+you can send the updating records keyed by `user` from an aggregating query.

Review comment:
       I would like to remove this paragraph. It is depends on the connector implementation (not work for old connector, PK definition is optional for HBase). We have explained how to declare a proper sink in the specific connector page. So I think it's fine to remove this. 

##########
File path: docs/dev/table/sql/create.md
##########
@@ -202,6 +209,39 @@ CREATE TABLE Orders (
 ) WITH ( . . . );
 {% endhighlight %}
 
+**PRIMARY KEY**
+
+Primary key constraint is a hint for Flink to leverage for optimizations. It tells that a column or a set of columns of a table or a view are unique and they **do not** contain null.
+Neither of columns in a primary can be nullable. Primary key therefore uniquely identify a row in a table.
+
+Primary key constraint can be either declared along with a column definition(a column constraint) or as a single line(a table constraint),

Review comment:
       add space before `(`.

##########
File path: docs/dev/table/sql/create.md
##########
@@ -127,14 +127,21 @@ CREATE TABLE [catalog_name.][db_name.]table_name
   (
     { <column_definition> | <computed_column_definition> }[ , ...n]
     [ <watermark_definition> ]
+    [ <out_of_line_constraint> ][ , ...n]
   )
   [COMMENT table_comment]
   [PARTITIONED BY (partition_column_name1, partition_column_name2, ...)]
   WITH (key1=val1, key2=val2, ...)
   [ LIKE source_table [( <like_options> )] ]
    
 <column_definition>:
-  column_name column_type [COMMENT column_comment]
+  column_name column_type [ <in_line_constraint> ] [COMMENT column_comment]
+  
+<in_line_constraint>:
+  [CONSTRAINT constraint_name] PRIMARY KEY [[NOT] ENFORCED]
+
+<out_of_line_constraint>:

Review comment:
       `table_constraint`? 

##########
File path: docs/dev/table/sql/create.md
##########
@@ -127,14 +127,21 @@ CREATE TABLE [catalog_name.][db_name.]table_name
   (
     { <column_definition> | <computed_column_definition> }[ , ...n]
     [ <watermark_definition> ]
+    [ <out_of_line_constraint> ][ , ...n]
   )
   [COMMENT table_comment]
   [PARTITIONED BY (partition_column_name1, partition_column_name2, ...)]
   WITH (key1=val1, key2=val2, ...)
   [ LIKE source_table [( <like_options> )] ]
    
 <column_definition>:
-  column_name column_type [COMMENT column_comment]
+  column_name column_type [ <in_line_constraint> ] [COMMENT column_comment]
+  
+<in_line_constraint>:
+  [CONSTRAINT constraint_name] PRIMARY KEY [[NOT] ENFORCED]
+
+<out_of_line_constraint>:
+  [CONSTRAINT constraint_name] PRIMARY KEY (column_name, ...) [[NOT] ENFORCED]

Review comment:
       The same to above. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646493174


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805",
       "triggerID" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646493174


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12713: [FLINK-18254][doc] Add documentation for primary key syntax

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12713:
URL: https://github.com/apache/flink/pull/12713#issuecomment-646493174


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805",
       "triggerID" : "9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "297f5423c8de573e9a067b75b75fa77eef0c3e1d",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3855",
       "triggerID" : "297f5423c8de573e9a067b75b75fa77eef0c3e1d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9fee9ddbf7b29c35a9b2848a7c8adb1f111249b2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3805) 
   * 297f5423c8de573e9a067b75b75fa77eef0c3e1d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3855) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org