You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/31 11:53:05 UTC

[GitHub] [hudi] JerryYue-M opened a new pull request, #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

JerryYue-M opened a new pull request, #7099:
URL: https://github.com/apache/hudi/pull/7099

   …urrent progress of writing data
   
   ### Change Logs
   
   _Describe context and summary for this change. Highlight if any code was copied._
   
   ### Impact
   
   _Describe any public API or user-facing feature change or any performance impact._
   
   ### Risk level (write none, low medium or high below)
   
   _If medium or high, explain what verification was done to mitigate the risks._
   
   ### Documentation Update
   
   _Describe any necessary documentation update if there is any new feature, config, or user-facing change_
   
   - _The config description must be updated if new configs are added or the default value of the configs are changed_
   - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
     ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
     changes to the website._
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1334005332

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333459704

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1338788359

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13472",
       "triggerID" : "1338631024",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13472) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333670877

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333659608

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1332197227

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 427bc0a5a8753dd4db51bc4a9201b62876a800d9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004) 
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1303836185

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9c8daede5152dec00545ce5c46772f568135d45a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1313699285

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9c8daede5152dec00545ce5c46772f568135d45a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788) 
   * 427bc0a5a8753dd4db51bc4a9201b62876a800d9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1332206793

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 427bc0a5a8753dd4db51bc4a9201b62876a800d9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004) 
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1297026932

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 225cef688aa785f6506c965abdf14fcd222cf9a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1303439103

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 225cef688aa785f6506c965abdf14fcd222cf9a9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691) 
   * 9c8daede5152dec00545ce5c46772f568135d45a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on a diff in pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on code in PR #7099:
URL: https://github.com/apache/hudi/pull/7099#discussion_r1010032811


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/common/AbstractWriteFunction.java:
##########
@@ -45,4 +92,46 @@
    * @param event The event
    */
   public abstract void handleOperatorEvent(OperatorEvent event);
+
+  /**
+   *  Extract TimeStamp from input value with specified event time field.
+   * @param value The input value
+   * @return the new timestamp for current.
+   */
+  public  long extractTimestamp(I value) {
+    if (value instanceof HoodieAvroRecord) {
+      return extractTimestamp((HoodieAvroRecord) value);
+    }
+    return extractTimestamp((RowData) value);
+  }

Review Comment:
   The reason why put it here is BulkInsertWriteFunction only extends BulkInsertWriteFunction instead of AbstractStreamWriteFunction, if moving this code to AbstractStreamWriteFunction will make a specified way to deal with BulkInsertWriteFunction



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] codope commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
codope commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1330671373

   @JerryYue-M Gentle ping to fix the checkstyle issues. This is very close to merging.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333074754

   @codope  @danny0405  @yuzhaojing @XuQianJin-Stars
   all the discussion and check style issues had been fixed and CI passed already
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1332443080

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1313973982

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 427bc0a5a8753dd4db51bc4a9201b62876a800d9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] yuzhaojing commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
yuzhaojing commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1314845112

   Please fix checkstyle to pass ci.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1297534340

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 225cef688aa785f6506c965abdf14fcd222cf9a9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] danny0405 commented on a diff in pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #7099:
URL: https://github.com/apache/hudi/pull/7099#discussion_r1010030040


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/common/AbstractWriteFunction.java:
##########
@@ -45,4 +92,46 @@
    * @param event The event
    */
   public abstract void handleOperatorEvent(OperatorEvent event);
+
+  /**
+   *  Extract TimeStamp from input value with specified event time field.
+   * @param value The input value
+   * @return the new timestamp for current.
+   */
+  public  long extractTimestamp(I value) {
+    if (value instanceof HoodieAvroRecord) {
+      return extractTimestamp((HoodieAvroRecord) value);
+    }
+    return extractTimestamp((RowData) value);
+  }

Review Comment:
   Can we move these logic into `AbstractStreamWriteFunction.java`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1313684175

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9c8daede5152dec00545ce5c46772f568135d45a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788) 
   * 427bc0a5a8753dd4db51bc4a9201b62876a800d9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1316230494

   @yuzhaojing  thanks for the review, I will fix the issue you mentioned


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333284437

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1297021246

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 225cef688aa785f6506c965abdf14fcd222cf9a9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] XuQianJin-Stars commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1313735056

   > @XuQianJin-Stars what do you think about this way to describe the progress of the streaming writer
   
   +1, This is also the implementation method used in our internal production.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333546146

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on a diff in pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on code in PR #7099:
URL: https://github.com/apache/hudi/pull/7099#discussion_r1035961377


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -503,10 +509,29 @@ private boolean commitInstant(String instant, long checkpointId) {
       sendCommitAckEvents(checkpointId);
       return false;
     }
+    setMinEventTime();
     doCommit(instant, writeResults);
+    resetMinEventTime();
     return true;
   }
 
+  public void setMinEventTime() {
+    if (commitEventTimeEnable) {
+      LOG.info("receive event time for current commit: {} ", Arrays.stream(eventBuffer).collect(Collectors.toMap(WriteMetadataEvent::getTaskID, WriteMetadataEvent::getMaxEventTime)));
+      this.minEventTime = Arrays.stream(eventBuffer)
+        .filter(Objects::nonNull)
+        .map(WriteMetadataEvent::getMaxEventTime)
+        .filter(maxEventTime -> maxEventTime > 0)
+        .min(Comparator.naturalOrder())
+        .map(aLong -> Math.min(aLong, this.minEventTime)).orElse(Long.MAX_VALUE);
+      LOG.info("[setMinEventTime] minEventTime: {} ", this.minEventTime);
+    }
+  }
+
+  public void resetMinEventTime() {

Review Comment:
   no, we need reset min event time for each commit



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on code in PR #7099:
URL: https://github.com/apache/hudi/pull/7099#discussion_r1034889485


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -503,10 +509,29 @@ private boolean commitInstant(String instant, long checkpointId) {
       sendCommitAckEvents(checkpointId);
       return false;
     }
+    setMinEventTime();
     doCommit(instant, writeResults);
+    resetMinEventTime();
     return true;
   }
 
+  public void setMinEventTime() {
+    if (commitEventTimeEnable) {
+      LOG.info("receive event time for current commit: {} ", Arrays.stream(eventBuffer).collect(Collectors.toMap(WriteMetadataEvent::getTaskID, WriteMetadataEvent::getMaxEventTime)));
+      this.minEventTime = Arrays.stream(eventBuffer)
+        .filter(Objects::nonNull)
+        .map(WriteMetadataEvent::getMaxEventTime)
+        .filter(maxEventTime -> maxEventTime > 0)
+        .min(Comparator.naturalOrder())
+        .map(aLong -> Math.min(aLong, this.minEventTime)).orElse(Long.MAX_VALUE);
+      LOG.info("[setMinEventTime] minEventTime: {} ", this.minEventTime);
+    }
+  }
+
+  public void resetMinEventTime() {

Review Comment:
   can remove this method?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] XuQianJin-Stars commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1338631024

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333560004

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333612941

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1333284342

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1338650852

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13004",
       "triggerID" : "427bc0a5a8753dd4db51bc4a9201b62876a800d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352",
       "triggerID" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372",
       "triggerID" : "1333284342",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333546146",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376",
       "triggerID" : "1333612941",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "f29644c28247895b62d42f0923bfe0bd4560dfa4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13472",
       "triggerID" : "1338631024",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * f29644c28247895b62d42f0923bfe0bd4560dfa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13352) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13372) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13379) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13376) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13472) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1338700017

   > Can we fetch the event time through the payload clazz event time field we the `HoodieRecordPayload#getInsertValue` is invoked ? And we can finalize the max timestamp in the coordinator when commit the metadata.
   @danny0405  thanks for your suggestion, it's  import for me
   the reason why i think we need implement in this way is :
   1. I think this is a lightweight and common behavior, no matter what type of payload it is used. if we implement this base on payload, it may be limited by payload class between flink and spark/presto eg. 
   2. for later we may need to add some metric  for the flink write function eg: processing latency metric , and also need to extract the timestamp from the record
   3. implement base payload may change some code in common module such as modify the HoodieAppendHandler or HoodieMergeHandler 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1303451290

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691",
       "triggerID" : "225cef688aa785f6506c965abdf14fcd222cf9a9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788",
       "triggerID" : "9c8daede5152dec00545ce5c46772f568135d45a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 225cef688aa785f6506c965abdf14fcd222cf9a9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12691) 
   * 9c8daede5152dec00545ce5c46772f568135d45a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12788) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] JerryYue-M commented on pull request #7099: [HUDI-5095] Flink: Stores a special watermark(flag) to identify the c…

Posted by GitBox <gi...@apache.org>.
JerryYue-M commented on PR #7099:
URL: https://github.com/apache/hudi/pull/7099#issuecomment-1313616275

   @XuQianJin-Stars  what do you think about this way to describe the progress of the streaming writer


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org