You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/02/10 22:40:38 UTC

[GitHub] [hudi] alexeykudinkin opened a new pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

alexeykudinkin opened a new pull request #4789:
URL: https://github.com/apache/hudi/pull/4789


   ## *Tips*
   - *Thank you very much for contributing to Apache Hudi.*
   - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
   
   ## What is the purpose of the pull request
   
   Bootstrapping initial support for Metadata Table in Spark Datasource
   
   ## Brief change log
   
    - Consolidated Avro/Row conversion utilities to center around Spark's `AvroDeserializer` ; removed duplication
    - Bootstrapped `HoodieBaseRelation`
    - Updated `HoodieMergeOnReadRDD` to be able to handle Metadata Table
    - Modified MOR relations to be able to read different Base File formats (Parquet, HFile)
    - Added tests
   
   ## Verify this pull request
   
   
   This pull request is already covered by existing tests, such as *(please describe tests)*.
   This change added tests and can be verified as follows:
   
     - TestMetadataTable
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039576906


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1037001375


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036656641


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036686233


   @bvaradar @vinothchandar : was there any specific reason why we were maintaining the row to avro and avro to row conversion code within Hudi instead of relying on the one that comes w/ spark-avro ? In this patch we are totally removing these and gonna rely on spark-avro classes to ser and deser. Can you folks review the patch.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1037001375


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036982866


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042083359


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042349803


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r808594567



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +130,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)
+          }
+
+        // Since caller might request to get records in a different ("evolved") schema, we will be rewriting from
+        // existing Writer's schema into Reader's (avro) schema
+        val writerAvroSchema = new Schema.Parser().parse(writerAvroSchemaStr)
+        val convert = AvroConversionUtils.createInternalRowToAvroConverter(writerSchema, writerAvroSchema, nullable = nullable)

Review comment:
       We can't -- schemas have to match

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -227,17 +223,20 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
                                          baseFileIterator: Iterator[InternalRow],
                                          config: Configuration): Iterator[InternalRow] =
     new Iterator[InternalRow] with Closeable {
-      private val tableAvroSchema = new Schema.Parser().parse(tableState.tableAvroSchema)
-      private val requiredAvroSchema = new Schema.Parser().parse(tableState.requiredAvroSchema)
-      private val serializer = HoodieAvroSerializer(tableState.tableStructSchema, tableAvroSchema, false)
-      private val requiredDeserializer = HoodieAvroDeserializer(requiredAvroSchema, tableState.requiredStructSchema)
+      private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+      private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+      private val requiredFieldPosition =
+        requiredSchema.structTypeSchema
+          .map(f => tableAvroSchema.getField(f.name).pos()).toList
+      private val requiredSerializer = HoodieAvroSerializer(requiredSchema.structTypeSchema, requiredAvroSchema, nullable = false)
+      private val requiredDeserializer = HoodieAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
       private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
-      private val unsafeProjection = UnsafeProjection.create(tableState.requiredStructSchema)
+      private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
       private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
       private val logRecords = logScanner.getRecords
       private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
       private val keyToSkip = mutable.Set.empty[String]
-      private val recordKeyPosition = if (recordKeyFieldOpt.isEmpty) HOODIE_RECORD_KEY_COL_POS else tableState.tableStructSchema.fieldIndex(recordKeyFieldOpt.get)
+      private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)

Review comment:
       Correct

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +319,48 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(
+        new HoodieLocalEngineContext(config), metadataConfig,
+        dataTableBasePath,
+        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+
+      // NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
+      //       of indirection among MT partitions)
+      val relativePartitionPath = FSUtils.getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+      metadataTable.getLogRecordScanner(split.logFiles.get.asJava, relativePartitionPath).getLeft
+    } else {
+      HoodieMergedLogRecordScanner.newBuilder()
+        .withFileSystem(fs)
+        .withBasePath(split.tablePath)
+        .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)
+        .withReaderSchema(logSchema)
+        .withLatestInstantTime(split.latestCommit)
+        .withReadBlocksLazily(
+          Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
+            HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
+            .getOrElse(false))
+        .withReverseReader(false)
+        .withBufferSize(
+          config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
+            HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
+        .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
+        .withSpillableMapBasePath(
+          config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
+            HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+        .build()
+    }
+  }
+
+  private def getPartitionPath(split: HoodieMergeOnReadFileSplit): Path = {

Review comment:
       This method is not bound to Metadata Table in any way, it just fetches partition path

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -125,41 +114,39 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
       val (requiredAvroSchema, requiredStructSchema) =
         HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
 
-      val hoodieTableState = HoodieMergeOnReadTableState(
-        tableStructSchema,
-        requiredStructSchema,
-        tableAvroSchema.toString,
-        requiredAvroSchema.toString,
-        fileIndex,
-        preCombineField,
-        Option.empty
-      )
-      val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val partitionSchema = StructType(Nil)
+      val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+      val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+      val fullSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = tableSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
-
-      val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val requiredSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = requiredSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
 
+      val hoodieTableState = HoodieMergeOnReadTableState(fileIndex, HoodieRecord.RECORD_KEY_METADATA_FIELD, preCombineFieldOpt)

Review comment:
       Good catch!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043707104


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043868633


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045361269


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045478976


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045395366


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045478976


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

yihua commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814429501



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       Sg.  Based on the info provided, in this case, looks like we can depend on `InternalRow` for now, for performance reasons.  For Row writer path, I think if we can leverage Spark's internal optimization with `InternalRow` that'd be good, without using `InternalRow` directly in Hudi code.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

xushiyan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814524456



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()

Review comment:
       hey @alexeykudinkin i wasn't being clear. I meant to say why we not taking any user-provided configs and pass them to `metadataConfig`; here it's just using the defaults in `HoodieMetadataConfig`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049397046


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252",
       "triggerID" : "1049396928",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

yihua commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813265854



##########
File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/BuiltinKeyGenerator.java
##########
@@ -69,7 +67,7 @@ protected BuiltinKeyGenerator(TypedProperties config) {
   @PublicAPIMethod(maturity = ApiMaturityLevel.EVOLVING)
   public String getRecordKey(Row row) {
     if (null == converterFn) {
-      converterFn = AvroConversionHelper.createConverterToAvro(row.schema(), STRUCT_NAME, NAMESPACE);
+      converterFn = AvroConversionUtils.createConverterToAvro(row.schema(), STRUCT_NAME, NAMESPACE);
     }
     GenericRecord genericRecord = (GenericRecord) converterFn.apply(row);
     return getKey(genericRecord).getRecordKey();

Review comment:
       nit: no need for type cast and directly do `return getKey(converterFn.apply(row)).getRecordKey();`?

##########
File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/BuiltinKeyGenerator.java
##########
@@ -84,7 +82,7 @@ public String getRecordKey(Row row) {
   @PublicAPIMethod(maturity = ApiMaturityLevel.EVOLVING)
   public String getPartitionPath(Row row) {
     if (null == converterFn) {
-      converterFn = AvroConversionHelper.createConverterToAvro(row.schema(), STRUCT_NAME, NAMESPACE);
+      converterFn = AvroConversionUtils.createConverterToAvro(row.schema(), STRUCT_NAME, NAMESPACE);
     }
     GenericRecord genericRecord = (GenericRecord) converterFn.apply(row);
     return getKey(genericRecord).getPartitionPath();

Review comment:
       same here

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -323,32 +322,60 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    val partitionPath: String = if (split.logPaths.isEmpty || split.logPaths.get.asJava.isEmpty) {
-      null
+    val logFiles = split.logFiles.get
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {

Review comment:
       I need some clarification here.  So why is metadata table treated differently here?  It's basically another Hudi table.

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -41,15 +54,105 @@ abstract class HoodieBaseRelation(
 
   protected val sparkSession: SparkSession = sqlContext.sparkSession
 
-  protected val tableAvroSchema: Schema = {
+  protected lazy val tableAvroSchema: Schema = {
     val schemaUtil = new TableSchemaResolver(metaClient)
-    Try (schemaUtil.getTableAvroSchema).getOrElse(SchemaConverters.toAvroType(userSchema.get))
+    Try(schemaUtil.getTableAvroSchema).getOrElse(
+      // If there is no commit in the table, we can't get the schema
+      // t/h [[TableSchemaResolver]], fallback to provided the [[userSchema]] instead.

Review comment:
       nit: typo  "fallback to provided the" -> "fallback to the provided"

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       @nsivabalan @alexeykudinkin @xushiyan This is more like an internal conversion.  And looks like it is only used by read path?  So does this expose `InternalRow` in some way to public APIs and data written, i.e., if `InternalRow` changes in Spark, could it break Hudi read/write flows?
   
   End-to-end wise for write side, I think eventually we want to avoid Row/Avro conversion for Spark Row Writer path.  On the write side, I don't think it's a good idea to depend on `InternalRow` which is Spark SQL's internal abstraction, i.e., on the write side `Row`s should be passed along e2e when row abstraction is required.
   
   cc @vinothchandar @bvaradar 

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -18,20 +18,105 @@
 
 package org.apache.hudi
 
-import org.apache.avro.Schema
-import org.apache.avro.JsonProperties
+import org.apache.avro.Schema.Type
 import org.apache.avro.generic.{GenericRecord, GenericRecordBuilder, IndexedRecord}
+import org.apache.avro.{AvroRuntimeException, JsonProperties, Schema}
+import org.apache.hudi.HoodieSparkUtils.sparkAdapter
 import org.apache.hudi.avro.HoodieAvroUtils
 import org.apache.spark.rdd.RDD
 import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.types.StructType
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.encoders.RowEncoder
+import org.apache.spark.sql.types.{DataType, StructType}
 import org.apache.spark.sql.{Dataset, Row, SparkSession}
 
-import scala.collection.JavaConverters._
 import scala.collection.JavaConversions._
+import scala.collection.JavaConverters._
 
 object AvroConversionUtils {
 
+  /**
+   * Check the nullability of the input Avro type and resolve it when it is nullable. The first
+   * return value is a [[Boolean]] indicating if the input Avro type is nullable. The second
+   * return value is either provided Avro type if it's not nullable, or its resolved non-nullable part
+   * in case it is
+   */
+  def resolveAvroTypeNullability(avroType: Schema): (Boolean, Schema) = {
+    if (avroType.getType == Type.UNION) {
+      val fields = avroType.getTypes.asScala
+      val actualType = fields.filter(_.getType != Type.NULL)
+      if (fields.length != 2 || actualType.length != 1) {
+        throw new AvroRuntimeException(
+          s"Unsupported Avro UNION type $avroType: Only UNION of a null type and a non-null " +
+            "type is supported")
+      }
+      (true, actualType.head)
+    } else {
+      (false, avroType)
+    }
+  }
+
+  /**
+   * Creates converter to transform Avro payload into Spark's Catalyst one
+   *
+   * @param rootAvroType Avro [[Schema]] to be transformed from
+   * @param rootCatalystType Catalyst [[StructType]] to be transformed into
+   * @return converter accepting Avro payload and transforming it into a Catalyst one (in the form of [[InternalRow]])
+   */
+  def createAvroToInternalRowConverter(rootAvroType: Schema, rootCatalystType: StructType): GenericRecord => Option[InternalRow] =
+    record => sparkAdapter.createAvroDeserializer(rootAvroType, rootCatalystType)
+      .deserialize(record)
+      .map(_.asInstanceOf[InternalRow])
+
+  /**
+   * Creates converter to transform Catalyst payload into Avro one
+   *
+   * @param rootCatalystType Catalyst [[StructType]] to be transformed from
+   * @param rootAvroType Avro [[Schema]] to be transformed into
+   * @param nullable whether Avro record is nullable
+   * @return converter accepting Catalyst payload (in the form of [[InternalRow]]) and transforming it into an Avro one
+   */
+  def createInternalRowToAvroConverter(rootCatalystType: StructType, rootAvroType: Schema, nullable: Boolean): InternalRow => GenericRecord = {
+    row => sparkAdapter.createAvroSerializer(rootCatalystType, rootAvroType, nullable)
+      .serialize(row)
+      .asInstanceOf[GenericRecord]
+  }
+
+  /**
+   * @deprecated please use [[AvroConversionUtils.createInternalRowToAvroConverter]]

Review comment:
       wrong ref to the new API?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042302553


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042478060


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r808593820



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       These are different APIs: one returns `Row` while other returns (Spark's internal) `InternalRow` allowing to avoid penalty of deserialization from one into another.
   
   The plan is to gradually weed out (where possible) replace `Row` converter w/ `InternalRow` one




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042514254


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043868633


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042562745


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035606910


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036982108


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035751118


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041009539


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049255120


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049285849


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035738461


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036700495


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035751118


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035611194


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039580411


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039682913


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042561271


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043863308


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1044052916


   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042349803


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042601092


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812949213



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()

Review comment:
       good catch. may be we need to instantiate using props. But this might need fixes in other read code paths as well. For eg, HoodieInputFormatUtils.java
   

##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -526,6 +526,7 @@ private boolean hasOperationField() {
       Schema tableAvroSchema = getTableAvroSchemaFromDataFile();
       return tableAvroSchema.getField(HoodieRecord.OPERATION_METADATA_FIELD) != null;
     } catch (Exception e) {
+      LOG.info(String.format("Failed to read operation field from avro schema (%s)", e.getMessage()));

Review comment:
       can we make this error log level instead of info? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] vinothchandar commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

vinothchandar commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813615713



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializerTrait.scala
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+/**
+ * Deserializes Avro payload into Catalyst object
+ *
+ * NOTE: This is low-level component operating on Spark internal data-types (comprising [[InternalRow]]).
+ *       If you're looking to convert Avro into "deserialized" [[Row]] (comprised of Java native types),
+ *       please check [[AvroConversionUtils]]
+ */
+trait HoodieAvroDeserializerTrait {
+  final def deserialize(data: Any): Option[Any] =
+    doDeserialize(data) match {
+      case opt: Option[_] => opt    // As of Spark 3.1, this will return data wrapped with Option, so we fetch the data

Review comment:
       if there is code specific to spark versions, can you move them into the adapters? how does this work with spark 2.x? 

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionHelper.scala
##########
@@ -1,380 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *      http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.hudi
-
-import java.nio.ByteBuffer
-import java.sql.{Date, Timestamp}
-import java.time.Instant
-
-import org.apache.avro.Conversions.DecimalConversion
-import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
-import org.apache.avro.Schema.Type._
-import org.apache.avro.generic.GenericData.{Fixed, Record}
-import org.apache.avro.generic.{GenericData, GenericFixed, GenericRecord}
-import org.apache.avro.{LogicalTypes, Schema}
-
-import org.apache.spark.sql.Row
-import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.catalyst.expressions.GenericRow
-import org.apache.spark.sql.catalyst.util.DateTimeUtils
-import org.apache.spark.sql.types._
-
-import org.apache.hudi.AvroConversionUtils._
-import org.apache.hudi.exception.HoodieIncompatibleSchemaException
-
-import scala.collection.JavaConverters._
-
-object AvroConversionHelper {
-
-  private def createDecimal(decimal: java.math.BigDecimal, precision: Int, scale: Int): Decimal = {
-    if (precision <= Decimal.MAX_LONG_DIGITS) {
-      // Constructs a `Decimal` with an unscaled `Long` value if possible.
-      Decimal(decimal.unscaledValue().longValue(), precision, scale)
-    } else {
-      // Otherwise, resorts to an unscaled `BigInteger` instead.
-      Decimal(decimal, precision, scale)
-    }
-  }
-
-  /**
-    *
-    * Returns a converter function to convert row in avro format to GenericRow of catalyst.
-    *
-    * @param sourceAvroSchema Source schema before conversion inferred from avro file by passed in
-    *                         by user.
-    * @param targetSqlType    Target catalyst sql type after the conversion.
-    * @return returns a converter function to convert row in avro format to GenericRow of catalyst.
-    */
-  def createConverterToRow(sourceAvroSchema: Schema,

Review comment:
       what happens to all this code? consolidated? if so, are we sure there are no subtle differences from the different conversion implementations?

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -18,20 +18,105 @@
 
 package org.apache.hudi
 
-import org.apache.avro.Schema
-import org.apache.avro.JsonProperties
+import org.apache.avro.Schema.Type
 import org.apache.avro.generic.{GenericRecord, GenericRecordBuilder, IndexedRecord}
+import org.apache.avro.{AvroRuntimeException, JsonProperties, Schema}
+import org.apache.hudi.HoodieSparkUtils.sparkAdapter
 import org.apache.hudi.avro.HoodieAvroUtils
 import org.apache.spark.rdd.RDD
 import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.types.StructType
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.encoders.RowEncoder
+import org.apache.spark.sql.types.{DataType, StructType}
 import org.apache.spark.sql.{Dataset, Row, SparkSession}
 
-import scala.collection.JavaConverters._
 import scala.collection.JavaConversions._
+import scala.collection.JavaConverters._
 
 object AvroConversionUtils {
 
+  /**
+   * Check the nullability of the input Avro type and resolve it when it is nullable. The first
+   * return value is a [[Boolean]] indicating if the input Avro type is nullable. The second
+   * return value is either provided Avro type if it's not nullable, or its resolved non-nullable part
+   * in case it is
+   */
+  def resolveAvroTypeNullability(avroType: Schema): (Boolean, Schema) = {

Review comment:
       My usual comment. :) where are these methods from? what has changed

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       >we even have out own HoodieInternalRow extension
   This is a different code path altogether. 
   
   but I think its okay to depend on InternalRow. Spark's undocumented APIs are a pain regardless

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       To be clear, we already have a row writer path, which is exactly on par with any other framework's parquet writing. On the upsert path, where this code is used, the bottleneck is indexing and we already have significant speedups due to the rich set of indexing schemes.That said, using internalrow is ok




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

xushiyan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814523043



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(
+        new HoodieLocalEngineContext(config), metadataConfig,
+        dataTableBasePath,
+        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+
+      // NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
+      //       of indirection among MT partitions)
+      val relativePartitionPath = FSUtils.getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+      metadataTable.getLogRecordScanner(split.logFiles.get.asJava, relativePartitionPath).getLeft
+    } else {
+      HoodieMergedLogRecordScanner.newBuilder()
+        .withFileSystem(fs)
+        .withBasePath(split.tablePath)
+        .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)

Review comment:
       @alexeykudinkin sorry not sure why the diff is shown differently maybe due to i was hiding whitespaces. My original question is on why `withLogFilePaths(split.logPaths.get.asJava)` is not good enough and you need to map it as shown in L340.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813383960



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       @yihua you brought up legitimate concerns, there are a few considerations here
   
    - First of all, we already depend on `InternalRow` quite a bit (we even have out own `HoodieInternalRow` extension)
    - `InternalRow` is a core component of Spark that is unlikely to change substantially (as that would mean that quite a bit of Spark will have to be re-written to accommodate for any substantial changes to it, which, again, i don't think are likely)
   
   At the same time avoiding `InternalRow` > `Row` conversion has some considerable performance advantages  , and it's exactly how Spark operates internally: all of its internal `Plan`s, expressions, operators, operate on `InternalRow` and defer such deserialization (`InternalRow` to `Row`) only to cases when you dereference it to `RDD[Row]`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042301794


   @hudi-bot run azure re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042601092


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045357161


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1044124061


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035845182


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036948138


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036839012


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r806235304



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,44 +89,48 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
+    val tableSchemas = HoodieTableSchemas(
+      tableSchema = tableStructSchema,
       partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
-      filters = Seq.empty,
+      requiredSchema = requiredStructSchema,
+      tableAvroSchema = tableAvroSchema.toString,
+      requiredAvroSchema = requiredAvroSchema.toString
+    )
+    val tableState = HoodieMergeOnReadTableState(tableSchemas, fileIndex, recordKeyField, preCombineFieldOpt)
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      tableSchemas = tableSchemas,
+      filters = filters,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      hadoopConf = conf
     )
-
-    val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(

Review comment:
       Good catch! Fixed the ref to use method from `DataSourceHelper`

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +329,33 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(new HoodieLocalEngineContext(config), metadataConfig, dataTableBasePath, "/tmp")
+
+      metadataTable.getLogRecordScanner(split.logPaths.get.asJava, "blah").getLeft

Review comment:
       > can we fix "blah" ? -> "files" we are going to fetch records only from "files" partition right? or even for any partition ?
   
   Yeah, we're fetching all partitions currently and we need to figure out a way how we want to expose these partitions since they're practically have different schemas

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -82,16 +76,11 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
 
   private val fileIndex = if (commitsToReturn.isEmpty) List() else buildFileIndex()
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
+  private val preCombineFieldOpt =
+    Option(metaClient.getTableConfig.getPreCombineField)
       // get preCombineFiled from the options if this is a old table which have not store
       // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
+      .orElse(optParams.get(DataSourceWriteOptions.PRECOMBINE_FIELD.key))

Review comment:
       They refer to the same key def




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1037001375






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041215482


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042302553


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041932668


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045462920


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043812869


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035738461


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035611194


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036693350


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036767829


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036512709


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036543432


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036543432


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] YannByron commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

YannByron commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r811530516



##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/RawTripTestPayload.java
##########
@@ -80,6 +80,23 @@ public RawTripTestPayload(String jsonData) throws IOException {
     this.isDeleted = false;
   }
 
+  /**
+   * @deprecated PLEASE READ THIS CAREFULLY
+   *
+   * Converting properly typed schemas into JSON leads to inevitable information loss, since JSON
+   * encodes only representation of the record (with no schema accompanying it), therefore occasionally
+   * losing nuances of the original data-types provided by the schema (for ex, with 1.23 literal it's
+   * impossible to tell whether original type was Double or Decimal).
+   *
+   * Multiplied by the fact that Spark 2 JSON schema inference has substantial gaps in it (see below),
+   * it's **NOT RECOMMENDED** to use this method. Instead please consider using {@link AvroConversionUtils#createDataframe()}
+   * method accepting list of {@link HoodieRecord} (as produced by the {@link HoodieTestDataGenerator}
+   * to create Spark's {@code Dataframe}s directly.
+   *
+   * REFs
+   * https://medium.com/swlh/notes-about-json-schema-handling-in-spark-sql-be1e7f13839d
+   */
+  @Deprecated

Review comment:
       yep, agree with you. There is a difference to parse some field types between spark versions.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049210651


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049210651


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049306709


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049429939


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252",
       "triggerID" : "1049396928",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

xushiyan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812935783



##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/RawTripTestPayload.java
##########
@@ -80,6 +80,23 @@ public RawTripTestPayload(String jsonData) throws IOException {
     this.isDeleted = false;
   }
 
+  /**
+   * @deprecated PLEASE READ THIS CAREFULLY
+   *
+   * Converting properly typed schemas into JSON leads to inevitable information loss, since JSON
+   * encodes only representation of the record (with no schema accompanying it), therefore occasionally
+   * losing nuances of the original data-types provided by the schema (for ex, with 1.23 literal it's
+   * impossible to tell whether original type was Double or Decimal).
+   *
+   * Multiplied by the fact that Spark 2 JSON schema inference has substantial gaps in it (see below),
+   * it's **NOT RECOMMENDED** to use this method. Instead please consider using {@link AvroConversionUtils#createDataframe()}
+   * method accepting list of {@link HoodieRecord} (as produced by the {@link HoodieTestDataGenerator}
+   * to create Spark's {@code Dataframe}s directly.
+   *
+   * REFs
+   * https://medium.com/swlh/notes-about-json-schema-handling-in-spark-sql-be1e7f13839d
+   */
+  @Deprecated

Review comment:
       ok noted. thanks. will convey this point.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035842914


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035609104


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035744808


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r805009590



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionHelper.scala
##########
@@ -18,363 +18,30 @@
 
 package org.apache.hudi
 
-import java.nio.ByteBuffer
-import java.sql.{Date, Timestamp}
-import java.time.Instant
-
-import org.apache.avro.Conversions.DecimalConversion
-import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
-import org.apache.avro.Schema.Type._
-import org.apache.avro.generic.GenericData.{Fixed, Record}
-import org.apache.avro.generic.{GenericData, GenericFixed, GenericRecord}
-import org.apache.avro.{LogicalTypes, Schema}
-
-import org.apache.spark.sql.Row
-import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.catalyst.expressions.GenericRow
-import org.apache.spark.sql.catalyst.util.DateTimeUtils
+import org.apache.avro.Schema
 import org.apache.spark.sql.types._
 
-import org.apache.hudi.AvroConversionUtils._
-import org.apache.hudi.exception.HoodieIncompatibleSchemaException
-
-import scala.collection.JavaConverters._
-
 object AvroConversionHelper {
 
-  private def createDecimal(decimal: java.math.BigDecimal, precision: Int, scale: Int): Decimal = {
-    if (precision <= Decimal.MAX_LONG_DIGITS) {
-      // Constructs a `Decimal` with an unscaled `Long` value if possible.
-      Decimal(decimal.unscaledValue().longValue(), precision, scale)
-    } else {
-      // Otherwise, resorts to an unscaled `BigInteger` instead.
-      Decimal(decimal, precision, scale)
-    }
-  }
-
   /**
-    *
-    * Returns a converter function to convert row in avro format to GenericRow of catalyst.
-    *
-    * @param sourceAvroSchema Source schema before conversion inferred from avro file by passed in
-    *                         by user.
-    * @param targetSqlType    Target catalyst sql type after the conversion.
-    * @return returns a converter function to convert row in avro format to GenericRow of catalyst.
+    * @deprecated please use [[AvroConversionUtils.createRowToAvroConverter]]

Review comment:
       guess you meant createAvroToRowConverter 

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -82,16 +76,11 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
 
   private val fileIndex = if (commitsToReturn.isEmpty) List() else buildFileIndex()
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
+  private val preCombineFieldOpt =
+    Option(metaClient.getTableConfig.getPreCombineField)
       // get preCombineFiled from the options if this is a old table which have not store
       // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
+      .orElse(optParams.get(DataSourceWriteOptions.PRECOMBINE_FIELD.key))

Review comment:
       guess this should be READ_PRE_COMBINE_FIELD

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +329,33 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(new HoodieLocalEngineContext(config), metadataConfig, dataTableBasePath, "/tmp")
+
+      metadataTable.getLogRecordScanner(split.logPaths.get.asJava, "blah").getLeft

Review comment:
       can we fix "blah" ? -> "files" we are going to fetch records only from "files" partition right? or even for any partition ? 

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -41,15 +58,91 @@ abstract class HoodieBaseRelation(
 
   protected val sparkSession: SparkSession = sqlContext.sparkSession
 
-  protected val tableAvroSchema: Schema = {
+  protected lazy val tableAvroSchema: Schema = {
     val schemaUtil = new TableSchemaResolver(metaClient)
-    Try (schemaUtil.getTableAvroSchema).getOrElse(SchemaConverters.toAvroType(userSchema.get))
+    Try(schemaUtil.getTableAvroSchema).getOrElse(
+      // If there is no commit in the table, we can't get the schema
+      // t/h [[TableSchemaResolver]], fallback to provided the [[userSchema]] instead.
+      userSchema match {
+        case Some(s) => SchemaConverters.toAvroType(s)
+        case _ => throw new IllegalArgumentException("User-provided schema is required in case the table is empty")
+      }
+    )
   }
 
   protected val tableStructSchema: StructType = AvroConversionUtils.convertAvroSchemaToStructType(tableAvroSchema)
 
   protected val partitionColumns: Array[String] = metaClient.getTableConfig.getPartitionFields.orElse(Array.empty)
 
-  override def schema: StructType = userSchema.getOrElse(tableStructSchema)
+  override def schema: StructType = tableStructSchema
+}
+
+object HoodieBaseRelation {
+
+  def isMetadataTable(metaClient: HoodieTableMetaClient) =
+    HoodieTableMetadata.isMetadataTable(metaClient.getBasePath)
 
+  /**
+   * Returns file-reader routine accepting [[PartitionedFile]] and returning an [[Iterator]]
+   * over [[InternalRow]]
+   */
+  def createBaseFileReader(spark: SparkSession,
+                           tableSchemas: HoodieTableSchemas,
+                           filters: Array[Filter],
+                           options: Map[String, String],
+                           hadoopConf: Configuration): PartitionedFile => Iterator[InternalRow] = {
+    val hfileReader = createHFileReader(

Review comment:
       can we move this to else block in L116. 

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -155,8 +151,7 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
     // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
     // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
+    df.queryExecution.toRdd.mapPartitions { records =>

Review comment:
       prior to this patch, records were Iterator[Row] and now its Iterator[InternalRow]. so the transition did not cause any issues is it? bcoz, in L157 
   ```
   AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace), 
   ```
   1st arg is 
   ```
   AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema)
   ```
   which refers to StructType. 
   can you help me understand. for Row we have a schema of type StructType, but for InternalRow there is no schema as such right. 
   
   
   
   

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -41,15 +58,91 @@ abstract class HoodieBaseRelation(
 
   protected val sparkSession: SparkSession = sqlContext.sparkSession
 
-  protected val tableAvroSchema: Schema = {
+  protected lazy val tableAvroSchema: Schema = {
     val schemaUtil = new TableSchemaResolver(metaClient)
-    Try (schemaUtil.getTableAvroSchema).getOrElse(SchemaConverters.toAvroType(userSchema.get))
+    Try(schemaUtil.getTableAvroSchema).getOrElse(
+      // If there is no commit in the table, we can't get the schema
+      // t/h [[TableSchemaResolver]], fallback to provided the [[userSchema]] instead.
+      userSchema match {
+        case Some(s) => SchemaConverters.toAvroType(s)
+        case _ => throw new IllegalArgumentException("User-provided schema is required in case the table is empty")
+      }
+    )
   }
 
   protected val tableStructSchema: StructType = AvroConversionUtils.convertAvroSchemaToStructType(tableAvroSchema)
 
   protected val partitionColumns: Array[String] = metaClient.getTableConfig.getPartitionFields.orElse(Array.empty)
 
-  override def schema: StructType = userSchema.getOrElse(tableStructSchema)
+  override def schema: StructType = tableStructSchema
+}
+
+object HoodieBaseRelation {
+
+  def isMetadataTable(metaClient: HoodieTableMetaClient) =
+    HoodieTableMetadata.isMetadataTable(metaClient.getBasePath)
 
+  /**
+   * Returns file-reader routine accepting [[PartitionedFile]] and returning an [[Iterator]]
+   * over [[InternalRow]]
+   */
+  def createBaseFileReader(spark: SparkSession,
+                           tableSchemas: HoodieTableSchemas,
+                           filters: Array[Filter],
+                           options: Map[String, String],
+                           hadoopConf: Configuration): PartitionedFile => Iterator[InternalRow] = {
+    val hfileReader = createHFileReader(
+      spark = spark,
+      tableSchemas = tableSchemas,
+      filters = filters,
+      options = options,
+      hadoopConf = hadoopConf
+    )
+    val parquetReader = new ParquetFileFormat().buildReaderWithPartitionValues(

Review comment:
       also, can we move this to L114

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,44 +89,48 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
+    val tableSchemas = HoodieTableSchemas(
+      tableSchema = tableStructSchema,
       partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
-      filters = Seq.empty,
+      requiredSchema = requiredStructSchema,
+      tableAvroSchema = tableAvroSchema.toString,
+      requiredAvroSchema = requiredAvroSchema.toString
+    )
+    val tableState = HoodieMergeOnReadTableState(tableSchemas, fileIndex, recordKeyField, preCombineFieldOpt)
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      tableSchemas = tableSchemas,
+      filters = filters,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      hadoopConf = conf
     )
-
-    val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(

Review comment:
       prior to this patch we had some manipulation of how we were reading parquet files
   ```
       file: PartitionedFile => {
         val iter = readParquetFile(file)
         val rows = iter.flatMap(_ match {
           case r: InternalRow => Seq(r)
           case b: ColumnarBatch => b.rowIterator().asScala
         })
         rows
       }
   ```
   excerpt from HoodieDataSourceHelper.buildHoodieParquetReader.
   Looks like we need this for vectorized reader. this is very critical for performance. May be we can embed this into HoodieBaseRelation,createBaseFileReader only so that all callers benefit. 

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +329,33 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(new HoodieLocalEngineContext(config), metadataConfig, dataTableBasePath, "/tmp")
+
+      metadataTable.getLogRecordScanner(split.logPaths.get.asJava, "blah").getLeft

Review comment:
       for last arg, can we do config.getSpillableMapBasePath

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -72,19 +66,18 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
 
   private val maxCompactionMemoryInBytes = getMaxCompactionMemoryInBytes(jobConf)
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
+  private val recordKeyField = metaClient.getTableConfig.getRecordKeyFieldProp
+  private val preCombineFieldOpt =
+    Option(metaClient.getTableConfig.getPreCombineField)
       // get preCombineFiled from the options if this is a old table which have not store
       // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
-  private var recordKeyFieldOpt = Option.empty[String]
-  if (!metaClient.getTableConfig.populateMetaFields()) {
-    recordKeyFieldOpt = Option(metaClient.getTableConfig.getRecordKeyFieldProp)
+      .orElse(optParams.get(DataSourceWriteOptions.PRECOMBINE_FIELD.key))

Review comment:
       DataSourceReadOptions.READ_PRE_COMBINE_FIELD




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036839012


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036984921


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036984169


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036984169


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041999593


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041215482


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin edited a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin edited a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042301794


   @hudi-bot run azure
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r811395407



##########
File path: hudi-spark-datasource/hudi-spark2/pom.xml
##########
@@ -199,6 +199,15 @@
       <groupId>org.apache.spark</groupId>
       <artifactId>spark-sql_${scala.binary.version}</artifactId>
       <version>${spark2.version}</version>
+      <scope>provided</scope>

Review comment:
       we also have few other poms that might need similar fixing. 
   hudi-spark-datasource/hudi-spark-common
   hudi-spark-datasource/hudi-spark3.1.x
   hudi-spark-datasource/hudi-spark
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] YannByron commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

YannByron commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r811529153



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -526,6 +526,7 @@ private boolean hasOperationField() {
       Schema tableAvroSchema = getTableAvroSchemaFromDataFile();
       return tableAvroSchema.getField(HoodieRecord.OPERATION_METADATA_FIELD) != null;
     } catch (Exception e) {
+      LOG.info(String.format("Failed to read operation field from avro schema (%s)", e.getMessage()));

Review comment:
       i recommend there is no logging. It'll affect the user experience. See the logging of CI, you can find this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814236329



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -18,20 +18,105 @@
 
 package org.apache.hudi
 
-import org.apache.avro.Schema
-import org.apache.avro.JsonProperties
+import org.apache.avro.Schema.Type
 import org.apache.avro.generic.{GenericRecord, GenericRecordBuilder, IndexedRecord}
+import org.apache.avro.{AvroRuntimeException, JsonProperties, Schema}
+import org.apache.hudi.HoodieSparkUtils.sparkAdapter
 import org.apache.hudi.avro.HoodieAvroUtils
 import org.apache.spark.rdd.RDD
 import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.types.StructType
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.encoders.RowEncoder
+import org.apache.spark.sql.types.{DataType, StructType}
 import org.apache.spark.sql.{Dataset, Row, SparkSession}
 
-import scala.collection.JavaConverters._
 import scala.collection.JavaConversions._
+import scala.collection.JavaConverters._
 
 object AvroConversionUtils {
 
+  /**
+   * Check the nullability of the input Avro type and resolve it when it is nullable. The first
+   * return value is a [[Boolean]] indicating if the input Avro type is nullable. The second
+   * return value is either provided Avro type if it's not nullable, or its resolved non-nullable part
+   * in case it is
+   */
+  def resolveAvroTypeNullability(avroType: Schema): (Boolean, Schema) = {

Review comment:
       This is borrowed from Spark 3

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionHelper.scala
##########
@@ -1,380 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *      http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.hudi
-
-import java.nio.ByteBuffer
-import java.sql.{Date, Timestamp}
-import java.time.Instant
-
-import org.apache.avro.Conversions.DecimalConversion
-import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
-import org.apache.avro.Schema.Type._
-import org.apache.avro.generic.GenericData.{Fixed, Record}
-import org.apache.avro.generic.{GenericData, GenericFixed, GenericRecord}
-import org.apache.avro.{LogicalTypes, Schema}
-
-import org.apache.spark.sql.Row
-import org.apache.spark.sql.avro.SchemaConverters
-import org.apache.spark.sql.catalyst.expressions.GenericRow
-import org.apache.spark.sql.catalyst.util.DateTimeUtils
-import org.apache.spark.sql.types._
-
-import org.apache.hudi.AvroConversionUtils._
-import org.apache.hudi.exception.HoodieIncompatibleSchemaException
-
-import scala.collection.JavaConverters._
-
-object AvroConversionHelper {
-
-  private def createDecimal(decimal: java.math.BigDecimal, precision: Int, scale: Int): Decimal = {
-    if (precision <= Decimal.MAX_LONG_DIGITS) {
-      // Constructs a `Decimal` with an unscaled `Long` value if possible.
-      Decimal(decimal.unscaledValue().longValue(), precision, scale)
-    } else {
-      // Otherwise, resorts to an unscaled `BigInteger` instead.
-      Decimal(decimal, precision, scale)
-    }
-  }
-
-  /**
-    *
-    * Returns a converter function to convert row in avro format to GenericRow of catalyst.
-    *
-    * @param sourceAvroSchema Source schema before conversion inferred from avro file by passed in
-    *                         by user.
-    * @param targetSqlType    Target catalyst sql type after the conversion.
-    * @return returns a converter function to convert row in avro format to GenericRow of catalyst.
-    */
-  def createConverterToRow(sourceAvroSchema: Schema,

Review comment:
       This code is just removed, instead we now rely on Spark's `Avro{Serializer,Deserializer}` impls.
   
   Only subtle difference b/w this code and Spark serializers/de is the fact that this one is doing "forgiving" conversion of struct schemas, while Spark's one is strict. This has already been addressed at the caller's side

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializerTrait.scala
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+/**
+ * Deserializes Avro payload into Catalyst object
+ *
+ * NOTE: This is low-level component operating on Spark internal data-types (comprising [[InternalRow]]).
+ *       If you're looking to convert Avro into "deserialized" [[Row]] (comprised of Java native types),
+ *       please check [[AvroConversionUtils]]
+ */
+trait HoodieAvroDeserializerTrait {
+  final def deserialize(data: Any): Option[Any] =
+    doDeserialize(data) match {
+      case opt: Option[_] => opt    // As of Spark 3.1, this will return data wrapped with Option, so we fetch the data

Review comment:
       Good catch! Code is already bifurcated b/w Spark 2 and Spark 3, this is just overlooked remnant




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049255120


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

yihua commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049269294


   @alexeykudinkin have you compared the performance of the old serde vs new ones?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036515084


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043812869


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043751432


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043907285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042572083


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045548402


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813312349



##########
File path: hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/PatchedAvroDeserializer.scala
##########
@@ -0,0 +1,398 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+import org.apache.avro.Conversions.DecimalConversion
+import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
+import org.apache.avro.Schema.Type._
+import org.apache.avro.generic._
+import org.apache.avro.util.Utf8
+import org.apache.avro.{LogicalTypes, Schema, SchemaBuilder}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{SpecificInternalRow, UnsafeArrayData}
+import org.apache.spark.sql.catalyst.util.{ArrayBasedMapData, ArrayData, DateTimeUtils, GenericArrayData}
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+import java.math.BigDecimal
+import java.nio.ByteBuffer
+import scala.collection.JavaConverters._
+import scala.collection.mutable.ArrayBuffer
+
+/**
+ * A deserializer to deserialize data in avro format to data in catalyst format.
+ *
+ * NOTE: This is a version of {@code AvroDeserializer} impl from Spark 2.4.4 w/ the fix for SPARK-30267

Review comment:
       I've applied just one patch that resolves the failures i was getting

##########
File path: hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/PatchedAvroDeserializer.scala
##########
@@ -0,0 +1,398 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+import org.apache.avro.Conversions.DecimalConversion
+import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
+import org.apache.avro.Schema.Type._
+import org.apache.avro.generic._
+import org.apache.avro.util.Utf8
+import org.apache.avro.{LogicalTypes, Schema, SchemaBuilder}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{SpecificInternalRow, UnsafeArrayData}
+import org.apache.spark.sql.catalyst.util.{ArrayBasedMapData, ArrayData, DateTimeUtils, GenericArrayData}
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+import java.math.BigDecimal
+import java.nio.ByteBuffer
+import scala.collection.JavaConverters._
+import scala.collection.mutable.ArrayBuffer
+
+/**
+ * A deserializer to deserialize data in avro format to data in catalyst format.
+ *
+ * NOTE: This is a version of {@code AvroDeserializer} impl from Spark 2.4.4 w/ the fix for SPARK-30267

Review comment:
       I've just replicated one patch that resolves the failures i was getting




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049288403


   > @alexeykudinkin have you compared the performance of the old serde vs new ones?
   
   Not yet, i was planning to take a test after the whole stack lands, as there are more fixes coming. Let me try measure this one in isolation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043707104


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049217866


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049396928


   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812932709



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +129,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = AvroConversionUtils.resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)

Review comment:
       sounds good ! 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049306709


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812321447



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -526,6 +526,7 @@ private boolean hasOperationField() {
       Schema tableAvroSchema = getTableAvroSchemaFromDataFile();
       return tableAvroSchema.getField(HoodieRecord.OPERATION_METADATA_FIELD) != null;
     } catch (Exception e) {
+      LOG.info(String.format("Failed to read operation field from avro schema (%s)", e.getMessage()));

Review comment:
       I've removed the exception logging, so it should not be noisy. I still do think that we should keep at least 1-line log-statement that something failed hence we fallback




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039576906


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039632509


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039632509


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041932668


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r808583562



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -227,17 +223,20 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
                                          baseFileIterator: Iterator[InternalRow],
                                          config: Configuration): Iterator[InternalRow] =
     new Iterator[InternalRow] with Closeable {
-      private val tableAvroSchema = new Schema.Parser().parse(tableState.tableAvroSchema)
-      private val requiredAvroSchema = new Schema.Parser().parse(tableState.requiredAvroSchema)
-      private val serializer = HoodieAvroSerializer(tableState.tableStructSchema, tableAvroSchema, false)
-      private val requiredDeserializer = HoodieAvroDeserializer(requiredAvroSchema, tableState.requiredStructSchema)
+      private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+      private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+      private val requiredFieldPosition =
+        requiredSchema.structTypeSchema
+          .map(f => tableAvroSchema.getField(f.name).pos()).toList
+      private val requiredSerializer = HoodieAvroSerializer(requiredSchema.structTypeSchema, requiredAvroSchema, nullable = false)
+      private val requiredDeserializer = HoodieAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
       private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
-      private val unsafeProjection = UnsafeProjection.create(tableState.requiredStructSchema)
+      private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
       private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
       private val logRecords = logScanner.getRecords
       private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
       private val keyToSkip = mutable.Set.empty[String]
-      private val recordKeyPosition = if (recordKeyFieldOpt.isEmpty) HOODIE_RECORD_KEY_COL_POS else tableState.tableStructSchema.fieldIndex(recordKeyFieldOpt.get)
+      private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)

Review comment:
       recordKeyField will either point to hoodie meta field or one of user's fields (incase of virtual cols) is it? and hence safe to directly fetch the fieldIndex ?

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       why using a deprecated api? can't we use newer apis? if its still in use, why deprecate it ?

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +130,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)
+          }
+
+        // Since caller might request to get records in a different ("evolved") schema, we will be rewriting from
+        // existing Writer's schema into Reader's (avro) schema
+        val writerAvroSchema = new Schema.Parser().parse(writerAvroSchemaStr)
+        val convert = AvroConversionUtils.createInternalRowToAvroConverter(writerSchema, writerAvroSchema, nullable = nullable)

Review comment:
       can't we directly use readerAvroSchemaStr here to convert from InternalRow to Avro. and we can avoid L 165 ?
   or am I missing something. 
   

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +319,48 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(
+        new HoodieLocalEngineContext(config), metadataConfig,
+        dataTableBasePath,
+        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+
+      // NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
+      //       of indirection among MT partitions)
+      val relativePartitionPath = FSUtils.getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+      metadataTable.getLogRecordScanner(split.logFiles.get.asJava, relativePartitionPath).getLeft
+    } else {
+      HoodieMergedLogRecordScanner.newBuilder()
+        .withFileSystem(fs)
+        .withBasePath(split.tablePath)
+        .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)
+        .withReaderSchema(logSchema)
+        .withLatestInstantTime(split.latestCommit)
+        .withReadBlocksLazily(
+          Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
+            HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
+            .getOrElse(false))
+        .withReverseReader(false)
+        .withBufferSize(
+          config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
+            HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
+        .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
+        .withSpillableMapBasePath(
+          config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
+            HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+        .build()
+    }
+  }
+
+  private def getPartitionPath(split: HoodieMergeOnReadFileSplit): Path = {

Review comment:
       getMetadataTablePartitionPath

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -125,41 +114,39 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
       val (requiredAvroSchema, requiredStructSchema) =
         HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
 
-      val hoodieTableState = HoodieMergeOnReadTableState(
-        tableStructSchema,
-        requiredStructSchema,
-        tableAvroSchema.toString,
-        requiredAvroSchema.toString,
-        fileIndex,
-        preCombineField,
-        Option.empty
-      )
-      val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val partitionSchema = StructType(Nil)
+      val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+      val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+      val fullSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = tableSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
-
-      val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val requiredSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = requiredSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
 
+      val hoodieTableState = HoodieMergeOnReadTableState(fileIndex, HoodieRecord.RECORD_KEY_METADATA_FIELD, preCombineFieldOpt)

Review comment:
       2nd arg might differ for metadata table I guess




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042544937


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036656641


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036695720


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035842914


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036564746


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036545819


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041019012


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041999593


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042544937


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041929724


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041929724


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043710170


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045395366


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045462920


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036984921


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036564746


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036698025


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036970896


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r805114615



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -41,15 +58,91 @@ abstract class HoodieBaseRelation(
 
   protected val sparkSession: SparkSession = sqlContext.sparkSession
 
-  protected val tableAvroSchema: Schema = {
+  protected lazy val tableAvroSchema: Schema = {
     val schemaUtil = new TableSchemaResolver(metaClient)
-    Try (schemaUtil.getTableAvroSchema).getOrElse(SchemaConverters.toAvroType(userSchema.get))
+    Try(schemaUtil.getTableAvroSchema).getOrElse(
+      // If there is no commit in the table, we can't get the schema
+      // t/h [[TableSchemaResolver]], fallback to provided the [[userSchema]] instead.
+      userSchema match {
+        case Some(s) => SchemaConverters.toAvroType(s)
+        case _ => throw new IllegalArgumentException("User-provided schema is required in case the table is empty")
+      }
+    )
   }
 
   protected val tableStructSchema: StructType = AvroConversionUtils.convertAvroSchemaToStructType(tableAvroSchema)
 
   protected val partitionColumns: Array[String] = metaClient.getTableConfig.getPartitionFields.orElse(Array.empty)
 
-  override def schema: StructType = userSchema.getOrElse(tableStructSchema)
+  override def schema: StructType = tableStructSchema
+}
+
+object HoodieBaseRelation {
+
+  def isMetadataTable(metaClient: HoodieTableMetaClient) =
+    HoodieTableMetadata.isMetadataTable(metaClient.getBasePath)
 
+  /**
+   * Returns file-reader routine accepting [[PartitionedFile]] and returning an [[Iterator]]
+   * over [[InternalRow]]
+   */
+  def createBaseFileReader(spark: SparkSession,
+                           tableSchemas: HoodieTableSchemas,
+                           filters: Array[Filter],
+                           options: Map[String, String],
+                           hadoopConf: Configuration): PartitionedFile => Iterator[InternalRow] = {
+    val hfileReader = createHFileReader(

Review comment:
       We can not b/c it have to be executed by the Driver




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041019012


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r806235304



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,44 +89,48 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
+    val tableSchemas = HoodieTableSchemas(
+      tableSchema = tableStructSchema,
       partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
-      filters = Seq.empty,
+      requiredSchema = requiredStructSchema,
+      tableAvroSchema = tableAvroSchema.toString,
+      requiredAvroSchema = requiredAvroSchema.toString
+    )
+    val tableState = HoodieMergeOnReadTableState(tableSchemas, fileIndex, recordKeyField, preCombineFieldOpt)
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      tableSchemas = tableSchemas,
+      filters = filters,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      hadoopConf = conf
     )
-
-    val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(

Review comment:
       Good catch! Fixed the ref to use method from `DataSourceHelper`

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -325,24 +329,33 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(new HoodieLocalEngineContext(config), metadataConfig, dataTableBasePath, "/tmp")
+
+      metadataTable.getLogRecordScanner(split.logPaths.get.asJava, "blah").getLeft

Review comment:
       > can we fix "blah" ? -> "files" we are going to fetch records only from "files" partition right? or even for any partition ?
   
   Yeah, we're fetching all partitions currently and we need to figure out a way how we want to expose these partitions since they're practically have different schemas

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -82,16 +76,11 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
 
   private val fileIndex = if (commitsToReturn.isEmpty) List() else buildFileIndex()
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
+  private val preCombineFieldOpt =
+    Option(metaClient.getTableConfig.getPreCombineField)
       // get preCombineFiled from the options if this is a old table which have not store
       // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
+      .orElse(optParams.get(DataSourceWriteOptions.PRECOMBINE_FIELD.key))

Review comment:
       They refer to the same key def




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041017395


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814278496



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializerTrait.scala
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+/**
+ * Deserializes Avro payload into Catalyst object
+ *
+ * NOTE: This is low-level component operating on Spark internal data-types (comprising [[InternalRow]]).
+ *       If you're looking to convert Avro into "deserialized" [[Row]] (comprised of Java native types),
+ *       please check [[AvroConversionUtils]]
+ */
+trait HoodieAvroDeserializerTrait {
+  final def deserialize(data: Any): Option[Any] =
+    doDeserialize(data) match {
+      case opt: Option[_] => opt    // As of Spark 3.1, this will return data wrapped with Option, so we fetch the data

Review comment:
       Addressing this in https://github.com/apache/hudi/pull/4818




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

xushiyan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812857595



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/ComplexAvroKeyGenerator.java
##########
@@ -32,10 +32,14 @@
 
   public ComplexAvroKeyGenerator(TypedProperties props) {
     super(props);
-    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
-    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
+    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());
+    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());

Review comment:
       @alexeykudinkin this is a bit hard for reviewers to tell if there is any logic change or just style change. let's try keep style changes out from big PRs like this. Once we got spotless to help standardize all formats (https://issues.apache.org/jira/browse/HUDI-1237), we can forget about formatting manually and always leave to the formatter.

##########
File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java
##########
@@ -37,10 +37,14 @@
 
   public ComplexKeyGenerator(TypedProperties props) {
     super(props);
-    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
-    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
+    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());
+    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());

Review comment:
       ditto

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()

Review comment:
       just init `metadataConfig` with everything defaults from HoodieMetadataConfig here?

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(
+        new HoodieLocalEngineContext(config), metadataConfig,
+        dataTableBasePath,
+        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+
+      // NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
+      //       of indirection among MT partitions)
+      val relativePartitionPath = FSUtils.getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+      metadataTable.getLogRecordScanner(split.logFiles.get.asJava, relativePartitionPath).getLeft
+    } else {
+      HoodieMergedLogRecordScanner.newBuilder()
+        .withFileSystem(fs)
+        .withBasePath(split.tablePath)
+        .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)

Review comment:
       can you help me understand why `logPaths` can't be used here?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812326499



##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/RawTripTestPayload.java
##########
@@ -80,6 +80,23 @@ public RawTripTestPayload(String jsonData) throws IOException {
     this.isDeleted = false;
   }
 
+  /**
+   * @deprecated PLEASE READ THIS CAREFULLY
+   *
+   * Converting properly typed schemas into JSON leads to inevitable information loss, since JSON
+   * encodes only representation of the record (with no schema accompanying it), therefore occasionally
+   * losing nuances of the original data-types provided by the schema (for ex, with 1.23 literal it's
+   * impossible to tell whether original type was Double or Decimal).
+   *
+   * Multiplied by the fact that Spark 2 JSON schema inference has substantial gaps in it (see below),
+   * it's **NOT RECOMMENDED** to use this method. Instead please consider using {@link AvroConversionUtils#createDataframe()}
+   * method accepting list of {@link HoodieRecord} (as produced by the {@link HoodieTestDataGenerator}
+   * to create Spark's {@code Dataframe}s directly.
+   *
+   * REFs
+   * https://medium.com/swlh/notes-about-json-schema-handling-in-spark-sql-be1e7f13839d
+   */
+  @Deprecated

Review comment:
       @xushiyan @YannByron i actually think that fair amount of compatibility issues as we transition from Spark 2 to 3 would be b/c of this. Would suggest to just point whoever is working on this transition to this new utility to address test failures




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035609104


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035606910


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036970896


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035757575


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036947027


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035744808


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bef62bd803d48f37b418f6bd7b94b2c6f3e51a46 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884) 
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035757575


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4faa17fccc26f7bec81f6322a102757473a1a3b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888) 
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036693350


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036696040


   This is current MT layout in Spark DS
   ```
   +------------------+----+--------------------+-------------------+-------------------+
   |               key|type|  filesystemMetadata|BloomFilterMetadata|ColumnStatsMetadata|
   +------------------+----+--------------------+-------------------+-------------------+
   |        2015/03/16|   2|{4b2d6e17-e6bf-4b...|               null|               null|
   |        2015/03/17|   2|{5e2c7eb4-e58d-48...|               null|               null|
   |        2016/03/15|   2|{1abe3051-0997-4d...|               null|               null|
   |__all_partitions__|   1|{2016/03/15 -> {0...|               null|               null|
   +------------------+----+--------------------+-------------------+-------------------+
   ```
   
   It differs from what we have discussed in [HUDI-1296](https://issues.apache.org/jira/browse/HUDI-1296), and as i've already mentioned in the description that would be addressed in a follow-up PR after current implementation of MOR Relations will be cleaned up (read de-duped)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036700495


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036982866


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043751432


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1044055206


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045357161


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042542397


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042484683


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039547627


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039580411


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002) 
   * 68c672a1ef959ed415f863ddee7aad31373d4758 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039547627






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045548402


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan merged pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan merged pull request #4789:
URL: https://github.com/apache/hudi/pull/4789


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813319386



##########
File path: hudi-spark-datasource/hudi-spark2/pom.xml
##########
@@ -199,6 +199,15 @@
       <groupId>org.apache.spark</groupId>
       <artifactId>spark-sql_${scala.binary.version}</artifactId>
       <version>${spark2.version}</version>
+      <scope>provided</scope>

Review comment:
       After closer examination, this `provided` scoping is redundant since all of this packages are marked as "provided" in the root pom




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813390665



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -323,32 +322,60 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    val partitionPath: String = if (split.logPaths.isEmpty || split.logPaths.get.asJava.isEmpty) {
-      null
+    val logFiles = split.logFiles.get
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {

Review comment:
       It's, but it's actually using its own standalone scanner (meaning it couldn't be read with the default one).
   
   I agree that we should trim down as much as possible such bifurcations and instead make sure that existing Hudi components are extended in a way that can handle MT, without a need for specialized one

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       @yihua you brought up legitimate concerns, there are a few considerations here
   
    - First of all, we already depend on `InternalRow` quite a bit (we even have out own `HoodieInternalRow` extension)
    - `InternalRow` is a core component of Spark that is unlikely to change substantially (as that would mean that quite a bit of Spark will have to be re-written to accommodate for any substantial changes to it, which, again, i don't think are likely)
   
   At the same time avoiding `InternalRow` > `Row` conversion has some considerable performance advantages  , and it's exactly how Spark operates internally: all of its internal `Plan`s, expressions, operators, operate on `InternalRow` and defer such deserialization (`InternalRow` to `Row`) only to cases when you dereference it to `RDD[Row]`.
   
   All in all, for us to be able to compete w/ Delta on performance we will have to pull the same tricks they are using and make sure our overhead is as lean as possible




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049285849


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036767829


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036948138


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036545819


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   * f58c2c79cc940081073ae0314229968741552938 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036515084


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036698025


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036695720


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f58c2c79cc940081073ae0314229968741552938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925) 
   * ca8e099133359ee1393607c02f12bea150fdcd62 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036952896


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   * 8c5553730559d2ee8a643042660e1750d32e3440 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036982108


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8c5553730559d2ee8a643042660e1750d32e3440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036512709


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   * 87d8557c5495870e638866422e7b0cfc42befb15 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043710170


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1044124061


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045467228


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   * 1560af3c8a56e345a03800f187d09c0391d9def0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041224488


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r808611371



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -125,41 +114,39 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
       val (requiredAvroSchema, requiredStructSchema) =
         HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
 
-      val hoodieTableState = HoodieMergeOnReadTableState(
-        tableStructSchema,
-        requiredStructSchema,
-        tableAvroSchema.toString,
-        requiredAvroSchema.toString,
-        fileIndex,
-        preCombineField,
-        Option.empty
-      )
-      val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val partitionSchema = StructType(Nil)
+      val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+      val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+      val fullSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = tableSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
-
-      val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val requiredSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = requiredSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
 
+      val hoodieTableState = HoodieMergeOnReadTableState(fileIndex, HoodieRecord.RECORD_KEY_METADATA_FIELD, preCombineFieldOpt)

Review comment:
       Good catch! MT is not supported in Incremental relation currently.
   
   That would be tackled in a follow-up PR making sure that Incremental/Snapshot have shared base




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042484683


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041017395


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041292999


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042561271


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041009539


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042514254


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r811230085



##########
File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java
##########
@@ -125,48 +127,65 @@ public HoodieMetadataPayload(GenericRecord record, Comparable<?> orderingVal) {
     this(Option.of(record));
   }
 
-  public HoodieMetadataPayload(Option<GenericRecord> record) {
-    if (record.isPresent()) {
+  public HoodieMetadataPayload(Option<GenericRecord> recordOpt) {
+    if (recordOpt.isPresent()) {
+      GenericRecord record = recordOpt.get();
       // This can be simplified using SpecificData.deepcopy once this bug is fixed
       // https://issues.apache.org/jira/browse/AVRO-1811
-      key = record.get().get(KEY_FIELD_NAME).toString();
-      type = (int) record.get().get(SCHEMA_FIELD_NAME_TYPE);
-      if (record.get().get(SCHEMA_FIELD_NAME_METADATA) != null) {
-        filesystemMetadata = (Map<String, HoodieMetadataFileInfo>) record.get().get("filesystemMetadata");
+      //
+      // NOTE: {@code HoodieMetadataRecord} has to always carry both "key" nad "type" fields

Review comment:
       minor typo. "nad" -> "and"

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +129,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = AvroConversionUtils.resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)

Review comment:
       do we know any perf implications due to this. Prior to this patch, we were using AvroConversionHelper.createConverterToAvro to convert from one schema to another. But here we are rewriting the GenericRecord again. I don't have much idea here on the implications. but wanted to call it out. 
   CC @xushiyan if you have any thoughts here.
   

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/catalog/ProvidesHoodieConfig.scala
##########
@@ -71,23 +74,25 @@ trait ProvidesHoodieConfig extends Logging {
         HoodieWriteConfig.UPSERT_PARALLELISM_VALUE.key -> "200",
         SqlKeyGenerator.PARTITION_SCHEMA -> hoodieCatalogTable.partitionSchema.toDDL
       )
+        .filter { case(_, v) => v != null }

Review comment:
       sorry, what is this filter for ? 

##########
File path: hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/PatchedAvroDeserializer.scala
##########
@@ -0,0 +1,398 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.avro
+
+import org.apache.avro.Conversions.DecimalConversion
+import org.apache.avro.LogicalTypes.{TimestampMicros, TimestampMillis}
+import org.apache.avro.Schema.Type._
+import org.apache.avro.generic._
+import org.apache.avro.util.Utf8
+import org.apache.avro.{LogicalTypes, Schema, SchemaBuilder}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{SpecificInternalRow, UnsafeArrayData}
+import org.apache.spark.sql.catalyst.util.{ArrayBasedMapData, ArrayData, DateTimeUtils, GenericArrayData}
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+import java.math.BigDecimal
+import java.nio.ByteBuffer
+import scala.collection.JavaConverters._
+import scala.collection.mutable.ArrayBuffer
+
+/**
+ * A deserializer to deserialize data in avro format to data in catalyst format.
+ *
+ * NOTE: This is a version of {@code AvroDeserializer} impl from Spark 2.4.4 w/ the fix for SPARK-30267

Review comment:
       May I know how did you deduce the list of patches to be applied. 

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -75,84 +77,89 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
 
   private val fileIndex = if (commitsToReturn.isEmpty) List() else buildFileIndex()
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
-      // get preCombineFiled from the options if this is a old table which have not store
-      // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
+  private val preCombineFieldOpt = getPrecombineFieldProperty
 
-  override def needConversion: Boolean = false
+  // Record filters making sure that only records w/in the requested bounds are being fetched as part of the
+  // scan collected by this relation
+  private lazy val incrementalSpanRecordsFilters: Seq[Filter] = {
+    val isNotNullFilter = IsNotNull(HoodieRecord.COMMIT_TIME_METADATA_FIELD)
+    val largerThanFilter = GreaterThanOrEqual(HoodieRecord.COMMIT_TIME_METADATA_FIELD, commitsToReturn.head.getTimestamp)
+    val lessThanFilter = LessThanOrEqual(HoodieRecord.COMMIT_TIME_METADATA_FIELD, commitsToReturn.last.getTimestamp)
+    Seq(isNotNullFilter, largerThanFilter, lessThanFilter)
+  }
 
-  override def unhandledFilters(filters: Array[Filter]): Array[Filter] = {

Review comment:
       why removed overriding unhandledFilters.  I do see we set pushFilters to same set of filters. But wondering if diff set of filters could mean or get applied differrently. 
   
   excerpt from DataSourceStrategy. 
   ```
       val (unhandledPredicates, pushedFilters, handledFilters) =
         selectFilters(relation.relation, candidatePredicates)
   ```
   Do you happened to know whats difference between setting some filter in unhandledPredicates vs pushedFilters here. 
   
   

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,45 +92,56 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
-      partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
+
+    val partitionSchema = StructType(Nil)
+    val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+    val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      partitionSchema = partitionSchema,
+      tableSchema = tableSchema,
+      requiredSchema = tableSchema,
+      // This file-reader is used to read base file records, subsequently merging them with the records
+      // stored in delta-log files. As such, we have to read _all_ records from the base file, while avoiding
+      // applying any filtering _before_ we complete combining them w/ delta-log records (to make sure that
+      // we combine them correctly)
       filters = Seq.empty,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      // NOTE: We have to fork the Hadoop Config here as Spark will be modifying it
+      //       to configure Parquet reader appropriately
+      hadoopConf = new Configuration(conf)

Review comment:
       so prior to this patch, it was co-incidental that our col projection wasn't kicking in and hence we got away with it is it? if not, this would have broken things right. 
   (wrt comment in L 118 to 121)
   

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       sounds good. 

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +130,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)
+          }
+
+        // Since caller might request to get records in a different ("evolved") schema, we will be rewriting from
+        // existing Writer's schema into Reader's (avro) schema
+        val writerAvroSchema = new Schema.Parser().parse(writerAvroSchemaStr)
+        val convert = AvroConversionUtils.createInternalRowToAvroConverter(writerSchema, writerAvroSchema, nullable = nullable)

Review comment:
       got you!

##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala
##########
@@ -56,7 +57,7 @@ case class InsertIntoHoodieTableCommand(
   }
 }
 
-object InsertIntoHoodieTableCommand extends Logging {
+object InsertIntoHoodieTableCommand extends Logging with ProvidesHoodieConfig {

Review comment:
       @YannByron : can you review the sql dml class changes in this patch. 

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -125,41 +114,39 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
       val (requiredAvroSchema, requiredStructSchema) =
         HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
 
-      val hoodieTableState = HoodieMergeOnReadTableState(
-        tableStructSchema,
-        requiredStructSchema,
-        tableAvroSchema.toString,
-        requiredAvroSchema.toString,
-        fileIndex,
-        preCombineField,
-        Option.empty
-      )
-      val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val partitionSchema = StructType(Nil)
+      val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+      val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+      val fullSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = tableSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
-
-      val requiredSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-        sparkSession = sqlContext.sparkSession,
-        dataSchema = tableStructSchema,
-        partitionSchema = StructType(Nil),
-        requiredSchema = tableStructSchema,
+      val requiredSchemaParquetReader = createBaseFileReader(
+        spark = sqlContext.sparkSession,
+        partitionSchema = partitionSchema,
+        tableSchema = tableSchema,
+        requiredSchema = requiredSchema,
         filters = pushDownFilter,
         options = optParams,
         hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
       )
 
+      val hoodieTableState = HoodieMergeOnReadTableState(fileIndex, HoodieRecord.RECORD_KEY_METADATA_FIELD, preCombineFieldOpt)

Review comment:
       👍 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r813173761



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -526,6 +526,7 @@ private boolean hasOperationField() {
       Schema tableAvroSchema = getTableAvroSchemaFromDataFile();
       return tableAvroSchema.getField(HoodieRecord.OPERATION_METADATA_FIELD) != null;
     } catch (Exception e) {
+      LOG.info(String.format("Failed to read operation field from avro schema (%s)", e.getMessage()));

Review comment:
       I would rather reserve "error" level for things that flag malfunction and as such require immediate attention. This is just referring that we're trying to fetch particular failed that wasn't found

##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/ComplexAvroKeyGenerator.java
##########
@@ -32,10 +32,14 @@
 
   public ComplexAvroKeyGenerator(TypedProperties props) {
     super(props);
-    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
-    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key())
-        .split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toList());
+    this.recordKeyFields = Arrays.stream(props.getString(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());
+    this.partitionPathFields = Arrays.stream(props.getString(KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME.key()).split(","))
+        .map(String::trim)
+        .filter(s -> !s.isEmpty())
+        .collect(Collectors.toList());

Review comment:
       Good call. This was actually changed originally to compensate for missing config property (which was previously stubbed w/ "" which didn't make sense) but then i had to rollback this change selectively since it was backfiring in other place, and that's how this has turned out to be here

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
+      val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+      val metadataTable = new HoodieBackedTableMetadata(
+        new HoodieLocalEngineContext(config), metadataConfig,
+        dataTableBasePath,
+        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+
+      // NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
+      //       of indirection among MT partitions)
+      val relativePartitionPath = FSUtils.getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+      metadataTable.getLogRecordScanner(split.logFiles.get.asJava, relativePartitionPath).getLeft
+    } else {
+      HoodieMergedLogRecordScanner.newBuilder()
+        .withFileSystem(fs)
+        .withBasePath(split.tablePath)
+        .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)

Review comment:
       Not sure i understand your Q -- this is requiring exactly log-paths, but am propagating `HoodieLogFile` as part of the split since it's necessary to be able to read MT records (line 335)

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -314,35 +309,65 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
 
       private def mergeRowWithLog(curRow: InternalRow, curKey: String) = {
         val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
-        logRecords.get(curKey).getData.combineAndGetUpdateValue(
-          historyAvroRecord, tableAvroSchema, payloadProps)
+        logRecords.get(curKey).getData
+          .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
       }
     }
-  }
+}
 
 private object HoodieMergeOnReadRDD {
   val CONFIG_INSTANTIATION_LOCK = new Object()
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    HoodieMergedLogRecordScanner.newBuilder()
-      .withFileSystem(fs)
-      .withBasePath(split.tablePath)
-      .withLogFilePaths(split.logPaths.get.asJava)
-      .withReaderSchema(logSchema)
-      .withLatestInstantTime(split.latestCommit)
-      .withReadBlocksLazily(
-        Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
-          HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
-          .getOrElse(false))
-      .withReverseReader(false)
-      .withBufferSize(
-        config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
-          HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
-      .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
-      .withSpillableMapBasePath(
-        config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
-          HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
-      .build()
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+      val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()

Review comment:
       Sorry, folks, not sure i understand what your point is: it's already init'd w/ mostly defaults.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049217866


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * 1560af3c8a56e345a03800f187d09c0391d9def0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138) 
   * 4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049397046


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252",
       "triggerID" : "1049396928",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6252) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1048832798


   @alexeykudinkin : have you addressed the pom feedback. LGTM overall. once you fix the pom and rebase with latest master, go to go from my end. Let's wait to see if vinoth wants to review. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

yihua commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814426514



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -323,32 +322,60 @@ private object HoodieMergeOnReadRDD {
 
   def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
     val fs = FSUtils.getFs(split.tablePath, config)
-    val partitionPath: String = if (split.logPaths.isEmpty || split.logPaths.get.asJava.isEmpty) {
-      null
+    val logFiles = split.logFiles.get
+
+    if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {

Review comment:
       I see.  That makes sense.  Yeah, at some point we should see how to reduce such bifurcations and make them easier to maintain.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045361269


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042478060


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066) 
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042083359


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 429b48e23c06020162ea3d13d0e0bbdec435c793 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042542397


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * 7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076) 
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041292999


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043863308


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6097dec23eac5753a517e6096dea546fc462d092 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108) 
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1044055206


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

alexeykudinkin commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r811428239



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/catalog/ProvidesHoodieConfig.scala
##########
@@ -71,23 +74,25 @@ trait ProvidesHoodieConfig extends Logging {
         HoodieWriteConfig.UPSERT_PARALLELISM_VALUE.key -> "200",
         SqlKeyGenerator.PARTITION_SCHEMA -> hoodieCatalogTable.partitionSchema.toDDL
       )
+        .filter { case(_, v) => v != null }

Review comment:
       Cleaning up any configs that might be nulls (Java Hashtable would fail if passed in any null values)

##########
File path: hudi-spark-datasource/hudi-spark2/pom.xml
##########
@@ -199,6 +199,15 @@
       <groupId>org.apache.spark</groupId>
       <artifactId>spark-sql_${scala.binary.version}</artifactId>
       <version>${spark2.version}</version>
+      <scope>provided</scope>

Review comment:
       Good call

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,45 +92,56 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
-      partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
+
+    val partitionSchema = StructType(Nil)
+    val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+    val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      partitionSchema = partitionSchema,
+      tableSchema = tableSchema,
+      requiredSchema = tableSchema,
+      // This file-reader is used to read base file records, subsequently merging them with the records
+      // stored in delta-log files. As such, we have to read _all_ records from the base file, while avoiding
+      // applying any filtering _before_ we complete combining them w/ delta-log records (to make sure that
+      // we combine them correctly)
       filters = Seq.empty,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      // NOTE: We have to fork the Hadoop Config here as Spark will be modifying it
+      //       to configure Parquet reader appropriately
+      hadoopConf = new Configuration(conf)

Review comment:
       Sorry, not sure that fully understood your point. Previously we're actually copying the Hadoop Conf from Spark  (with `hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()`), but it wasn't clarified what was its purpose and i got burnt by it, so rectifying problem at the root adding the comment here.

##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala
##########
@@ -56,7 +57,7 @@ case class InsertIntoHoodieTableCommand(
   }
 }
 
-object InsertIntoHoodieTableCommand extends Logging {
+object InsertIntoHoodieTableCommand extends Logging with ProvidesHoodieConfig {

Review comment:
       This is simply removing duplication of the identical method building Hudi's write config and re-using the trait

##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala
##########
@@ -75,84 +77,89 @@ class MergeOnReadIncrementalRelation(sqlContext: SQLContext,
 
   private val fileIndex = if (commitsToReturn.isEmpty) List() else buildFileIndex()
 
-  private val preCombineField = {
-    val preCombineFieldFromTableConfig = metaClient.getTableConfig.getPreCombineField
-    if (preCombineFieldFromTableConfig != null) {
-      Some(preCombineFieldFromTableConfig)
-    } else {
-      // get preCombineFiled from the options if this is a old table which have not store
-      // the field to hoodie.properties
-      optParams.get(DataSourceReadOptions.READ_PRE_COMBINE_FIELD.key)
-    }
-  }
+  private val preCombineFieldOpt = getPrecombineFieldProperty
 
-  override def needConversion: Boolean = false
+  // Record filters making sure that only records w/in the requested bounds are being fetched as part of the
+  // scan collected by this relation
+  private lazy val incrementalSpanRecordsFilters: Seq[Filter] = {
+    val isNotNullFilter = IsNotNull(HoodieRecord.COMMIT_TIME_METADATA_FIELD)
+    val largerThanFilter = GreaterThanOrEqual(HoodieRecord.COMMIT_TIME_METADATA_FIELD, commitsToReturn.head.getTimestamp)
+    val lessThanFilter = LessThanOrEqual(HoodieRecord.COMMIT_TIME_METADATA_FIELD, commitsToReturn.last.getTimestamp)
+    Seq(isNotNullFilter, largerThanFilter, lessThanFilter)
+  }
 
-  override def unhandledFilters(filters: Array[Filter]): Array[Filter] = {

Review comment:
       `unhandledFilters` is assumed to return **subset** of the filters that were passed to it as the ones that relation was not able to handle.
   
   In this case Relation was actually returning additional filters that were never passed in -- which is very misleading, since these filters **will not be** applied, but previous implementation actually left such an impression 

##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -133,46 +129,49 @@ object HoodieSparkUtils extends SparkAdapterSupport {
     new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty, fileStatusCache)
   }
 
-  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean, latestTableSchema:
-  org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
-    val dfWriteSchema = AvroConversionUtils.convertStructTypeToAvroSchema(df.schema, structName, recordNamespace)
-    var writeSchema : Schema = null;
-    var toReconcileSchema : Schema = null;
-    if (reconcileToLatestSchema && latestTableSchema.isPresent) {
-      // if reconcileToLatestSchema is set to true and latestSchema is present, then try to leverage latestTableSchema.
-      // this code path will handle situations where records are serialized in odl schema, but callers wish to convert
-      // to Rdd[GenericRecord] using different schema(could be evolved schema or could be latest table schema)
-      writeSchema = dfWriteSchema
-      toReconcileSchema = latestTableSchema.get()
-    } else {
-      // there are paths where callers wish to use latestTableSchema to convert to Rdd[GenericRecords] and not use
-      // row's schema. So use latestTableSchema if present. if not available, fallback to using row's schema.
-      writeSchema = if (latestTableSchema.isPresent) { latestTableSchema.get()} else { dfWriteSchema}
-    }
-    createRddInternal(df, writeSchema, toReconcileSchema, structName, recordNamespace)
+  /**
+   * @deprecated please use other overload [[createRdd]]
+   */
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, reconcileToLatestSchema: Boolean,
+                latestTableSchema: org.apache.hudi.common.util.Option[Schema] = org.apache.hudi.common.util.Option.empty()): RDD[GenericRecord] = {
+    val latestTableSchemaConverted = if (latestTableSchema.isPresent && reconcileToLatestSchema) Some(latestTableSchema.get()) else None
+    createRdd(df, structName, recordNamespace, latestTableSchemaConverted)
   }
 
-  def createRddInternal(df: DataFrame, writeSchema: Schema, latestTableSchema: Schema, structName: String, recordNamespace: String)
-  : RDD[GenericRecord] = {
-    // Use the write avro schema to derive the StructType which has the correct nullability information
-    val writeDataType = AvroConversionUtils.convertAvroSchemaToStructType(writeSchema)
-    val encoder = RowEncoder.apply(writeDataType).resolveAndBind()
-    val deserializer = sparkAdapter.createSparkRowSerDe(encoder)
-    // if records were serialized with old schema, but an evolved schema was passed in with latestTableSchema, we need
-    // latestTableSchema equivalent datatype to be passed in to AvroConversionHelper.createConverterToAvro()
-    val reconciledDataType =
-      if (latestTableSchema != null) AvroConversionUtils.convertAvroSchemaToStructType(latestTableSchema) else writeDataType
-    // Note: deserializer.deserializeRow(row) is not capable of handling evolved schema. i.e. if Row was serialized in
-    // old schema, but deserializer was created with an encoder with evolved schema, deserialization fails.
-    // Hence we always need to deserialize in the same schema as serialized schema.
-    df.queryExecution.toRdd.map(row => deserializer.deserializeRow(row))
-      .mapPartitions { records =>
-        if (records.isEmpty) Iterator.empty
-        else {
-          val convertor = AvroConversionHelper.createConverterToAvro(reconciledDataType, structName, recordNamespace)
-          records.map { x => convertor(x).asInstanceOf[GenericRecord] }
-        }
+  def createRdd(df: DataFrame, structName: String, recordNamespace: String, readerAvroSchemaOpt: Option[Schema]): RDD[GenericRecord] = {
+    val writerSchema = df.schema
+    val writerAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(writerSchema, structName, recordNamespace)
+    val readerAvroSchema = readerAvroSchemaOpt.getOrElse(writerAvroSchema)
+    // We check whether passed in reader schema is identical to writer schema to avoid costly serde loop of
+    // making Spark deserialize its internal representation [[InternalRow]] into [[Row]] for subsequent conversion
+    // (and back)
+    val sameSchema = writerAvroSchema.equals(readerAvroSchema)
+    val (nullable, _) = AvroConversionUtils.resolveAvroTypeNullability(writerAvroSchema)
+
+    // NOTE: We have to serialize Avro schema, and then subsequently parse it on the executor node, since Spark
+    //       serializer is not able to digest it
+    val readerAvroSchemaStr = readerAvroSchema.toString
+    val writerAvroSchemaStr = writerAvroSchema.toString
+    // NOTE: We're accessing toRdd here directly to avoid [[InternalRow]] to [[Row]] conversion
+    df.queryExecution.toRdd.mapPartitions { rows =>
+      if (rows.isEmpty) {
+        Iterator.empty
+      } else {
+        val transform: GenericRecord => GenericRecord =
+          if (sameSchema) identity
+          else {
+            val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
+            rewriteRecord(_, readerAvroSchema)

Review comment:
       So this actually would be faster: 
   
   - Before, we're converting `InternalRow` > `Row` > Avro
   - After, we're doing `InternalRow` > `Avro`, and then rewrite only if schema is actually divergent
   
   So in best case (when there's no schema change) we save one full hierarchical traversal, in the worst case these conversions are identical




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1043907285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * cbabee469b17d9e9386cebc3cd4422fe08c74eac Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042572083


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1042562745


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d5add66be6f81e07eb973ba6cdcf6a8319ad6f24 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081) 
   * 45d5951f4d70d40064ebf3e1b898e85021c6b616 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1045467228


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136) 
   * 1560af3c8a56e345a03800f187d09c0391d9def0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049361602


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

xushiyan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r814457167



##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -41,8 +107,8 @@ object AvroConversionUtils {
         else {
           val schema = new Schema.Parser().parse(schemaStr)
           val dataType = convertAvroSchemaToStructType(schema)
-          val convertor = AvroConversionHelper.createConverterToRow(schema, dataType)
-          records.map { x => convertor(x).asInstanceOf[Row] }
+          val converter = createConverterToRow(schema, dataType)

Review comment:
       @yihua i think even in row writing path, we still need to use `InternalRow` quite a bit, to achieve some data manipulation. Sorry i made the discussion way out of scope from here. iceberg made a similar extension to InternalRow for writing https://github.com/apache/iceberg/blob/24896d84c5/spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/StructInternalRow.java
   cc @vinothchandar @alexeykudinkin @nsivabalan 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1049361602


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3b96c428e1a09b3548c1be81517e644cedfaf14d",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6064",
       "triggerID" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "triggerType" : "PUSH"
     }, {
       "hash" : "429b48e23c06020162ea3d13d0e0bbdec435c793",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6066",
       "triggerID" : "1042301794",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6076",
       "triggerID" : "7bfc5b4ee9e308574a93f0f6d9fd61df5cf09e0e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6081",
       "triggerID" : "d5add66be6f81e07eb973ba6cdcf6a8319ad6f24",
       "triggerType" : "PUSH"
     }, {
       "hash" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6083",
       "triggerID" : "45d5951f4d70d40064ebf3e1b898e85021c6b616",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1207f3bf3fcf3d87daaecce2bd3a5343e31f053",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6097dec23eac5753a517e6096dea546fc462d092",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6108",
       "triggerID" : "6097dec23eac5753a517e6096dea546fc462d092",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6111",
       "triggerID" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbabee469b17d9e9386cebc3cd4422fe08c74eac",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6116",
       "triggerID" : "1044052916",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "6b6a2dd4791210bbd112187797c4349723100365",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6b6a2dd4791210bbd112187797c4349723100365",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6136",
       "triggerID" : "f94a9da52e7b0bacd0b9d0bb8d2c8155cc352290",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6138",
       "triggerID" : "1560af3c8a56e345a03800f187d09c0391d9def0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6246",
       "triggerID" : "4a83f9c00387b8cd06a2c9d864fa065ffde0ee3b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249",
       "triggerID" : "f61e13ccf025b19ff3b1a3771776acc6fdf6914b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 3b96c428e1a09b3548c1be81517e644cedfaf14d UNKNOWN
   * d1207f3bf3fcf3d87daaecce2bd3a5343e31f053 UNKNOWN
   * 6b6a2dd4791210bbd112187797c4349723100365 UNKNOWN
   * f61e13ccf025b19ff3b1a3771776acc6fdf6914b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6249) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on a change in pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

nsivabalan commented on a change in pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#discussion_r812940541



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##########
@@ -96,45 +92,56 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
     log.debug(s" buildScan requiredColumns = ${requiredColumns.mkString(",")}")
     log.debug(s" buildScan filters = ${filters.mkString(",")}")
 
+    // NOTE: In case list of requested columns doesn't contain the Primary Key one, we
+    //       have to add it explicitly so that
+    //          - Merging could be performed correctly
+    //          - In case 0 columns are to be fetched (for ex, when doing {@code count()} on Spark's [[Dataset]],
+    //          Spark still fetches all the rows to execute the query correctly
+    //
+    //       It's okay to return columns that have not been requested by the caller, as those nevertheless will be
+    //       filtered out upstream
+    val fetchedColumns: Array[String] = appendMandatoryColumns(requiredColumns)
+
     val (requiredAvroSchema, requiredStructSchema) =
-      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, requiredColumns)
+      HoodieSparkUtils.getRequiredSchema(tableAvroSchema, fetchedColumns)
     val fileIndex = buildFileIndex(filters)
-    val hoodieTableState = HoodieMergeOnReadTableState(
-      tableStructSchema,
-      requiredStructSchema,
-      tableAvroSchema.toString,
-      requiredAvroSchema.toString,
-      fileIndex,
-      preCombineField,
-      recordKeyFieldOpt
-    )
-    val fullSchemaParquetReader = HoodieDataSourceHelper.buildHoodieParquetReader(
-      sparkSession = sqlContext.sparkSession,
-      dataSchema = tableStructSchema,
-      partitionSchema = StructType(Nil),
-      requiredSchema = tableStructSchema,
+
+    val partitionSchema = StructType(Nil)
+    val tableSchema = HoodieTableSchema(tableStructSchema, tableAvroSchema.toString)
+    val requiredSchema = HoodieTableSchema(requiredStructSchema, requiredAvroSchema.toString)
+
+    val fullSchemaParquetReader = createBaseFileReader(
+      spark = sqlContext.sparkSession,
+      partitionSchema = partitionSchema,
+      tableSchema = tableSchema,
+      requiredSchema = tableSchema,
+      // This file-reader is used to read base file records, subsequently merging them with the records
+      // stored in delta-log files. As such, we have to read _all_ records from the base file, while avoiding
+      // applying any filtering _before_ we complete combining them w/ delta-log records (to make sure that
+      // we combine them correctly)
       filters = Seq.empty,
       options = optParams,
-      hadoopConf = sqlContext.sparkSession.sessionState.newHadoopConf()
+      // NOTE: We have to fork the Hadoop Config here as Spark will be modifying it
+      //       to configure Parquet reader appropriately
+      hadoopConf = new Configuration(conf)

Review comment:
       I get it. I mistook that sqlContext.sparkSession.sessionState.newHadoopConf() returns an empty Configuration. but guess its returning a copy so that any modification will not affect the original hadoop conf. 
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1035845182


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036952896


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933) 
   * 8c5553730559d2ee8a643042660e1750d32e3440 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot commented on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1036947027


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929) 
   * 80aa7f9caceefe6f268d1e47130eb5b60b4d46e7 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039547627


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 9279fc266ad56f47dcffbd6f56b75a6072f21f35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941) 
   * edcd818e555ce207bdebe5f52ee6693bf8abf8f4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1039682913


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 68c672a1ef959ed415f863ddee7aad31373d4758 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4789: [HUDI-1296] Support Metadata Table in Spark Datasource

Posted by GitBox <gi...@apache.org>.

hudi-bot removed a comment on pull request #4789:
URL: https://github.com/apache/hudi/pull/4789#issuecomment-1041224488


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5884",
       "triggerID" : "bef62bd803d48f37b418f6bd7b94b2c6f3e51a46",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5888",
       "triggerID" : "4faa17fccc26f7bec81f6322a102757473a1a3b6",
       "triggerType" : "PUSH"
     }, {
       "hash" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5893",
       "triggerID" : "d9e7f2fda99bb1a0c0deff6f6478f8e95ee8e098",
       "triggerType" : "PUSH"
     }, {
       "hash" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5921",
       "triggerID" : "87d8557c5495870e638866422e7b0cfc42befb15",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f58c2c79cc940081073ae0314229968741552938",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5925",
       "triggerID" : "f58c2c79cc940081073ae0314229968741552938",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5927",
       "triggerID" : "ca8e099133359ee1393607c02f12bea150fdcd62",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5929",
       "triggerID" : "e7dda833ea10ce8bb0d4cc13e1e9c3ccf230565c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5933",
       "triggerID" : "80aa7f9caceefe6f268d1e47130eb5b60b4d46e7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5939",
       "triggerID" : "8c5553730559d2ee8a643042660e1750d32e3440",
       "triggerType" : "PUSH"
     }, {
       "hash" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5941",
       "triggerID" : "9279fc266ad56f47dcffbd6f56b75a6072f21f35",
       "triggerType" : "PUSH"
     }, {
       "hash" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6002",
       "triggerID" : "edcd818e555ce207bdebe5f52ee6693bf8abf8f4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6004",
       "triggerID" : "68c672a1ef959ed415f863ddee7aad31373d4758",
       "triggerType" : "PUSH"
     }, {
       "hash" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "956c9e9568acb6efe8b77d5715cbafefcd8656eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038",
       "triggerID" : "6a6f03bd100becf8c1decf4aad72445f8b03e01b",
       "triggerType" : "PUSH"
     }, {
       "hash" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054",
       "triggerID" : "db5b6f3c44046131647a6b3eb631f3c73ae6eeb1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 740ea9d79bd54d02a869c8f27eb06c9ea54d7fa5 UNKNOWN
   * 956c9e9568acb6efe8b77d5715cbafefcd8656eb UNKNOWN
   * 6a6f03bd100becf8c1decf4aad72445f8b03e01b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6038) 
   * db5b6f3c44046131647a6b3eb631f3c73ae6eeb1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6054) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org