You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/24 09:46:38 UTC

[GitHub] [hudi] cxzl25 opened a new pull request, #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

cxzl25 opened a new pull request, #5959:
URL: https://github.com/apache/hudi/pull/5959

   ## What is the purpose of the pull request
   
   Several PRs have supported using the configuration `hoodie.common.spillable.diskmap.type` to decide which diskmap to use before constructing the `HoodieMergedLogRecordScanner`.
   But there are several places where such a configuration is not supported.
   
   https://issues.apache.org/jira/browse/HUDI-2044
   
   https://issues.apache.org/jira/browse/HUDI-4151
   
   ## Brief change log
   
     - *Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner*
   
   ## Verify this pull request
   
     - *Manually verified the change by running a job locally.*
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1166582862

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0ff436d4c402242b3c85456c271d81b9a5862c33 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518) 
   * b510e7959c62a7e1d88b4ef88d43a80cca4afd49 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1166149267

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0ff436d4c402242b3c85456c271d81b9a5862c33 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1165809742

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * a9eb2c4c516481a7a26719377f02634006295b92 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505) 
   * 0ff436d4c402242b3c85456c271d81b9a5862c33 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1165442790

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * a9eb2c4c516481a7a26719377f02634006295b92 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] danny0405 merged pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
danny0405 merged PR #5959:
URL: https://github.com/apache/hudi/pull/5959


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1165812834

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * a9eb2c4c516481a7a26719377f02634006295b92 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505) 
   * 0ff436d4c402242b3c85456c271d81b9a5862c33 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1166641229

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9558",
       "triggerID" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * b510e7959c62a7e1d88b4ef88d43a80cca4afd49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9558) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1165704354

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * a9eb2c4c516481a7a26719377f02634006295b92 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1166583977

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9505",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518",
       "triggerID" : "0ff436d4c402242b3c85456c271d81b9a5862c33",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9558",
       "triggerID" : "b510e7959c62a7e1d88b4ef88d43a80cca4afd49",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0ff436d4c402242b3c85456c271d81b9a5862c33 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9518) 
   * b510e7959c62a7e1d88b4ef88d43a80cca4afd49 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9558) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #5959: [HUDI-4316] Support for spillable diskmap configuration when constructing HoodieMergedLogRecordScanner

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5959:
URL: https://github.com/apache/hudi/pull/5959#issuecomment-1165408076

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "a9eb2c4c516481a7a26719377f02634006295b92",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * a9eb2c4c516481a7a26719377f02634006295b92 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org