You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/08/14 21:41:40 UTC
[GitHub] [hudi] satishkotha opened a new pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
satishkotha opened a new pull request #3475:
URL: https://github.com/apache/hudi/pull/3475
## What is the purpose of the pull request
Add range index support with metadata table
## Brief change log
* This is a WIP PR primarily used for data collection.
* Lot of fundamental metadata table bottlenecks have to be resolved. For example, bootstrapping rangeIndex for 30 partitions of large table (4000 columns) runs into memory issues and driver result too large errors. We also need to implement async metadata table design in V2
* Tried to implement primitive bucketing by column name to improve index locality
* Only implemented commit integration. Clean and other action integration needs to be added.
## Verify this pull request
* Verified end-to-end by bootstrapping range index on few partitions of large table (4000 columns)
*
* Unit Tests TBD
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e685a5227e042bae771da874627278f5cc549b06 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-961588176
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-961588176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e685a5227e042bae771da874627278f5cc549b06 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754)
* e41d68d93250ce90b76440ffb9bb1e4019852bae Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e685a5227e042bae771da874627278f5cc549b06 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e685a5227e042bae771da874627278f5cc549b06 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754)
* e41d68d93250ce90b76440ffb9bb1e4019852bae UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e41d68d93250ce90b76440ffb9bb1e4019852bae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763)
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#discussion_r734976136
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##########
@@ -671,7 +671,7 @@ public String getBaseFileExtension() {
}
public boolean requireSortedRecords() {
- return getBaseFileFormat() == HoodieFileFormat.HFILE;
+ return getBaseFileFormat() == HoodieFileFormat.HFILE || config.requiresSorting();
Review comment:
@satishkotha : may I know why we need this extra condition. base file is always going to be HFile for metadata table right? Or do we have plans for any other partition where base file will not be Hfile in metadata table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-1019650737
closing this as have another PR https://github.com/apache/hudi/pull/4352
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e685a5227e042bae771da874627278f5cc549b06 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] satishkotha commented on a change in pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
satishkotha commented on a change in pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#discussion_r744391803
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##########
@@ -671,7 +671,7 @@ public String getBaseFileExtension() {
}
public boolean requireSortedRecords() {
- return getBaseFileFormat() == HoodieFileFormat.HFILE;
+ return getBaseFileFormat() == HoodieFileFormat.HFILE || config.requiresSorting();
Review comment:
@nsivabalan We were initially considering parquet file format as an option of range index. So I added it as a config to support sorting on parquet files in metadata table. Now that we have finalized HFile format (at least for short term), we can probably remove config. Sorry for delay, I missed your mention. Feel free to ping me on slack if you have any followups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan closed pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
nsivabalan closed pull request #3475:
URL: https://github.com/apache/hudi/pull/3475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e41d68d93250ce90b76440ffb9bb1e4019852bae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3475: [HUDI-1822][RFC-27][WIP] range index support with metadata table
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3475:
URL: https://github.com/apache/hudi/pull/3475#issuecomment-898962060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e685a5227e042bae771da874627278f5cc549b06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754",
"triggerID" : "e685a5227e042bae771da874627278f5cc549b06",
"triggerType" : "PUSH"
}, {
"hash" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763",
"triggerID" : "e41d68d93250ce90b76440ffb9bb1e4019852bae",
"triggerType" : "PUSH"
}, {
"hash" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804",
"triggerID" : "75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e41d68d93250ce90b76440ffb9bb1e4019852bae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1763)
* 75b5bc5d06bd53f449fe9b244f7d56b5fe5a2848 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org