You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/05/11 00:09:22 UTC
[GitHub] [incubator-hudi] garyli1019 opened a new pull request #1612: HUDI-528 Handle empty commit in incremental pulling
garyli1019 opened a new pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612
## What is the purpose of the pull request
https://issues.apache.org/jira/browse/HUDI-528
## Brief change log
- Avoid loading empty instant in `IncrementalRelation.scala`
## Verify this pull request
This change added tests and can be verified as follows:
- Added an empty data frame test case in TestDataSource
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] bvaradar merged pull request #1612: [HUDI-528] Handle empty commit in incremental pulling
Posted by GitBox <gi...@apache.org>.
bvaradar merged pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1612: [HUDI-528] Handle empty commit in incremental pulling
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612#issuecomment-626417448
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=h1) Report
> Merging [#1612](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/3a2fe13fcb7c168f8ff023e3bdb6ae482b400316&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1612 +/- ##
============================================
+ Coverage 71.77% 71.81% +0.03%
Complexity 1087 1087
============================================
Files 385 385
Lines 16591 16587 -4
Branches 1669 1668 -1
============================================
+ Hits 11909 11912 +3
+ Misses 3953 3949 -4
+ Partials 729 726 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSW5jcmVtZW50YWxSZWxhdGlvbi5zY2FsYQ==) | `72.41% <100.00%> (-0.17%)` | `0.00 <0.00> (ø)` | |
| [...java/org/apache/hudi/common/util/ParquetUtils.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvUGFycXVldFV0aWxzLmphdmE=) | `73.68% <0.00%> (-2.64%)` | `0.00% <0.00%> (ø%)` | |
| [...che/hudi/common/util/BufferedRandomAccessFile.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQnVmZmVyZWRSYW5kb21BY2Nlc3NGaWxlLmphdmE=) | `55.26% <0.00%> (+0.87%)` | `0.00% <0.00%> (ø%)` | |
| [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSG9vZGllU3BhcmtTcWxXcml0ZXIuc2NhbGE=) | `55.08% <0.00%> (+1.79%)` | `0.00% <0.00%> (ø%)` | |
| [...in/scala/org/apache/hudi/AvroConversionUtils.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvQXZyb0NvbnZlcnNpb25VdGlscy5zY2FsYQ==) | `58.33% <0.00%> (+4.16%)` | `0.00% <0.00%> (ø%)` | |
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=footer). Last update [3a2fe13...f1dde61](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io commented on pull request #1612: [HUDI-528] Handle empty commit in incremental pulling
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612#issuecomment-626417448
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=h1) Report
> Merging [#1612](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/fa6aba751d8de16d9d109a8cfc21150b17b59cff&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `90.00%`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1612 +/- ##
============================================
+ Coverage 71.78% 71.81% +0.02%
Complexity 1087 1087
============================================
Files 385 385
Lines 16575 16578 +3
Branches 1668 1669 +1
============================================
+ Hits 11899 11906 +7
+ Misses 3947 3944 -3
+ Partials 729 728 -1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSW5jcmVtZW50YWxSZWxhdGlvbi5zY2FsYQ==) | `72.30% <90.00%> (-0.28%)` | `0.00 <0.00> (ø)` | |
| [...e/hudi/common/table/log/HoodieLogFormatWriter.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRXcml0ZXIuamF2YQ==) | `76.92% <0.00%> (+0.96%)` | `0.00% <0.00%> (ø%)` | |
| [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSG9vZGllU3BhcmtTcWxXcml0ZXIuc2NhbGE=) | `55.15% <0.00%> (+1.81%)` | `0.00% <0.00%> (ø%)` | |
| [...in/scala/org/apache/hudi/AvroConversionUtils.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvQXZyb0NvbnZlcnNpb25VdGlscy5zY2FsYQ==) | `58.33% <0.00%> (+4.16%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=footer). Last update [fa6aba7...de5e4cd](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1612: [HUDI-528] Handle empty commit in incremental pulling
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612#issuecomment-626417448
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=h1) Report
> Merging [#1612](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/3a2fe13fcb7c168f8ff023e3bdb6ae482b400316&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1612 +/- ##
============================================
+ Coverage 71.77% 71.81% +0.03%
Complexity 1087 1087
============================================
Files 385 385
Lines 16591 16587 -4
Branches 1669 1668 -1
============================================
+ Hits 11909 11912 +3
+ Misses 3953 3949 -4
+ Partials 729 726 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSW5jcmVtZW50YWxSZWxhdGlvbi5zY2FsYQ==) | `72.41% <100.00%> (-0.17%)` | `0.00 <0.00> (ø)` | |
| [...java/org/apache/hudi/common/util/ParquetUtils.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvUGFycXVldFV0aWxzLmphdmE=) | `73.68% <0.00%> (-2.64%)` | `0.00% <0.00%> (ø%)` | |
| [...che/hudi/common/util/BufferedRandomAccessFile.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQnVmZmVyZWRSYW5kb21BY2Nlc3NGaWxlLmphdmE=) | `55.26% <0.00%> (+0.87%)` | `0.00% <0.00%> (ø%)` | |
| [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSG9vZGllU3BhcmtTcWxXcml0ZXIuc2NhbGE=) | `55.08% <0.00%> (+1.79%)` | `0.00% <0.00%> (ø%)` | |
| [...in/scala/org/apache/hudi/AvroConversionUtils.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvQXZyb0NvbnZlcnNpb25VdGlscy5zY2FsYQ==) | `58.33% <0.00%> (+4.16%)` | `0.00% <0.00%> (ø%)` | |
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=footer). Last update [3a2fe13...f1dde61](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1612: [HUDI-528] Handle empty commit in incremental pulling
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1612:
URL: https://github.com/apache/incubator-hudi/pull/1612#issuecomment-626417448
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=h1) Report
> Merging [#1612](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/3a2fe13fcb7c168f8ff023e3bdb6ae482b400316&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1612 +/- ##
============================================
+ Coverage 71.77% 71.79% +0.01%
Complexity 1087 1087
============================================
Files 385 385
Lines 16591 16588 -3
Branches 1669 1668 -1
============================================
+ Hits 11909 11910 +1
+ Misses 3953 3952 -1
+ Partials 729 726 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...in/scala/org/apache/hudi/IncrementalRelation.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSW5jcmVtZW50YWxSZWxhdGlvbi5zY2FsYQ==) | `72.88% <100.00%> (+0.30%)` | `0.00 <0.00> (ø)` | |
| [...java/org/apache/hudi/common/util/ParquetUtils.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvUGFycXVldFV0aWxzLmphdmE=) | `73.68% <0.00%> (-2.64%)` | `0.00% <0.00%> (ø%)` | |
| [...che/hudi/common/util/BufferedRandomAccessFile.java](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQnVmZmVyZWRSYW5kb21BY2Nlc3NGaWxlLmphdmE=) | `55.26% <0.00%> (+0.87%)` | `0.00% <0.00%> (ø%)` | |
| [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvSG9vZGllU3BhcmtTcWxXcml0ZXIuc2NhbGE=) | `55.08% <0.00%> (+1.79%)` | `0.00% <0.00%> (ø%)` | |
| [...in/scala/org/apache/hudi/AvroConversionUtils.scala](https://codecov.io/gh/apache/incubator-hudi/pull/1612/diff?src=pr&el=tree#diff-aHVkaS1zcGFyay9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2h1ZGkvQXZyb0NvbnZlcnNpb25VdGlscy5zY2FsYQ==) | `58.33% <0.00%> (+4.16%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=footer). Last update [3a2fe13...f1dde61](https://codecov.io/gh/apache/incubator-hudi/pull/1612?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org