You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/04/12 14:22:24 UTC

[GitHub] [incubator-hudi] liujianhuiouc opened a new pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

liujianhuiouc opened a new pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509
 
 
   
   add insert info with insert records num in the HoodieCommitMeta.
   because of the file id is unknown, set it as empty string
   
   testing result:
   
   
     "partitionToWriteStats" : {
       "americas/brazil/sao_paulo" : [ {
         "fileId" : "",
         "path" : null,
         "prevCommit" : null,
         "numWrites" : 0,
         "numDeletes" : 0,
         "numUpdateWrites" : 0,
         "numInserts" : 3,
         "totalWriteBytes" : 0,
         "totalWriteErrors" : 0,
         "tempPath" : null,
         "partitionPath" : null,
         "totalLogRecords" : 0,
         "totalLogFilesCompacted" : 0,
         "totalLogSizeCompacted" : 0,
         "totalUpdatedRecordsCompacted" : 0,
         "totalLogBlocks" : 0,
         "totalCorruptLogBlock" : 0,
         "totalRollbackBlocks" : 0,
         "fileSizeInBytes" : 0
       } ],

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] vinothchandar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614376368
 
 
   @n3nash can you actually help review this.. does this pose any issues for rollback today? (IIUC for rollback of inserts, we will list and delete files written as of that delta commit instant.. ) delta commit ultimately will have insert info... So really, this is just completeness fix with no real impact on operations we do? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612738631
 
 
   Hi @liujianhuiouc, thanks for your contribution, your email may be wrong, check it `git config --list | grep email`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-613353236
 
 
   hi, you can use following git operations, try again
   ```
   git reset --hard 4fb34d918dc23c18ba67801121d0a5bee6512843
   git pull --rebase origin master
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hudi] n3nash merged pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
n3nash merged pull request #1509:
URL: https://github.com/apache/hudi/pull/1509


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612771752
 
 
   thank you for you reminding, it's my company email, i have set it with personal mailbox 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] codecov-io commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612629725
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=h1) Report
   > Merging [#1509](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/5d717a28f45137bea71dffa31b0ae7ccbf1bda00&el=desc) will **decrease** coverage by `0.02%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-hudi/pull/1509/graphs/tree.svg?width=650&height=150&src=pr&token=VTTXabwbs2)](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1509      +/-   ##
   ============================================
   - Coverage     72.23%   72.20%   -0.03%     
     Complexity      289      289              
   ============================================
     Files           338      338              
     Lines         15947    15952       +5     
     Branches       1624     1624              
   ============================================
   - Hits          11519    11518       -1     
   - Misses         3700     3707       +7     
   + Partials        728      727       -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...java/org/apache/hudi/client/HoodieWriteClient.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVdyaXRlQ2xpZW50LmphdmE=) | `69.56% <100.00%> (+0.33%)` | `0.00 <0.00> (ø)` | |
   | [...g/apache/hudi/metrics/InMemoryMetricsReporter.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9Jbk1lbW9yeU1ldHJpY3NSZXBvcnRlci5qYXZh) | `40.00% <0.00%> (-40.00%)` | `0.00% <0.00%> (ø%)` | |
   | [...src/main/java/org/apache/hudi/metrics/Metrics.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9NZXRyaWNzLmphdmE=) | `58.33% <0.00%> (-13.89%)` | `0.00% <0.00%> (ø%)` | |
   | [.../org/apache/hudi/table/HoodieMergeOnReadTable.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvSG9vZGllTWVyZ2VPblJlYWRUYWJsZS5qYXZh) | `83.75% <0.00%> (+0.62%)` | `0.00% <0.00%> (ø%)` | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=footer). Last update [5d717a2...4fb34d9](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612734787
 
 
   @n3nash help to merge that

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612629725
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=h1) Report
   > Merging [#1509](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/644c1cc8bd1965271c4433edccb17aa8fda5f403&el=desc) will **increase** coverage by `0.03%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-hudi/pull/1509/graphs/tree.svg?width=650&height=150&src=pr&token=VTTXabwbs2)](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1509      +/-   ##
   ============================================
   + Coverage     72.17%   72.21%   +0.03%     
     Complexity      289      289              
   ============================================
     Files           372      372              
     Lines         16267    16272       +5     
     Branches       1637     1637              
   ============================================
   + Hits          11741    11751      +10     
   + Misses         3791     3787       -4     
   + Partials        735      734       -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../table/action/commit/BaseCommitActionExecutor.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL2NvbW1pdC9CYXNlQ29tbWl0QWN0aW9uRXhlY3V0b3IuamF2YQ==) | `85.47% <100.00%> (+0.64%)` | `0.00 <0.00> (ø)` | |
   | [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `79.31% <0.00%> (-10.35%)` | `0.00% <0.00%> (ø%)` | |
   | [...on/rollback/MergeOnReadRollbackActionExecutor.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL3JvbGxiYWNrL01lcmdlT25SZWFkUm9sbGJhY2tBY3Rpb25FeGVjdXRvci5qYXZh) | `86.11% <0.00%> (+1.38%)` | `0.00% <0.00%> (ø%)` | |
   | [...src/main/java/org/apache/hudi/metrics/Metrics.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9NZXRyaWNzLmphdmE=) | `72.22% <0.00%> (+13.88%)` | `0.00% <0.00%> (ø%)` | |
   | [...g/apache/hudi/metrics/InMemoryMetricsReporter.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9Jbk1lbW9yeU1ldHJpY3NSZXBvcnRlci5qYXZh) | `80.00% <0.00%> (+40.00%)` | `0.00% <0.00%> (ø%)` | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=footer). Last update [644c1cc...5f9c174](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614374608
 
 
   @lamber-ken please help to merge

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
lamber-ken commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-613186819
 
 
   LGTM, can you please rebase and resolve the conflicts?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-613450174
 
 
   done, thank you

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hudi] leesf commented on pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
leesf commented on pull request #1509:
URL: https://github.com/apache/hudi/pull/1509#issuecomment-652781339


   @n3nash @bvaradar kindly remind to land this PR?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-613326434
 
 
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612629725
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=h1) Report
   > Merging [#1509](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/5d717a28f45137bea71dffa31b0ae7ccbf1bda00&el=desc) will **decrease** coverage by `0.02%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-hudi/pull/1509/graphs/tree.svg?width=650&height=150&src=pr&token=VTTXabwbs2)](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1509      +/-   ##
   ============================================
   - Coverage     72.23%   72.20%   -0.03%     
     Complexity      289      289              
   ============================================
     Files           338      338              
     Lines         15947    15952       +5     
     Branches       1624     1624              
   ============================================
   - Hits          11519    11518       -1     
   - Misses         3700     3707       +7     
   + Partials        728      727       -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...java/org/apache/hudi/client/HoodieWriteClient.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVdyaXRlQ2xpZW50LmphdmE=) | `69.56% <100.00%> (+0.33%)` | `0.00 <0.00> (ø)` | |
   | [...g/apache/hudi/metrics/InMemoryMetricsReporter.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9Jbk1lbW9yeU1ldHJpY3NSZXBvcnRlci5qYXZh) | `40.00% <0.00%> (-40.00%)` | `0.00% <0.00%> (ø%)` | |
   | [...src/main/java/org/apache/hudi/metrics/Metrics.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0cmljcy9NZXRyaWNzLmphdmE=) | `58.33% <0.00%> (-13.89%)` | `0.00% <0.00%> (ø%)` | |
   | [.../org/apache/hudi/table/HoodieMergeOnReadTable.java](https://codecov.io/gh/apache/incubator-hudi/pull/1509/diff?src=pr&el=tree#diff-aHVkaS1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvSG9vZGllTWVyZ2VPblJlYWRUYWJsZS5qYXZh) | `83.75% <0.00%> (+0.62%)` | `0.00% <0.00%> (ø%)` | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=footer). Last update [5d717a2...4fb34d9](https://codecov.io/gh/apache/incubator-hudi/pull/1509?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hudi] n3nash commented on pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
n3nash commented on pull request #1509:
URL: https://github.com/apache/hudi/pull/1509#issuecomment-668903770


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] bvaradar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-614381927
 
 
   @vinothchandar This fix should reflect only in inflight commit metadata and rollback works in the way you mentioned. As it stands now, the current rollback logic can benefit with this change. We can optimize the rollback logic to only look at the partitions in the inflight file for rollbacks instead of scanning all partitions. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-612623104
 
 
   the origin pull is https://github.com/apache/incubator-hudi/pull/1216

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] n3nash commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
n3nash commented on issue #1509:
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-617372917


   LGTM, thanks @liujianhuiouc for the contribution and persistence on this issue :) @vinothchandar yes, this does not change existing functionality but just add more information, as @bvaradar pointed out, can be used to optimize rollback code


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] n3nash edited a comment on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
n3nash edited a comment on issue #1509:
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-617372917


   LGTM, thanks @liujianhuiouc for the contribution and persistence on this issue. Can you please merge your 5 commits into 1 and I'll merge the diff ? :) @vinothchandar yes, this does not change existing functionality but just add more information, as @bvaradar pointed out, can be used to optimize rollback code


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] liujianhuiouc commented on issue #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
liujianhuiouc commented on issue #1509:
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-617524571


   squash the commits @n3nash 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-hudi] vinothchandar commented on pull request #1509: [HUDI-525] lack of insert info in delta_commit inflight

Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #1509:
URL: https://github.com/apache/incubator-hudi/pull/1509#issuecomment-628902977


   @n3nash is this ready to land 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org