You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/03/28 07:14:58 UTC

[GitHub] [hudi] leo-Iamok opened a new pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

leo-Iamok opened a new pull request #2734:
URL: https://github.com/apache/hudi/pull/2734


   Add "Java" prefix for "UpsertPartitioner" in hudi-java-client to differentiate it from another same fully qualified name "UpsertPartitioner" in hudi-spark-client, otherwise when both modules are included in classpath, the two classes would conflict.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yanghua commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
yanghua commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-809244541


   Hi @garyli1019 I suggest that we include this PR into 0.8.0. Currently, we have three classes with the same full-qualified name(`UpsertPartitioner`) in spark/java/flink module. It is extremely confusing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yanghua commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
yanghua commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-810722159


   @leo-Iamok I have opened a PR #2746 to handle the class in flink module, so you do not need to care about that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] codecov-io commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-808859046


   # [Codecov](https://codecov.io/gh/apache/hudi/pull/2734?src=pr&el=h1) Report
   > Merging [#2734](https://codecov.io/gh/apache/hudi/pull/2734?src=pr&el=desc) (eef9891) into [master](https://codecov.io/gh/apache/hudi/commit/bec70413c0943f38ee5cdf62fa3a79af44d8cded?el=desc) (bec7041) will **decrease** coverage by `42.33%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/hudi/pull/2734/graphs/tree.svg?width=650&height=150&src=pr&token=VTTXabwbs2)](https://codecov.io/gh/apache/hudi/pull/2734?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #2734       +/-   ##
   ============================================
   - Coverage     51.73%   9.40%   -42.34%     
   + Complexity     3607      48     -3559     
   ============================================
     Files           476      54      -422     
     Lines         22614    1989    -20625     
     Branches       2410     236     -2174     
   ============================================
   - Hits          11700     187    -11513     
   + Misses         9892    1789     -8103     
   + Partials       1022      13     -1009     
   ```
   
   | Flag | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | hudicli | `?` | `?` | |
   | hudiclient | `?` | `?` | |
   | hudicommon | `?` | `?` | |
   | hudiflink | `?` | `?` | |
   | hudihadoopmr | `?` | `?` | |
   | hudisparkdatasource | `?` | `?` | |
   | hudisync | `?` | `?` | |
   | huditimelineservice | `?` | `?` | |
   | hudiutilities | `9.40% <ø> (-60.39%)` | `0.00 <ø> (ø)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2734?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-2.00%)` | |
   | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | |
   | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | [...rg/apache/hudi/utilities/sources/CsvDFSSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQ3N2REZTU291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-10.00%)` | |
   | [...g/apache/hudi/utilities/sources/JsonDFSSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvbkRGU1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | [...apache/hudi/utilities/sources/JsonKafkaSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvbkthZmthU291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-6.00%)` | |
   | [...pache/hudi/utilities/sources/ParquetDFSSource.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUGFycXVldERGU1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-5.00%)` | |
   | [...lities/schema/SchemaProviderWithPostProcessor.java](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFQcm92aWRlcldpdGhQb3N0UHJvY2Vzc29yLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | ... and [453 more](https://codecov.io/gh/apache/hudi/pull/2734/diff?src=pr&el=tree-more) | |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] garyli1019 commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
garyli1019 commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-809308540


   > Hi @garyli1019 I suggest that we include this PR into 0.8.0. Currently, we have three classes with the same full-qualified name(`UpsertPartitioner`) in spark/java/flink module. It is extremely confusing.
   
   Hi @yanghua , we have almost done all the pre-release verification work. If adding more commits, we need to start over. So let's keep this for the next release cuz this doesn't look like a blocker.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yanghua commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
yanghua commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-809224318


   @leo-Iamok Thanks for your report. Would you also please consider flink module? cc @danny0405 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] danny0405 commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
danny0405 commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-810073379


   `UpsertPartitioner` is useless in Flink now, just remove it!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yanghua merged pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
yanghua merged pull request #2734:
URL: https://github.com/apache/hudi/pull/2734


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yanghua commented on pull request #2734: [HUDI-1731] Rename UpsertPartitioner in both hudi-java-client and hud…

Posted by GitBox <gi...@apache.org>.
yanghua commented on pull request #2734:
URL: https://github.com/apache/hudi/pull/2734#issuecomment-809311188


   > > Hi @garyli1019 I suggest that we include this PR into 0.8.0. Currently, we have three classes with the same full-qualified name(`UpsertPartitioner`) in spark/java/flink module. It is extremely confusing.
   > 
   > Hi @yanghua , we have almost done all the pre-release verification work. If adding more commits, we need to start over. So let's keep this for the next release cuz this doesn't look like a blocker.
   
   OK, it will cause an error when users load multiple `UpsertPartitioner`s. As an RM, you have the right to make the decision.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org