You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/08/09 08:27:16 UTC
[GitHub] [incubator-uniffle] izchen opened a new pull request, #149: Tagged version
izchen opened a new pull request, #149:
URL: https://github.com/apache/incubator-uniffle/pull/149
### What changes were proposed in this pull request?
Tag Spark and Hadoop version in release package.
1. Add version information to client jar name
2. Add version information in RELEASE file
An example：
```
./build_distribution.sh --spark3-profile 'spark3.0' --spark3-mvn '-Dx.x1=1 -Dspark.version=3.0.2' -Dhadoop.version=2.8.0 -Dx.x2=2
```
Before PR：
```
$ tree jars/client
jars/client
├── mr
│ └── rss-client-mr-0.6.0-snapshot-shaded.jar
├── spark2
│ └── rss-client-spark2-0.6.0-snapshot-shaded.jar
└── spark3
└── rss-client-spark3-0.6.0-snapshot-shaded.jar
$ cat RELEASE
RSS 0.6.0-snapshot (git revision 04cbdbb) built
Build flags: -Dhadoop.version=2.8.0 -Dx.x3=3
```
After PR：
```
$ tree jars/client
jars/client
├── mr
│ └── rss-client-mr2.8.0-0.6.0-snapshot-shaded.jar
├── spark2
│ └── rss-client-spark2.4.6-0.6.0-snapshot-shaded.jar
└── spark3
└── rss-client-spark3.0.2-0.6.0-snapshot-shaded.jar
$ cat RELEASE
RSS 0.6.0-snapshot (git revision 27032c1) built for Hadoop 2.8.0 Spark2 2.4.6 Spark3 3.0.2
Build flags: --spark2-profile 'spark2' --spark2-mvn '' --spark3-profile 'spark3.0' --spark3-mvn '-Dx.x1=1 -Dspark.version=3.0.2' -Dhadoop.version=2.8.0 -Dx.x2=2
```
### Why are the changes needed?
Unable to determine spark and hadoop version of release package, which may cause confusion in usage.
### Does this PR introduce _any_ user-facing change?
Yes.
Changed client jar name.
### How was this patch tested?
local test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi merged pull request #149: [MINOR] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
jerqi merged PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #149: [MINOR][IMPROVEMENT] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1209097075
# [Codecov](https://codecov.io/gh/apache/incubator-uniffle/pull/149?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#149](https://codecov.io/gh/apache/incubator-uniffle/pull/149?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f56cc06) into [master](https://codecov.io/gh/apache/incubator-uniffle/commit/04cbdbb6eec98cdd02fcc770e0cf1298db281c4f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (04cbdbb) will **decrease** coverage by `0.04%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #149 +/- ##
============================================
- Coverage 57.20% 57.15% -0.05%
+ Complexity 1200 1199 -1
============================================
Files 150 150
Lines 8185 8185
Branches 773 773
============================================
- Hits 4682 4678 -4
- Misses 3257 3259 +2
- Partials 246 248 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-uniffle/pull/149?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/uniffle/server/ShuffleFlushManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/149/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9TaHVmZmxlRmx1c2hNYW5hZ2VyLmphdmE=) | `76.83% <0.00%> (-1.70%)` | :arrow_down: |
| [...pache/hadoop/mapreduce/task/reduce/RssFetcher.java](https://codecov.io/gh/apache/incubator-uniffle/pull/149/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL3Rhc2svcmVkdWNlL1Jzc0ZldGNoZXIuamF2YQ==) | `89.06% <0.00%> (-1.57%)` | :arrow_down: |
| [...e/uniffle/server/storage/SingleStorageManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/149/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9zdG9yYWdlL1NpbmdsZVN0b3JhZ2VNYW5hZ2VyLmphdmE=) | `67.21% <0.00%> (+1.63%)` | :arrow_up: |
:mega: Codecov can now indicate which changes are the most critical in Pull Requests. [Learn more](https://about.codecov.io/product/feature/runtime-insights/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] izchen commented on pull request #149: [MINOR][IMPROVEMENT] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
izchen commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1210202431
Removed client name changes. only changed the RELEASE file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #149: [MINOR][IMPROVEMENT] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
jerqi commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1209304088
cc @colinmjj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #149: [MINOR] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
jerqi commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1210417630
I verify this patch by hand. It's ok, thanks @izchen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] colinmjj commented on pull request #149: [MINOR][IMPROVEMENT] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
colinmjj commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1210108182
@izchen When package with profile "Spark3.0", "Spark3.2", the result client jar can be used in both spark3.0, spark3.2.
I prefer not to add spark version in client jar if the jar is worked for all spark 3.x.
If the above rule is changed, for example, the result client jar is not compatible between spark3.2 and spark 3.5, I agree update the name of client jar at that time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] colinmjj commented on pull request #149: [MINOR] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
colinmjj commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1210308426
Agree to add package info into RELEASE file, +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] jerqi commented on pull request #149: [MINOR][IMPROVEMENT] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
jerqi commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1209317967
I search the Spark project. Spark use xxx_scala_2.1.1-3.2.1xxx as version number. Should we imitate Spark to use `rss-client-spark_3.0.2-0.6.0-snapshot-shaded.jar`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] izchen commented on pull request #149: [MINOR] Tagged spark hadoop version in release package
Posted by GitBox <gi...@apache.org>.
izchen commented on PR #149:
URL: https://github.com/apache/incubator-uniffle/pull/149#issuecomment-1210203353
Thanks for your review, @jerqi @colinmjj !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org