You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2022/08/24 08:13:55 UTC
[GitHub] [bigtop] kevinw66 opened a new pull request, #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
kevinw66 opened a new pull request, #986:
URL: https://github.com/apache/bigtop/pull/986
…ion path
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
- [ ] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
sekikn commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1230469802
I also ran `./gradlew allclean zookeeper-pkg hive-pkg kafka-pkg -Pprefix=/usr/bigtop`, and confirmed that the files under /usr were relocated to `$prefix/$bigtop_version/usr`, and the ones under /etc and /var were not.
It almost looks good, but is it OK that /etc/(zookeeper|hive|kafka) aren't relocated, because configuration files can be incompatible between different versions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1230477498
`install-hive.sh` is outdated for build hive-*.rpm, since `log4j` is replaced with `log4j2`.
The difference in other rpm package might be a bug, let me check on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1232643708
Pull requests will be created for each component for easy review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1225355072
There is only prototype for hive-rpm right now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1230482603
> I also ran `./gradlew allclean zookeeper-pkg hive-pkg kafka-pkg -Pprefix=/usr/bigtop`, and confirmed that the files under /usr were relocated to `$prefix/$bigtop_version/usr`, and the ones under /etc and /var were not. It almost looks good, but is it OK that /etc/(zookeeper|hive|kafka) aren't relocated, because configuration files can be incompatible between different versions?
I'm thinking we should create symlink from `alternatives confdir` to `/usr/lib/%{name}/conf`, it'll be easier for components' upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
sekikn commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1230433161
Thank you so much for addressing this, @kevinw66!
As the first step, I tried to build ZK, Hive and Kafka without the "prefix" option for regression test, and found a few differences between the original and patched versions in zookeeper-debuginfo-\*.rpm, hive-\*.rpm, and hive-hcatalog-\*.rpm. Are they ignorable?
```
$ ./gradlew allclean zookeeper-pkg hive-pkg kafka-pkg
...
BUILD SUCCESSFUL in 10m 10s
61 actionable tasks: 60 executed, 1 up-to-date
$ mv output /tmp
$ curl -sL https://github.com/apache/bigtop/pull/986.patch | git apply
$ ./gradlew allclean zookeeper-pkg hive-pkg kafka-pkg
...
BUILD SUCCESSFUL in 10m 31s
61 actionable tasks: 61 executed
$ for i in $(ls output/zookeeper/x86_64/zookeeper-*.rpm output/hive/noarch/hive-*.rpm output/kafka/noarch/kafka-*.rpm); do diff <(rpm2cpio $i | cpio -t) <(rpm2cpio /tmp/$i | cpio -t); done
...
5,6c5,6
< ./etc/hive/conf.dist/hive-exec-log4j2.properties
< ./etc/hive/conf.dist/hive-log4j2.properties
---
> ./etc/hive/conf.dist/hive-exec-log4j2.properties.template
> ./etc/hive/conf.dist/hive-log4j2.properties.template
...
19a20
> ./usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar
20a22
> ./usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter.jar
21a24
> ./usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-server-extensions.jar
22a26
> ./usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-streaming.jar
...
3,5c3,8
< ./usr/lib/debug/.build-id/25
< ./usr/lib/debug/.build-id/25/99a4194db78c707491676ca8a885aaa4b4847b
< ./usr/lib/debug/.build-id/25/99a4194db78c707491676ca8a885aaa4b4847b.debug
---
> ./usr/lib/debug/.build-id/1c
> ./usr/lib/debug/.build-id/1c/2a4bc08aa51c0a0e7046efe024bb300adf178c
> ./usr/lib/debug/.build-id/1c/2a4bc08aa51c0a0e7046efe024bb300adf178c.debug
> ./usr/lib/debug/.build-id/51
> ./usr/lib/debug/.build-id/51/c986cae6ab713722a029b7ac694cfe8d69d4eb
> ./usr/lib/debug/.build-id/51/c986cae6ab713722a029b7ac694cfe8d69d4eb.debug
11,16c14,16
< ./usr/lib/debug/.build-id/b6
< ./usr/lib/debug/.build-id/b6/cfa769954d585b08ae64bd5c63e4c35e01289e
< ./usr/lib/debug/.build-id/b6/cfa769954d585b08ae64bd5c63e4c35e01289e.debug
< ./usr/lib/debug/.build-id/d0
< ./usr/lib/debug/.build-id/d0/955dae110a27a1c642f8a10ef92741bc9d7141
< ./usr/lib/debug/.build-id/d0/955dae110a27a1c642f8a10ef92741bc9d7141.debug
---
> ./usr/lib/debug/.build-id/d8
> ./usr/lib/debug/.build-id/d8/379b9351c595c39bfc3d00613cfbdf84650222
> ./usr/lib/debug/.build-id/d8/379b9351c595c39bfc3d00613cfbdf84650222.debug
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1225357232
@guyuqi @iwasakims @sekikn Could you give some advice? After that I'll start working on the rest of the components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1231188134
> `install-hive.sh` is outdated for build hive-*.rpm, since `log4j` is replaced with `log4j2`.
Let me create a separate PR for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 closed pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 closed pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
URL: https://github.com/apache/bigtop/pull/986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1231552169
> > I also ran `./gradlew allclean zookeeper-pkg hive-pkg kafka-pkg -Pprefix=/usr/bigtop`, and confirmed that the files under /usr were relocated to `$prefix/$bigtop_version/usr`, and the ones under /etc and /var were not. It almost looks good, but is it OK that /etc/(zookeeper|hive|kafka) aren't relocated, because configuration files can be incompatible between different versions?
>
> `/var/log`,`/var/run`,`/etc/%{name}/conf` won't be relocated to `/usr/bigtop/${version}/`, but `/var/lib` will.
>
> I'm thinking we should create symlink from `alternatives confdir` to `/usr/lib/%{name}/conf`, it'll be easier for components' upgrade.
I think I can relocated all dir to `/usr/bigtop/${version}/`, and create symlink for `/var/log`,`/var/run`,`/etc/%{name}/conf` in bigtop-select in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] kevinw66 commented on pull request #986: [WIP]BIGTOP-3785: Improve the build scripts to configure package installation path
Posted by GitBox <gi...@apache.org>.
kevinw66 commented on PR #986:
URL: https://github.com/apache/bigtop/pull/986#issuecomment-1230535824
@sekikn Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org