You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Sean Mackrory (JIRA)" <ji...@apache.org> on 2012/07/13 23:18:34 UTC
[jira] [Created] (BIGTOP-674) BigTop
Sean Mackrory created BIGTOP-674:
------------------------------------
Summary: BigTop
Key: BIGTOP-674
URL: https://issues.apache.org/jira/browse/BIGTOP-674
Project: Bigtop
Issue Type: Bug
Environment: Debian 12.04, probably other DEB and RPM-based distros
Reporter: Sean Mackrory
Assignee: Sean Mackrory
When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Mackrory updated BIGTOP-674:
---------------------------------
Comment: was deleted
(was: diff --git a/bigtop-packages/src/deb/hadoop/hadoop-mapreduce.postinst b/bigtop-packages/src/deb/hadoop/hadoop-mapreduce.postinst
index 5f6977c..85da543 100644
--- a/bigtop-packages/src/deb/hadoop/hadoop-mapreduce.postinst
+++ b/bigtop-packages/src/deb/hadoop/hadoop-mapreduce.postinst
@@ -22,6 +22,7 @@ set -e
case "$1" in
configure)
mkdir -p /var/log/hadoop-mapreduce /var/run/hadoop-mapreduce || :
+ getent group hadoop >/dev/null || groupadd -r hadoop
chgrp -R hadoop /var/log/hadoop-mapreduce /var/run/hadoop-mapreduce
chmod g+w /var/run/hadoop-mapreduce /var/log/hadoop-mapreduce
chown mapred:hadoop /var/lib/hadoop-mapreduce /var/lib/hadoop-mapreduce/cache
diff --git a/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec b/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec
index f5a7125..cd1819b 100644
--- a/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec
+++ b/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec
@@ -523,6 +523,7 @@ getent passwd yarn >/dev/null || /usr/sbin/useradd --comment "Hadoop Yarn" --she
%pre mapreduce
getent group mapred >/dev/null || groupadd -r mapred
+getent group hadoop >/dev/null || groupadd -r hadoop
getent passwd mapred >/dev/null || /usr/sbin/useradd --comment "Hadoop MapReduce" --shell /bin/bash -M -r -g mapred -G hadoop --home %{state_mapreduce} mapred
%post)
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Mackrory reassigned BIGTOP-674:
------------------------------------
Assignee: (was: Sean Mackrory)
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414529#comment-13414529 ]
Bruno Mahé commented on BIGTOP-674:
-----------------------------------
How did you install hadoop-mapreduce package?
What error do you see?
What steps have you been through?
The group hadoop is created as part of the package hadoop and therefore will be created before hadoop-mapreduce gets installed.
See https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/deb/hadoop/hadoop.preinst#L35
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-674) Installing hadoop-mapreduce
package uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414071#comment-13414071 ]
Sean Mackrory edited comment on BIGTOP-674 at 7/13/12 9:26 PM:
---------------------------------------------------------------
Patch tested on DEBs, still working on verifying on RPMs, but that might me take a little while.
was (Author: mackrorysd):
Tested in DEBs, still working on verifying for RPMs, but that might me take a little while.
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415491#comment-13415491 ]
Sean Mackrory commented on BIGTOP-674:
--------------------------------------
Although the package hadoop is ultimately a dependency, the group hadoop is assumed to have been created in the preinst stage of the package hadoop-mapreduce. Debian Policy Manual states "Only essential packages and pre-dependencies (Pre-Depends) may be assumed to be available." As you'll see in the output I've copied below, dpkg does indeed list hadoop-yarn as a dependency (which subsequently lists hadoop as a dependency), but it does so only after the error is encountered, so it's confusing to a user at best. I've had this problem when using an aptitude repository as well (albeit not as consistently), which I think is even worse. Perhaps a more correct solution would be to add "Pre-Depends: hadoop (= ${source:Version})" to the deb package hadoop-mapreduce.
As for the rpm package, I'm not sure I can find a better solution for RPMs than what I already submitted. Perhaps using "PreReq" instrad of "Requires", but it sounds like that difference will not be reliable anyway: http://www.rpm.org/max-rpm-snapshot/s1-rpm-depend-manual-dependencies.html#S3-RPM-DEPEND-PREREQ
$ sudo dpkg -i hadoop-mapreduce_2.0.0-1_all.deb
Selecting previously unselected package hadoop-mapreduce.
(Reading database ... 51084 files and directories currently installed.)
Unpacking hadoop-mapreduce (from hadoop-mapreduce_2.0.0-1_all.deb) ...
usermod: group 'hadoop' does not exist
dpkg: dependency problems prevent configuration of hadoop-mapreduce:
hadoop-mapreduce depends on bigtop-utils; however:
Package bigtop-utils is not installed.
hadoop-mapreduce depends on hadoop-yarn (= 2.0.0-1); however:
Package hadoop-yarn is not installed.
dpkg: error processing hadoop-mapreduce (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
hadoop-mapreduce
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruno Mahé reassigned BIGTOP-674:
---------------------------------
Assignee: Bruno Mahé
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Assignee: Bruno Mahé
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Mackrory updated BIGTOP-674:
---------------------------------
Attachment: BIGTOP-674.patch
Tested in DEBs, still working on verifying for RPMs, but that might me take a little while.
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: BIGTOP-674.patch
>
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-674) Installing hadoop-mapreduce package
uses 'hadoop' group before creation
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Mackrory updated BIGTOP-674:
---------------------------------
Summary: Installing hadoop-mapreduce package uses 'hadoop' group before creation (was: BigTop)
> Installing hadoop-mapreduce package uses 'hadoop' group before creation
> -----------------------------------------------------------------------
>
> Key: BIGTOP-674
> URL: https://issues.apache.org/jira/browse/BIGTOP-674
> Project: Bigtop
> Issue Type: Bug
> Environment: Debian 12.04, probably other DEB and RPM-based distros
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
>
> When installing the hadoop-mapreduce package, users and files are assigned to the 'hadoop' group before that group has been created.
> I'm still working on verifying all the details for RPMs, but I will attached a patch that fixes this for DEBs (and quite possibly for RPMs too). It doesn't make sense to me why the group has not been added earlier. The hadoop group is added in the hadoop.spec file on line 510, and in the hadoop.preinst script - and I think both of these are supposed to execute before the lines where I've added the group in the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira