You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2016/01/27 10:29:54 UTC

Review Request 42847: Systemd on RHEL/CentOS 7.2 breaks ambari-server startup

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42847/
-----------------------------------------------------------

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-14807
    https://issues.apache.org/jira/browse/AMBARI-14807


Repository: ambari


Description
-------

Problem:

Systemd on RHEL/CentOS 7.2 breaks Ambari startup.

Business Impact:

Customers are unable to automate the startup of Ambari services

Steps to Reproduce:

1\. Install RHEL 7.2  
2\. Configure env for Ambari  
3\. Install Ambari 2.2.0  
4\. Configure Ambari  
5\. Start Ambari  
6\. Try using systemd to work with Ambari eg. systemctl status ambari-server

Expected behaviour:

Under RHEL 7.1 this worked, sample output:

    
    
    [root@node1 ~]# systemctl status ambari-server
    ambari-server.service - SYSV: ambari-server daemon
       Loaded: loaded (/etc/rc.d/init.d/ambari-server)
       Active: active (running) since Mon 2016-01-18 15:46:08 GMT; 16h ago
      Process: 903 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS)
       CGroup: /system.slice/ambari-server.service
               ??2343 /usr/jdk64/jdk1.8.0_40/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInit...
    
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Ambari Server running with administrator privileges.
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Running initdb: This may take upto a minute.
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: About to start PostgreSQL
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Organizing resource files at /var/lib/ambari-server/resources...
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server PID at: /var/run/ambari-server/ambari-server.pid
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server out at: /var/log/ambari-server/ambari-server.out
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server log at: /var/log/ambari-server/ambari-server.log
    Jan 18 15:46:08 node1.node1.com ambari-server[903]: Waiting for server start...................
    Jan 18 15:46:08 node1.node1.com ambari-server[903]: Ambari Server 'start' completed successfully.
    Jan 18 15:46:08 node1.node1.com systemd[1]: Started SYSV: ambari-server daemon.
    

Actual Behaviour:

    
    
    [root@ip-172-30-0-10 ec2-user]# systemctl status ambari-server
    ? ambari-server.service
       Loaded: not-found (Reason: No such file or directory)
       Active: inactive (dead)
    [root@ip-172-30-0-10 ec2-user]#
    

Support Analysis:

I have tested this on Ambari 2.2.0 on CentOS 7.1 and RHEL 7.2 and can confirm
that it's broken on 7.2.

Rackspace discovered this and provided the following feedback:

Version 7.2 of CentOS/RHEL includes a new version of systemd that breaks how
the ambari-server is started up.  
The init.d script from the ambari-server package is not recognized anymore and
not added to systemd, thus ambari-server cannot start at boot time anymore:

    
    
    [root@server-02 ~]# rpm -qa|grep ambari 
    ambari-agent-2.2.0.0-1310.x86_64 
    ambari-server-2.2.0.0-1310.x86_64 
    [root@server-02 ~]# cat /etc/redhat-release 
    CentOS Linux release 7.2.1511 (Core) 
    [root@server-02 ~]# systemctl daemon-reload 
    [root@server-02 ~]# systemctl status ambari-server 
    ? ambari-server.service 
    Loaded: not-found (Reason: No such file or directory) 
    Active: inactive (dead) 
    [root@server-02 ~]# 
    

I believe this is because /etc/rc.d/init.d/ambari-server is a symlink.  
ambari-agent init script still works just fine:

    
    
    [root@server-02 ~]# ls -l /etc/rc.d/init.d/ambari-* 
    -rwxr-xr-x. 1 root root 1628 Dec 20 23:34 /etc/rc.d/init.d/ambari-agent 
    lrwxrwxrwx. 1 root root 23 Jan 20 10:18 /etc/rc.d/init.d/ambari-server -> /usr/sbin/ambari-server 
    [root@server-02 ~]# systemctl status ambari-agent 
    ? ambari-agent.service - SYSV: ambari-agent daemon 
    Loaded: loaded (/etc/rc.d/init.d/ambari-agent) 
    

If I remove the symlink and copy the /usr/sbin/ambari-server then systemd
works again just fine:

    
    
    [root@server-02 ~]# unlink /etc/rc.d/init.d/ambari-server && cp -a /usr/sbin/ambari-server /etc/rc.d/init.d/ambari-server && systemctl daemon-reload 
    [root@server-02 ~]# systemctl status ambari-server 
    ? ambari-server.service - SYSV: ambari-server daemon 
    Loaded: loaded (/etc/rc.d/init.d/ambari-server) 
    

So please have this fixed as currently, by default, ambari-server startup is
broken in CentOS/RHEL 7.2


Diffs
-----

  ambari-server/pom.xml fc57ebd 
  ambari-server/src/main/package/deb/control/postinst 629ab09 
  ambari-server/src/main/package/deb/control/prerm 15d05c4 
  ambari-server/src/main/package/rpm/postinstall.sh d47a00c 
  ambari-server/src/main/package/rpm/preremove.sh 12b69b6 

Diff: https://reviews.apache.org/r/42847/diff/


Testing
-------

mvn clean test


Thanks,

Andrew Onischuk


Re: Review Request 42847: Systemd on RHEL/CentOS 7.2 breaks ambari-server startup

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42847/#review116565
-----------------------------------------------------------


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Січ. 27, 2016, 9:30 до полудня, Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42847/
> -----------------------------------------------------------
> 
> (Updated Січ. 27, 2016, 9:30 до полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-14807
>     https://issues.apache.org/jira/browse/AMBARI-14807
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Systemd on RHEL/CentOS 7.2 breaks Ambari startup.
> 
> Users are unable to automate the startup of Ambari services
> 
> Steps to Reproduce:
> 
> 1\. Install RHEL 7.2  
> 2\. Configure env for Ambari  
> 3\. Install Ambari 2.2.0  
> 4\. Configure Ambari  
> 5\. Start Ambari  
> 6\. Try using systemd to work with Ambari eg. systemctl status ambari-server
> 
> Expected behaviour:
> 
> Under RHEL 7.1 this worked, sample output:
> 
>     
>     
>     [root@node1 ~]# systemctl status ambari-server
>     ambari-server.service - SYSV: ambari-server daemon
>        Loaded: loaded (/etc/rc.d/init.d/ambari-server)
>        Active: active (running) since Mon 2016-01-18 15:46:08 GMT; 16h ago
>       Process: 903 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS)
>        CGroup: /system.slice/ambari-server.service
>                ??2343 /usr/jdk64/jdk1.8.0_40/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInit...
>     
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Ambari Server running with administrator privileges.
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Running initdb: This may take upto a minute.
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: About to start PostgreSQL
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Organizing resource files at /var/lib/ambari-server/resources...
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server PID at: /var/run/ambari-server/ambari-server.pid
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server out at: /var/log/ambari-server/ambari-server.out
>     Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server log at: /var/log/ambari-server/ambari-server.log
>     Jan 18 15:46:08 node1.node1.com ambari-server[903]: Waiting for server start...................
>     Jan 18 15:46:08 node1.node1.com ambari-server[903]: Ambari Server 'start' completed successfully.
>     Jan 18 15:46:08 node1.node1.com systemd[1]: Started SYSV: ambari-server daemon.
>     
> 
> Actual Behaviour:
> 
>     
>     
>     [root@ip-172-30-0-10 ec2-user]# systemctl status ambari-server
>     ? ambari-server.service
>        Loaded: not-found (Reason: No such file or directory)
>        Active: inactive (dead)
>     [root@ip-172-30-0-10 ec2-user]#
>     
> 
> Support Analysis:
> 
> I have tested this on Ambari 2.2.0 on CentOS 7.1 and RHEL 7.2 and can confirm
> that it's broken on 7.2.
> 
> Rackspace discovered this and provided the following feedback:
> 
> Version 7.2 of CentOS/RHEL includes a new version of systemd that breaks how
> the ambari-server is started up.  
> The init.d script from the ambari-server package is not recognized anymore and
> not added to systemd, thus ambari-server cannot start at boot time anymore:
> 
>     
>     
>     [root@server-02 ~]# rpm -qa|grep ambari 
>     ambari-agent-2.2.0.0-1310.x86_64 
>     ambari-server-2.2.0.0-1310.x86_64 
>     [root@server-02 ~]# cat /etc/redhat-release 
>     CentOS Linux release 7.2.1511 (Core) 
>     [root@server-02 ~]# systemctl daemon-reload 
>     [root@server-02 ~]# systemctl status ambari-server 
>     ? ambari-server.service 
>     Loaded: not-found (Reason: No such file or directory) 
>     Active: inactive (dead) 
>     [root@server-02 ~]# 
>     
> 
> I believe this is because /etc/rc.d/init.d/ambari-server is a symlink.  
> ambari-agent init script still works just fine:
> 
>     
>     
>     [root@server-02 ~]# ls -l /etc/rc.d/init.d/ambari-* 
>     -rwxr-xr-x. 1 root root 1628 Dec 20 23:34 /etc/rc.d/init.d/ambari-agent 
>     lrwxrwxrwx. 1 root root 23 Jan 20 10:18 /etc/rc.d/init.d/ambari-server -> /usr/sbin/ambari-server 
>     [root@server-02 ~]# systemctl status ambari-agent 
>     ? ambari-agent.service - SYSV: ambari-agent daemon 
>     Loaded: loaded (/etc/rc.d/init.d/ambari-agent) 
>     
> 
> If I remove the symlink and copy the /usr/sbin/ambari-server then systemd
> works again just fine:
> 
>     
>     
>     [root@server-02 ~]# unlink /etc/rc.d/init.d/ambari-server && cp -a /usr/sbin/ambari-server /etc/rc.d/init.d/ambari-server && systemctl daemon-reload 
>     [root@server-02 ~]# systemctl status ambari-server 
>     ? ambari-server.service - SYSV: ambari-server daemon 
>     Loaded: loaded (/etc/rc.d/init.d/ambari-server) 
>     
> 
> So please have this fixed as currently, by default, ambari-server startup is
> broken in CentOS/RHEL 7.2
> 
> 
> Diffs
> -----
> 
>   ambari-server/pom.xml fc57ebd 
>   ambari-server/src/main/package/deb/control/postinst 629ab09 
>   ambari-server/src/main/package/deb/control/prerm 15d05c4 
>   ambari-server/src/main/package/rpm/postinstall.sh d47a00c 
>   ambari-server/src/main/package/rpm/preremove.sh 12b69b6 
> 
> Diff: https://reviews.apache.org/r/42847/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>


Re: Review Request 42847: Systemd on RHEL/CentOS 7.2 breaks ambari-server startup

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42847/
-----------------------------------------------------------

(Updated Jan. 27, 2016, 9:30 a.m.)


Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-14807
    https://issues.apache.org/jira/browse/AMBARI-14807


Repository: ambari


Description (updated)
-------

Systemd on RHEL/CentOS 7.2 breaks Ambari startup.

Users are unable to automate the startup of Ambari services

Steps to Reproduce:

1\. Install RHEL 7.2  
2\. Configure env for Ambari  
3\. Install Ambari 2.2.0  
4\. Configure Ambari  
5\. Start Ambari  
6\. Try using systemd to work with Ambari eg. systemctl status ambari-server

Expected behaviour:

Under RHEL 7.1 this worked, sample output:

    
    
    [root@node1 ~]# systemctl status ambari-server
    ambari-server.service - SYSV: ambari-server daemon
       Loaded: loaded (/etc/rc.d/init.d/ambari-server)
       Active: active (running) since Mon 2016-01-18 15:46:08 GMT; 16h ago
      Process: 903 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS)
       CGroup: /system.slice/ambari-server.service
               ??2343 /usr/jdk64/jdk1.8.0_40/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInit...
    
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Ambari Server running with administrator privileges.
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Running initdb: This may take upto a minute.
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: About to start PostgreSQL
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Organizing resource files at /var/lib/ambari-server/resources...
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server PID at: /var/run/ambari-server/ambari-server.pid
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server out at: /var/log/ambari-server/ambari-server.out
    Jan 18 15:45:51 node1.node1.com ambari-server[903]: Server log at: /var/log/ambari-server/ambari-server.log
    Jan 18 15:46:08 node1.node1.com ambari-server[903]: Waiting for server start...................
    Jan 18 15:46:08 node1.node1.com ambari-server[903]: Ambari Server 'start' completed successfully.
    Jan 18 15:46:08 node1.node1.com systemd[1]: Started SYSV: ambari-server daemon.
    

Actual Behaviour:

    
    
    [root@ip-172-30-0-10 ec2-user]# systemctl status ambari-server
    ? ambari-server.service
       Loaded: not-found (Reason: No such file or directory)
       Active: inactive (dead)
    [root@ip-172-30-0-10 ec2-user]#
    

Support Analysis:

I have tested this on Ambari 2.2.0 on CentOS 7.1 and RHEL 7.2 and can confirm
that it's broken on 7.2.

Rackspace discovered this and provided the following feedback:

Version 7.2 of CentOS/RHEL includes a new version of systemd that breaks how
the ambari-server is started up.  
The init.d script from the ambari-server package is not recognized anymore and
not added to systemd, thus ambari-server cannot start at boot time anymore:

    
    
    [root@server-02 ~]# rpm -qa|grep ambari 
    ambari-agent-2.2.0.0-1310.x86_64 
    ambari-server-2.2.0.0-1310.x86_64 
    [root@server-02 ~]# cat /etc/redhat-release 
    CentOS Linux release 7.2.1511 (Core) 
    [root@server-02 ~]# systemctl daemon-reload 
    [root@server-02 ~]# systemctl status ambari-server 
    ? ambari-server.service 
    Loaded: not-found (Reason: No such file or directory) 
    Active: inactive (dead) 
    [root@server-02 ~]# 
    

I believe this is because /etc/rc.d/init.d/ambari-server is a symlink.  
ambari-agent init script still works just fine:

    
    
    [root@server-02 ~]# ls -l /etc/rc.d/init.d/ambari-* 
    -rwxr-xr-x. 1 root root 1628 Dec 20 23:34 /etc/rc.d/init.d/ambari-agent 
    lrwxrwxrwx. 1 root root 23 Jan 20 10:18 /etc/rc.d/init.d/ambari-server -> /usr/sbin/ambari-server 
    [root@server-02 ~]# systemctl status ambari-agent 
    ? ambari-agent.service - SYSV: ambari-agent daemon 
    Loaded: loaded (/etc/rc.d/init.d/ambari-agent) 
    

If I remove the symlink and copy the /usr/sbin/ambari-server then systemd
works again just fine:

    
    
    [root@server-02 ~]# unlink /etc/rc.d/init.d/ambari-server && cp -a /usr/sbin/ambari-server /etc/rc.d/init.d/ambari-server && systemctl daemon-reload 
    [root@server-02 ~]# systemctl status ambari-server 
    ? ambari-server.service - SYSV: ambari-server daemon 
    Loaded: loaded (/etc/rc.d/init.d/ambari-server) 
    

So please have this fixed as currently, by default, ambari-server startup is
broken in CentOS/RHEL 7.2


Diffs
-----

  ambari-server/pom.xml fc57ebd 
  ambari-server/src/main/package/deb/control/postinst 629ab09 
  ambari-server/src/main/package/deb/control/prerm 15d05c4 
  ambari-server/src/main/package/rpm/postinstall.sh d47a00c 
  ambari-server/src/main/package/rpm/preremove.sh 12b69b6 

Diff: https://reviews.apache.org/r/42847/diff/


Testing
-------

mvn clean test


Thanks,

Andrew Onischuk