You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2015/12/18 11:32:56 UTC
Review Request 41542: Get rid of wildcards in package names.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41542/
-----------------------------------------------------------
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-14430
https://issues.apache.org/jira/browse/AMBARI-14430
Repository: ambari
Description
-------
**Problem**:
Wildcards in packages bring and bunch of problems connected with RU and normal
installs.
a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
installed, but hadoop_2_2_0_1234 fails, we can no longer install master
package.
b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
installed before hive_2_2_*, we get problems with inability to install. I've
bumped into a couple of different situations like this. (storm_2_2* and
storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
are real bugs which we had to hack out
c) We install a lot of packages which are not needed by default like hadoop-
2_2_...fuse, just by asking hadoop_2_2_*.
d) Situtations like 47931 can happen in RU.
e) The bug in RU where we add a service after install_packages was triggered.
**Proposal:**
1\. Replace * by ${hdp_version} in metainfo.xml.
E.g.:
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_2_3_*</name>
</package>
<package>
<name>phoenix_2_3_*</name>
</package>
</packages>
</osSpecific>
->
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_${hdp_version}</name>
</package>
<package>
<name>hbase_${hdp_version}-regionserver</name>
</package>
<package>
<name>phoenix_${hdp_version}</name>
</package>
</packages>
</osSpecific>
Note: this step will also let us not override packages in metainfo.xml with
every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.
2\. During INSTALL action (script.py) or during install packages
(install_packages.py) get the version we are currently using from hdp-select.
e.g. for centos:
[root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
2.3.4.0
[root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
3204 # a bit slow for every INSTALL task
Another better aproach is to install hdp-select and than use rpm/dpkg, which
is faster than yum/zypper/apt-get, considering we will do this on every
INSTALL task this is important.
[root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g' # faster approach
2.3.4.0-3204
u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}'
2.3.4.0-3475
Vs other approaches:
I think it's the only possible approach to get the version here, sending
version which is specified by user during creating of a version. Seems to be a
bad idea as the user can specify wrong version. Also we will need to use this
during fresh install, where we don't have it specified at all.
3\. Replace ${hdp_version} with the version we got in step 2.
I also discussed this with dlysnichenko this seems to be feasible in terms of
RU/EU.
The overall idea was discussed in 47931
cc smohanty
Diffs
-----
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 70a553d
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py 87938ad
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 1fbc9bf
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py 6dbcef5
ambari-common/src/main/python/resource_management/libraries/functions/packages_analyzer.py 7f34cc1
ambari-common/src/main/python/resource_management/libraries/script/script.py c954a15
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_service.py 0d9687b
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py d39ccbf
ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/metainfo.xml fe47276
ambari-server/src/main/resources/stacks/HDP/2.2/services/FLUME/metainfo.xml 4fa4ff7
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/metainfo.xml 71297b1
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 0366758
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml d59b89e
ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/metainfo.xml aa164e3
ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/metainfo.xml 481fca9
ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/metainfo.xml db7e025
ambari-server/src/main/resources/stacks/HDP/2.2/services/PIG/metainfo.xml 5472f43
ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/metainfo.xml 0c457f7
ambari-server/src/main/resources/stacks/HDP/2.2/services/SLIDER/metainfo.xml 47bb207
ambari-server/src/main/resources/stacks/HDP/2.2/services/SPARK/metainfo.xml 5cb598e
ambari-server/src/main/resources/stacks/HDP/2.2/services/SQOOP/metainfo.xml e7ff122
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/metainfo.xml ccf765b
ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/metainfo.xml b0ccddd
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml 189defc
ambari-server/src/main/resources/stacks/HDP/2.2/services/ZOOKEEPER/metainfo.xml d0b37cf
ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/metainfo.xml b19aac1
ambari-server/src/main/resources/stacks/HDP/2.3/services/ATLAS/metainfo.xml c3a3c81
ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/metainfo.xml 6fdad8b
ambari-server/src/main/resources/stacks/HDP/2.3/services/FLUME/metainfo.xml 5031f85
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metainfo.xml a37378c
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml d103caf
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml b6b6bb8
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/metainfo.xml 330b542
ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/metainfo.xml d1172b1
ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 2fe0ed0
ambari-server/src/main/resources/stacks/HDP/2.3/services/PIG/metainfo.xml 8ba2426
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml bc52e85
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/metainfo.xml b1a5584
ambari-server/src/main/resources/stacks/HDP/2.3/services/SLIDER/metainfo.xml 8287b50
ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml 2cf290b
ambari-server/src/main/resources/stacks/HDP/2.3/services/SQOOP/metainfo.xml 1290cde
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/metainfo.xml 00af620
ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/metainfo.xml 7720a3d
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/metainfo.xml 088bb16
ambari-server/src/main/resources/stacks/HDP/2.3/services/ZOOKEEPER/metainfo.xml c2fe742
ambari-server/src/main/resources/stacks/HDP/2.4/services/ACCUMULO/metainfo.xml 9e82a82
ambari-server/src/main/resources/stacks/HDP/2.4/services/ATLAS/metainfo.xml 158ef8c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FALCON/metainfo.xml 183d55c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FLUME/metainfo.xml b7cd3d1
ambari-server/src/main/resources/stacks/HDP/2.4/services/HBASE/metainfo.xml a1c2bd0
ambari-server/src/main/resources/stacks/HDP/2.4/services/HDFS/metainfo.xml ce3fb95
ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 5ad52b6
ambari-server/src/main/resources/stacks/HDP/2.4/services/KAFKA/metainfo.xml 90bc472
ambari-server/src/main/resources/stacks/HDP/2.4/services/KNOX/metainfo.xml e465d42
ambari-server/src/main/resources/stacks/HDP/2.4/services/OOZIE/metainfo.xml 2f09e88
ambari-server/src/main/resources/stacks/HDP/2.4/services/PIG/metainfo.xml 9ae1f42
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml d76ad5c
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER_KMS/metainfo.xml a27bf31
ambari-server/src/main/resources/stacks/HDP/2.4/services/SLIDER/metainfo.xml ff998f0
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml f4880e2
ambari-server/src/main/resources/stacks/HDP/2.4/services/SQOOP/metainfo.xml 721cc33
ambari-server/src/main/resources/stacks/HDP/2.4/services/STORM/metainfo.xml 36b01e4
ambari-server/src/main/resources/stacks/HDP/2.4/services/TEZ/metainfo.xml 356225e
ambari-server/src/main/resources/stacks/HDP/2.4/services/YARN/metainfo.xml c61a270
ambari-server/src/main/resources/stacks/HDP/2.4/services/ZOOKEEPER/metainfo.xml 4bf4198
ambari-server/src/test/python/custom_actions/TestInstallPackages.py 6d8c2f0
ambari-server/src/test/python/custom_actions/configs/install_packages_config.json 50e327a
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 5a95fe2
Diff: https://reviews.apache.org/r/41542/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 41542: Get rid of wildcards in package names.
Posted by Myroslav Papirkovskyy <mp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41542/#review111165
-----------------------------------------------------------
Ship it!
Ship It!
- Myroslav Papirkovskyy
On Гру. 18, 2015, 12:35 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41542/
> -----------------------------------------------------------
>
> (Updated Гру. 18, 2015, 12:35 після полудня)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-14430
> https://issues.apache.org/jira/browse/AMBARI-14430
>
>
> Repository: ambari
>
>
> Description
> -------
>
> **Problem**:
> Wildcards in packages bring and bunch of problems connected with RU and normal
> installs.
> a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
> installed, but hadoop_2_2_0_1234 fails, we can no longer install master
> package.
> b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
> installed before hive_2_2_*, we get problems with inability to install. I've
> bumped into a couple of different situations like this. (storm_2_2* and
> storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
> are real bugs which we had to hack out
> c) We install a lot of packages which are not needed by default like hadoop-
> 2_2_...fuse, just by asking hadoop_2_2_*.
> d) Situtations like 47931 can happen in RU.
> e) The bug in RU where we add a service after install_packages was triggered.
>
> **Proposal:**
> 1\. Replace * by ${hdp_version} in metainfo.xml.
> E.g.:
>
>
>
> <osSpecific>
> <osFamily>redhat7,redhat6,suse11</osFamily>
> <packages>
> <package>
> <name>hbase_2_3_*</name>
> </package>
> <package>
> <name>phoenix_2_3_*</name>
> </package>
> </packages>
> </osSpecific>
>
>
> ->
>
>
> <osSpecific>
> <osFamily>redhat7,redhat6,suse11</osFamily>
> <packages>
> <package>
> <name>hbase_${hdp_version}</name>
> </package>
> <package>
> <name>hbase_${hdp_version}-regionserver</name>
> </package>
> <package>
> <name>phoenix_${hdp_version}</name>
> </package>
> </packages>
> </osSpecific>
>
>
> Note: this step will also let us not override packages in metainfo.xml with
> every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.
>
> 2\. During INSTALL action (script.py) or during install packages
> (install_packages.py) get the version we are currently using from hdp-select.
>
> e.g. for centos:
>
>
>
> [root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
> 2.3.4.0
> [root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
> 3204 # a bit slow for every INSTALL task
>
>
> Another better aproach is to install hdp-select and than use rpm/dpkg, which
> is faster than yum/zypper/apt-get, considering we will do this on every
> INSTALL task this is important.
>
>
>
> [root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g' # faster approach
> 2.3.4.0-3204
>
>
>
> u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}'
> 2.3.4.0-3475
>
>
> Vs other approaches:
> I think it's the only possible approach to get the version here, sending
> version which is specified by user during creating of a version. Seems to be a
> bad idea as the user can specify wrong version. Also we will need to use this
> during fresh install, where we don't have it specified at all.
>
> 3\. Replace ${hdp_version} with the version we got in step 2.
>
> I also discussed this with dlysnichenko this seems to be feasible in terms of
> RU/EU.
> The overall idea was discussed in 47931
> cc smohanty
>
>
> Also after talking to dlysnichenko he said we were excluding repos because of issues with wildcard on suse when it is not excluded and mutptiple packages is found for a wildcard. That's why when we stop using wildcards we can remove those exclude repos, which as well caused a bunch of bugs.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py 70a553d
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py 87938ad
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 1fbc9bf
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py 6dbcef5
> ambari-common/src/main/python/resource_management/libraries/functions/packages_analyzer.py 7f34cc1
> ambari-common/src/main/python/resource_management/libraries/script/script.py c954a15
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_service.py 0d9687b
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py d39ccbf
> ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/metainfo.xml fe47276
> ambari-server/src/main/resources/stacks/HDP/2.2/services/FLUME/metainfo.xml 4fa4ff7
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/metainfo.xml 71297b1
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 0366758
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml d59b89e
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/metainfo.xml aa164e3
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/metainfo.xml 481fca9
> ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/metainfo.xml db7e025
> ambari-server/src/main/resources/stacks/HDP/2.2/services/PIG/metainfo.xml 5472f43
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/metainfo.xml 0c457f7
> ambari-server/src/main/resources/stacks/HDP/2.2/services/SLIDER/metainfo.xml 47bb207
> ambari-server/src/main/resources/stacks/HDP/2.2/services/SPARK/metainfo.xml 5cb598e
> ambari-server/src/main/resources/stacks/HDP/2.2/services/SQOOP/metainfo.xml e7ff122
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/metainfo.xml ccf765b
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/metainfo.xml b0ccddd
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml 189defc
> ambari-server/src/main/resources/stacks/HDP/2.2/services/ZOOKEEPER/metainfo.xml d0b37cf
> ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/metainfo.xml b19aac1
> ambari-server/src/main/resources/stacks/HDP/2.3/services/ATLAS/metainfo.xml c3a3c81
> ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/metainfo.xml 6fdad8b
> ambari-server/src/main/resources/stacks/HDP/2.3/services/FLUME/metainfo.xml 5031f85
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metainfo.xml a37378c
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml d103caf
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml b6b6bb8
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/metainfo.xml 330b542
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/metainfo.xml d1172b1
> ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 2fe0ed0
> ambari-server/src/main/resources/stacks/HDP/2.3/services/PIG/metainfo.xml 8ba2426
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml bc52e85
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/metainfo.xml b1a5584
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SLIDER/metainfo.xml 8287b50
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml 2cf290b
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SQOOP/metainfo.xml 1290cde
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/metainfo.xml 00af620
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/metainfo.xml 7720a3d
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/metainfo.xml 088bb16
> ambari-server/src/main/resources/stacks/HDP/2.3/services/ZOOKEEPER/metainfo.xml c2fe742
> ambari-server/src/main/resources/stacks/HDP/2.4/services/ACCUMULO/metainfo.xml 9e82a82
> ambari-server/src/main/resources/stacks/HDP/2.4/services/ATLAS/metainfo.xml 158ef8c
> ambari-server/src/main/resources/stacks/HDP/2.4/services/FALCON/metainfo.xml 183d55c
> ambari-server/src/main/resources/stacks/HDP/2.4/services/FLUME/metainfo.xml b7cd3d1
> ambari-server/src/main/resources/stacks/HDP/2.4/services/HBASE/metainfo.xml a1c2bd0
> ambari-server/src/main/resources/stacks/HDP/2.4/services/HDFS/metainfo.xml ce3fb95
> ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 5ad52b6
> ambari-server/src/main/resources/stacks/HDP/2.4/services/KAFKA/metainfo.xml 90bc472
> ambari-server/src/main/resources/stacks/HDP/2.4/services/KNOX/metainfo.xml e465d42
> ambari-server/src/main/resources/stacks/HDP/2.4/services/OOZIE/metainfo.xml 2f09e88
> ambari-server/src/main/resources/stacks/HDP/2.4/services/PIG/metainfo.xml 9ae1f42
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml d76ad5c
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER_KMS/metainfo.xml a27bf31
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SLIDER/metainfo.xml ff998f0
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml f4880e2
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SQOOP/metainfo.xml 721cc33
> ambari-server/src/main/resources/stacks/HDP/2.4/services/STORM/metainfo.xml 36b01e4
> ambari-server/src/main/resources/stacks/HDP/2.4/services/TEZ/metainfo.xml 356225e
> ambari-server/src/main/resources/stacks/HDP/2.4/services/YARN/metainfo.xml c61a270
> ambari-server/src/main/resources/stacks/HDP/2.4/services/ZOOKEEPER/metainfo.xml 4bf4198
> ambari-server/src/test/python/custom_actions/TestInstallPackages.py 6d8c2f0
> ambari-server/src/test/python/custom_actions/configs/install_packages_config.json 50e327a
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 5a95fe2
>
> Diff: https://reviews.apache.org/r/41542/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 41542: Get rid of wildcards in package names.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41542/
-----------------------------------------------------------
(Updated Dec. 18, 2015, 10:35 a.m.)
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-14430
https://issues.apache.org/jira/browse/AMBARI-14430
Repository: ambari
Description (updated)
-------
**Problem**:
Wildcards in packages bring and bunch of problems connected with RU and normal
installs.
a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
installed, but hadoop_2_2_0_1234 fails, we can no longer install master
package.
b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
installed before hive_2_2_*, we get problems with inability to install. I've
bumped into a couple of different situations like this. (storm_2_2* and
storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
are real bugs which we had to hack out
c) We install a lot of packages which are not needed by default like hadoop-
2_2_...fuse, just by asking hadoop_2_2_*.
d) Situtations like 47931 can happen in RU.
e) The bug in RU where we add a service after install_packages was triggered.
**Proposal:**
1\. Replace * by ${hdp_version} in metainfo.xml.
E.g.:
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_2_3_*</name>
</package>
<package>
<name>phoenix_2_3_*</name>
</package>
</packages>
</osSpecific>
->
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_${hdp_version}</name>
</package>
<package>
<name>hbase_${hdp_version}-regionserver</name>
</package>
<package>
<name>phoenix_${hdp_version}</name>
</package>
</packages>
</osSpecific>
Note: this step will also let us not override packages in metainfo.xml with
every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.
2\. During INSTALL action (script.py) or during install packages
(install_packages.py) get the version we are currently using from hdp-select.
e.g. for centos:
[root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
2.3.4.0
[root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
3204 # a bit slow for every INSTALL task
Another better aproach is to install hdp-select and than use rpm/dpkg, which
is faster than yum/zypper/apt-get, considering we will do this on every
INSTALL task this is important.
[root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g' # faster approach
2.3.4.0-3204
u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}'
2.3.4.0-3475
Vs other approaches:
I think it's the only possible approach to get the version here, sending
version which is specified by user during creating of a version. Seems to be a
bad idea as the user can specify wrong version. Also we will need to use this
during fresh install, where we don't have it specified at all.
3\. Replace ${hdp_version} with the version we got in step 2.
I also discussed this with dlysnichenko this seems to be feasible in terms of
RU/EU.
The overall idea was discussed in 47931
cc smohanty
Also after talking to dlysnichenko he said we were excluding repos because of issues with wildcard on suse when it is not excluded and mutptiple packages is found for a wildcard. That's why when we stop using wildcards we can remove those exclude repos, which as well caused a bunch of bugs.
Diffs
-----
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 70a553d
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py 87938ad
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 1fbc9bf
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py 6dbcef5
ambari-common/src/main/python/resource_management/libraries/functions/packages_analyzer.py 7f34cc1
ambari-common/src/main/python/resource_management/libraries/script/script.py c954a15
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_service.py 0d9687b
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py d39ccbf
ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/metainfo.xml fe47276
ambari-server/src/main/resources/stacks/HDP/2.2/services/FLUME/metainfo.xml 4fa4ff7
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/metainfo.xml 71297b1
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 0366758
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml d59b89e
ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/metainfo.xml aa164e3
ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/metainfo.xml 481fca9
ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/metainfo.xml db7e025
ambari-server/src/main/resources/stacks/HDP/2.2/services/PIG/metainfo.xml 5472f43
ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/metainfo.xml 0c457f7
ambari-server/src/main/resources/stacks/HDP/2.2/services/SLIDER/metainfo.xml 47bb207
ambari-server/src/main/resources/stacks/HDP/2.2/services/SPARK/metainfo.xml 5cb598e
ambari-server/src/main/resources/stacks/HDP/2.2/services/SQOOP/metainfo.xml e7ff122
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/metainfo.xml ccf765b
ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/metainfo.xml b0ccddd
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml 189defc
ambari-server/src/main/resources/stacks/HDP/2.2/services/ZOOKEEPER/metainfo.xml d0b37cf
ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/metainfo.xml b19aac1
ambari-server/src/main/resources/stacks/HDP/2.3/services/ATLAS/metainfo.xml c3a3c81
ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/metainfo.xml 6fdad8b
ambari-server/src/main/resources/stacks/HDP/2.3/services/FLUME/metainfo.xml 5031f85
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metainfo.xml a37378c
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml d103caf
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml b6b6bb8
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/metainfo.xml 330b542
ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/metainfo.xml d1172b1
ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 2fe0ed0
ambari-server/src/main/resources/stacks/HDP/2.3/services/PIG/metainfo.xml 8ba2426
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml bc52e85
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/metainfo.xml b1a5584
ambari-server/src/main/resources/stacks/HDP/2.3/services/SLIDER/metainfo.xml 8287b50
ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml 2cf290b
ambari-server/src/main/resources/stacks/HDP/2.3/services/SQOOP/metainfo.xml 1290cde
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/metainfo.xml 00af620
ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/metainfo.xml 7720a3d
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/metainfo.xml 088bb16
ambari-server/src/main/resources/stacks/HDP/2.3/services/ZOOKEEPER/metainfo.xml c2fe742
ambari-server/src/main/resources/stacks/HDP/2.4/services/ACCUMULO/metainfo.xml 9e82a82
ambari-server/src/main/resources/stacks/HDP/2.4/services/ATLAS/metainfo.xml 158ef8c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FALCON/metainfo.xml 183d55c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FLUME/metainfo.xml b7cd3d1
ambari-server/src/main/resources/stacks/HDP/2.4/services/HBASE/metainfo.xml a1c2bd0
ambari-server/src/main/resources/stacks/HDP/2.4/services/HDFS/metainfo.xml ce3fb95
ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml 5ad52b6
ambari-server/src/main/resources/stacks/HDP/2.4/services/KAFKA/metainfo.xml 90bc472
ambari-server/src/main/resources/stacks/HDP/2.4/services/KNOX/metainfo.xml e465d42
ambari-server/src/main/resources/stacks/HDP/2.4/services/OOZIE/metainfo.xml 2f09e88
ambari-server/src/main/resources/stacks/HDP/2.4/services/PIG/metainfo.xml 9ae1f42
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml d76ad5c
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER_KMS/metainfo.xml a27bf31
ambari-server/src/main/resources/stacks/HDP/2.4/services/SLIDER/metainfo.xml ff998f0
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml f4880e2
ambari-server/src/main/resources/stacks/HDP/2.4/services/SQOOP/metainfo.xml 721cc33
ambari-server/src/main/resources/stacks/HDP/2.4/services/STORM/metainfo.xml 36b01e4
ambari-server/src/main/resources/stacks/HDP/2.4/services/TEZ/metainfo.xml 356225e
ambari-server/src/main/resources/stacks/HDP/2.4/services/YARN/metainfo.xml c61a270
ambari-server/src/main/resources/stacks/HDP/2.4/services/ZOOKEEPER/metainfo.xml 4bf4198
ambari-server/src/test/python/custom_actions/TestInstallPackages.py 6d8c2f0
ambari-server/src/test/python/custom_actions/configs/install_packages_config.json 50e327a
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 5a95fe2
Diff: https://reviews.apache.org/r/41542/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk