You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2020/07/29 07:58:57 UTC
[GitHub] [bigtop] sekikn opened a new pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
sekikn opened a new pull request #656:
URL: https://github.com/apache/bigtop/pull/656
@guyuqi @mattAndruff @masatana Would you review if possible? In this PR, I intended:
* Specify python package name explicitly as 'python2' in spec files so that they work with CentOS 8 and Fedora 31
* Replace Ambari functions that depend on the rpm-python package, which is not provided on CentOS 8, with an implementation using the `rpm` command
* Define the `python` command as an alias of `python2` on CentOS 8 and Fedora 31, which don't provide unversioned python command by default
I confirmed that Ambari, Livy and Zeppelin (the latter of two use Python3, in contrast to Ambari) are successfully built on CentOS 7, 8 and Fedora 31 with this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] masatana edited a comment on pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
masatana edited a comment on pull request #656:
URL: https://github.com/apache/bigtop/pull/656#issuecomment-665539868
@sekikn Thanks for working on this! I haven't tested this patch yet, but it looks good except for one point.
I'll test this patch on my environment later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] masatana commented on a change in pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
masatana commented on a change in pull request #656:
URL: https://github.com/apache/bigtop/pull/656#discussion_r463315391
##########
File path: bigtop-packages/src/common/ambari/patch4-AMBARI-CentOS8.diff
##########
@@ -0,0 +1,59 @@
+diff --git a/ambari-agent/src/main/package/dependencies.properties b/ambari-agent/src/main/package/dependencies.properties
+index 07b0b68319..9cfc2daeb5 100644
+--- a/ambari-agent/src/main/package/dependencies.properties
++++ b/ambari-agent/src/main/package/dependencies.properties
+@@ -28,5 +28,5 @@
+ # Such a format is respected by install_ambari_tarball.py by default,
+ # however should be encouraged manually in pom.xml.
+
+-rpm.dependency.list=openssl,\nRequires: rpm-python,\nRequires: zlib,\nRequires: python >= 2.6
+-deb.dependency.list=openssl, zlibc, python (>= 2.6)
+\ No newline at end of file
++rpm.dependency.list=openssl,\nRequires: zlib,\nRequires: python2 >= 2.6
++deb.dependency.list=openssl, zlibc, python (>= 2.6)
+diff --git a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+index 799cf3eed0..ade58292cb 100644
+--- a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
++++ b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+@@ -359,9 +359,8 @@ class RPMBasedPackageProvider(PackageProvider):
+ INSTALLED_PACKAGE_VERSION_COMMAND = "rpm -q --queryformat '%{{version}}-%{{release}}' \"{0}\""
+
+ def rpm_check_package_available(self, name):
+- import rpm # this is faster then calling 'rpm'-binary externally.
+- ts = rpm.TransactionSet()
+- packages = ts.dbMatch()
++ import os
++ packages = os.popen("rpm -qa --queryformat '%{name} '").read().split()
Review comment:
+1. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on a change in pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
sekikn commented on a change in pull request #656:
URL: https://github.com/apache/bigtop/pull/656#discussion_r462265131
##########
File path: bigtop-packages/src/common/ambari/patch4-AMBARI-CentOS8.diff
##########
@@ -0,0 +1,59 @@
+diff --git a/ambari-agent/src/main/package/dependencies.properties b/ambari-agent/src/main/package/dependencies.properties
+index 07b0b68319..9cfc2daeb5 100644
+--- a/ambari-agent/src/main/package/dependencies.properties
++++ b/ambari-agent/src/main/package/dependencies.properties
+@@ -28,5 +28,5 @@
+ # Such a format is respected by install_ambari_tarball.py by default,
+ # however should be encouraged manually in pom.xml.
+
+-rpm.dependency.list=openssl,\nRequires: rpm-python,\nRequires: zlib,\nRequires: python >= 2.6
+-deb.dependency.list=openssl, zlibc, python (>= 2.6)
+\ No newline at end of file
++rpm.dependency.list=openssl,\nRequires: zlib,\nRequires: python2 >= 2.6
++deb.dependency.list=openssl, zlibc, python (>= 2.6)
+diff --git a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+index 799cf3eed0..ade58292cb 100644
+--- a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
++++ b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+@@ -359,9 +359,8 @@ class RPMBasedPackageProvider(PackageProvider):
+ INSTALLED_PACKAGE_VERSION_COMMAND = "rpm -q --queryformat '%{{version}}-%{{release}}' \"{0}\""
+
+ def rpm_check_package_available(self, name):
+- import rpm # this is faster then calling 'rpm'-binary externally.
+- ts = rpm.TransactionSet()
+- packages = ts.dbMatch()
++ import os
++ packages = os.popen("rpm -qa --queryformat '%{name} '").read().split()
Review comment:
That's right, thanks! I fixed the succeeding for-loop.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] masatana commented on pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
masatana commented on pull request #656:
URL: https://github.com/apache/bigtop/pull/656#issuecomment-665539868
@sekikn Thanks for working on this! I haven't tested this patch yet, but I left one comment.
I'll test this patch on my environment later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] masatana commented on a change in pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
masatana commented on a change in pull request #656:
URL: https://github.com/apache/bigtop/pull/656#discussion_r462151638
##########
File path: bigtop-packages/src/common/ambari/patch4-AMBARI-CentOS8.diff
##########
@@ -0,0 +1,59 @@
+diff --git a/ambari-agent/src/main/package/dependencies.properties b/ambari-agent/src/main/package/dependencies.properties
+index 07b0b68319..9cfc2daeb5 100644
+--- a/ambari-agent/src/main/package/dependencies.properties
++++ b/ambari-agent/src/main/package/dependencies.properties
+@@ -28,5 +28,5 @@
+ # Such a format is respected by install_ambari_tarball.py by default,
+ # however should be encouraged manually in pom.xml.
+
+-rpm.dependency.list=openssl,\nRequires: rpm-python,\nRequires: zlib,\nRequires: python >= 2.6
+-deb.dependency.list=openssl, zlibc, python (>= 2.6)
+\ No newline at end of file
++rpm.dependency.list=openssl,\nRequires: zlib,\nRequires: python2 >= 2.6
++deb.dependency.list=openssl, zlibc, python (>= 2.6)
+diff --git a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+index 799cf3eed0..ade58292cb 100644
+--- a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
++++ b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+@@ -359,9 +359,8 @@ class RPMBasedPackageProvider(PackageProvider):
+ INSTALLED_PACKAGE_VERSION_COMMAND = "rpm -q --queryformat '%{{version}}-%{{release}}' \"{0}\""
+
+ def rpm_check_package_available(self, name):
+- import rpm # this is faster then calling 'rpm'-binary externally.
+- ts = rpm.TransactionSet()
+- packages = ts.dbMatch()
++ import os
++ packages = os.popen("rpm -qa --queryformat '%{name} '").read().split()
Review comment:
`packages` variable should contain a dict-like object as this is used like [this](https://github.com/apache/ambari/blob/release-2.6.1/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py#L370)
##########
File path: bigtop-packages/src/common/ambari/patch4-AMBARI-CentOS8.diff
##########
@@ -0,0 +1,59 @@
+diff --git a/ambari-agent/src/main/package/dependencies.properties b/ambari-agent/src/main/package/dependencies.properties
+index 07b0b68319..9cfc2daeb5 100644
+--- a/ambari-agent/src/main/package/dependencies.properties
++++ b/ambari-agent/src/main/package/dependencies.properties
+@@ -28,5 +28,5 @@
+ # Such a format is respected by install_ambari_tarball.py by default,
+ # however should be encouraged manually in pom.xml.
+
+-rpm.dependency.list=openssl,\nRequires: rpm-python,\nRequires: zlib,\nRequires: python >= 2.6
+-deb.dependency.list=openssl, zlibc, python (>= 2.6)
+\ No newline at end of file
++rpm.dependency.list=openssl,\nRequires: zlib,\nRequires: python2 >= 2.6
++deb.dependency.list=openssl, zlibc, python (>= 2.6)
+diff --git a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+index 799cf3eed0..ade58292cb 100644
+--- a/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
++++ b/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
+@@ -359,9 +359,8 @@ class RPMBasedPackageProvider(PackageProvider):
+ INSTALLED_PACKAGE_VERSION_COMMAND = "rpm -q --queryformat '%{{version}}-%{{release}}' \"{0}\""
+
+ def rpm_check_package_available(self, name):
+- import rpm # this is faster then calling 'rpm'-binary externally.
+- ts = rpm.TransactionSet()
+- packages = ts.dbMatch()
++ import os
++ packages = os.popen("rpm -qa --queryformat '%{name} '").read().split()
Review comment:
`packages` variable should contain a dict-like object as this is used like [this](https://github.com/apache/ambari/blob/release-2.6.1/ambari-common/src/main/python/resource_management/core/providers/package/__init__.py#L370) ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] masatana commented on pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
masatana commented on pull request #656:
URL: https://github.com/apache/bigtop/pull/656#issuecomment-666764147
LGTM (non-binding). I also built (`./gradlew toolchain` && `./gradlew ambari-pkg`) and installed on CentOS8 with success.
If we encounter any problems with running Ambari on CentOS8, let's fix it with another addendum patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #656:
URL: https://github.com/apache/bigtop/pull/656#issuecomment-666768379
Thank you for the review @masatana @iwasakims, just merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn merged pull request #656: BIGTOP-3371. Unable to build Apache Ambari on RHEL8.
Posted by GitBox <gi...@apache.org>.
sekikn merged pull request #656:
URL: https://github.com/apache/bigtop/pull/656
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org