You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2021/08/20 00:02:36 UTC

[GitHub] [trafficcontrol] zrhoffman opened a new pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

zrhoffman opened a new pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128


   <!--
   Thank you for contributing! Please be sure to read our contribution guidelines: https://github.com/apache/trafficcontrol/blob/master/CONTRIBUTING.md
   If this closes or relates to an existing issue, please reference it using one of the following:
   
   Closes: #ISSUE
   Related: #ISSUE
   
   If this PR fixes a security vulnerability, DO NOT submit! Instead, contact
   the Apache Traffic Control Security Team at security@trafficcontrol.apache.org and follow the
   guidelines at https://apache.org/security regarding vulnerability disclosure.
   -->
   This PR
   - Fixes an issue introduced by #6096 where, where when migrating from Goose, `db/admin dbversion` would return `0` instead of Goose migration version (e080781e61)
   - Fixes the issue that #6096 was supposed to fix: Running the first unsquashed migration without checking whether the past migration exists if migrating from the last squashed migration (f102c654ff).
   - Fixes a bug in the Traffic Ops DB tests where `dropdb` did not specify the database to drop
   
   
   <!-- **^ Add meaningful description above** --><hr>
   
   ## Which Traffic Control components are affected by this PR?
   <!-- Please delete all components from this list that are NOT affected by this PR.
   Feel free to add the name of a tool or script that is affected but not on the list.
   -->
   - Traffic Ops
   
   ## What is the best way to verify this PR?
   <!-- Please include here ALL the steps necessary to test your PR.
   If your PR has tests (and most should), provide the steps needed to run the tests.
   If not, please provide step-by-step instructions to test the PR manually and explain why your PR does not need tests. -->
   1. Delete the `2021080408053529_fix_indices` migrations (unnecessary once #6127 is fixed)
   2. Migrate the Traffic Ops database at [ATC 5.1.2](https://github.com/apache/trafficcontrol/commits/RELEASE-5.1.2)
   3. Dump the database, move the dump to `traffic_ops_db/test/docker/initdb.d`, and rebuild the `db` service
   4. Build a Traffic Ops RPM, move it to `traffic_ops_db/test/docker/traffic_ops.rpm` and rebuild the `trafficops-db-admin` service
   5. Run the Traffic Ops DB tests
   
   ## If this is a bugfix, which Traffic Control versions contained the bug?
   <!-- Delete this section if the PR is not a bugfix, or if the bug is only in the master branch.
   Examples:
   - 5.1.2
   - 5.1.3 (RC1)
    -->
   - 6.0.x release branch
   
   ## PR submission checklist
   - [x] This PR modifies existing tests<!-- If not, please delete this text and explain why this PR does not need tests. -->
   - [ ] This PR has documentation <!-- If not, please delete this text and explain why this PR does not need documentation. -->
   - [ ] This PR has a CHANGELOG.md entry <!-- A fix for a bug from an ATC release, an improvement, or a new feature should have a changelog entry. -->
   - [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://apache.org/security) for details)
   
   <!--
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.
   -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#discussion_r694357735



##########
File path: traffic_ops_db/test/docker/Dockerfile-db-admin
##########
@@ -23,25 +23,43 @@ FROM centos:7
 ARG POSTGRES_VERSION=13.2
 ENV POSTGRES_VERSION=$POSTGRES_VERSION
 
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
 RUN yum install -y \
-    epel-release \
-    centos-release-scl-rh \
-    https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+		epel-release \
+		centos-release-scl-rh \
+		https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
     git && \
-    yum -y install golang
+    yum -y install           \
+        cpanminus            \
+        expat-devel          \
+        libcap               \
+        libcurl-devel        \
+        libidn-devel         \
+        libpcap-devel        \
+        mkisofs              \
+        openssl-devel        \
+        perl-core            \
+        perl-Crypt-ScryptKDF \
+        perl-DBD-Pg          \
+        perl-DBI             \
+        perl-Digest-SHA1     \
+        perl-JSON            \
+        perl-libwww-perl     \
+        perl-TermReadKey     \
+        perl-Test-CPAN-Meta  \
+        perl-WWW-Curl        \
+        postgresql13         \
+        postgresql13-devel &&\
+    yum clean all
 
 # Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=...  Can be local file or http://...
 ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
 ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
-        /$(basename $TRAFFIC_OPS_RPM) && \
-        rm /$(basename $TRAFFIC_OPS_RPM) && \
-    yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+    rm $(basename $TRAFFIC_OPS_RPM)

Review comment:
       Oh ok so `rpm` will still install the required dependencies even if they change in the RPM but aren't updated in the packages above?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] zrhoffman commented on pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#issuecomment-904180499


   > Uh oh, looks like the DB tests will fail perpetually until they're fixed to ignore the migration that was edited.
   
   cf59b77ede: No longer failing if that migration is not changed in the PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#discussion_r694355842



##########
File path: traffic_ops_db/test/docker/Dockerfile-db-admin
##########
@@ -23,25 +23,43 @@ FROM centos:7
 ARG POSTGRES_VERSION=13.2
 ENV POSTGRES_VERSION=$POSTGRES_VERSION
 
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
 RUN yum install -y \
-    epel-release \
-    centos-release-scl-rh \
-    https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+		epel-release \
+		centos-release-scl-rh \
+		https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
     git && \
-    yum -y install golang
+    yum -y install           \
+        cpanminus            \
+        expat-devel          \
+        libcap               \
+        libcurl-devel        \
+        libidn-devel         \
+        libpcap-devel        \
+        mkisofs              \
+        openssl-devel        \
+        perl-core            \
+        perl-Crypt-ScryptKDF \
+        perl-DBD-Pg          \
+        perl-DBI             \
+        perl-Digest-SHA1     \
+        perl-JSON            \
+        perl-libwww-perl     \
+        perl-TermReadKey     \
+        perl-Test-CPAN-Meta  \
+        perl-WWW-Curl        \
+        postgresql13         \
+        postgresql13-devel &&\
+    yum clean all
 
 # Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=...  Can be local file or http://...
 ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
 ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
-        /$(basename $TRAFFIC_OPS_RPM) && \
-        rm /$(basename $TRAFFIC_OPS_RPM) && \
-    yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+    rm $(basename $TRAFFIC_OPS_RPM)

Review comment:
       Installing the TO RPM dependencies in a layer before installing the TO RPM is just an optimization to allow a user to rebuild the `trafficops-db-admin` image with different TO RPMs without needing to download all the Yum packages every time.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#discussion_r694359121



##########
File path: traffic_ops_db/test/docker/Dockerfile-db-admin
##########
@@ -23,25 +23,43 @@ FROM centos:7
 ARG POSTGRES_VERSION=13.2
 ENV POSTGRES_VERSION=$POSTGRES_VERSION
 
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
 RUN yum install -y \
-    epel-release \
-    centos-release-scl-rh \
-    https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+		epel-release \
+		centos-release-scl-rh \
+		https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
     git && \
-    yum -y install golang
+    yum -y install           \
+        cpanminus            \
+        expat-devel          \
+        libcap               \
+        libcurl-devel        \
+        libidn-devel         \
+        libpcap-devel        \
+        mkisofs              \
+        openssl-devel        \
+        perl-core            \
+        perl-Crypt-ScryptKDF \
+        perl-DBD-Pg          \
+        perl-DBI             \
+        perl-Digest-SHA1     \
+        perl-JSON            \
+        perl-libwww-perl     \
+        perl-TermReadKey     \
+        perl-Test-CPAN-Meta  \
+        perl-WWW-Curl        \
+        postgresql13         \
+        postgresql13-devel &&\
+    yum clean all
 
 # Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=...  Can be local file or http://...
 ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
 ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
-        /$(basename $TRAFFIC_OPS_RPM) && \
-        rm /$(basename $TRAFFIC_OPS_RPM) && \
-    yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+    rm $(basename $TRAFFIC_OPS_RPM)

Review comment:
       It will not, but if those dependencies are not satisfied when `rpm` runs, `rpm` will fail.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#discussion_r694360748



##########
File path: traffic_ops_db/test/docker/Dockerfile-db-admin
##########
@@ -23,25 +23,43 @@ FROM centos:7
 ARG POSTGRES_VERSION=13.2
 ENV POSTGRES_VERSION=$POSTGRES_VERSION
 
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
 RUN yum install -y \
-    epel-release \
-    centos-release-scl-rh \
-    https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+		epel-release \
+		centos-release-scl-rh \
+		https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
     git && \
-    yum -y install golang
+    yum -y install           \
+        cpanminus            \
+        expat-devel          \
+        libcap               \
+        libcurl-devel        \
+        libidn-devel         \
+        libpcap-devel        \
+        mkisofs              \
+        openssl-devel        \
+        perl-core            \
+        perl-Crypt-ScryptKDF \
+        perl-DBD-Pg          \
+        perl-DBI             \
+        perl-Digest-SHA1     \
+        perl-JSON            \
+        perl-libwww-perl     \
+        perl-TermReadKey     \
+        perl-Test-CPAN-Meta  \
+        perl-WWW-Curl        \
+        postgresql13         \
+        postgresql13-devel &&\
+    yum clean all
 
 # Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=...  Can be local file or http://...
 ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
 ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
-        /$(basename $TRAFFIC_OPS_RPM) && \
-        rm /$(basename $TRAFFIC_OPS_RPM) && \
-    yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+    rm $(basename $TRAFFIC_OPS_RPM)

Review comment:
       Ok, good to know. So it will be obvious when the TO RPM dependencies change that this list will need to be updated as well. Thanks!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#discussion_r694347381



##########
File path: traffic_ops_db/test/docker/Dockerfile-db-admin
##########
@@ -23,25 +23,43 @@ FROM centos:7
 ARG POSTGRES_VERSION=13.2
 ENV POSTGRES_VERSION=$POSTGRES_VERSION
 
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
 RUN yum install -y \
-    epel-release \
-    centos-release-scl-rh \
-    https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+		epel-release \
+		centos-release-scl-rh \
+		https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
     git && \
-    yum -y install golang
+    yum -y install           \
+        cpanminus            \
+        expat-devel          \
+        libcap               \
+        libcurl-devel        \
+        libidn-devel         \
+        libpcap-devel        \
+        mkisofs              \
+        openssl-devel        \
+        perl-core            \
+        perl-Crypt-ScryptKDF \
+        perl-DBD-Pg          \
+        perl-DBI             \
+        perl-Digest-SHA1     \
+        perl-JSON            \
+        perl-libwww-perl     \
+        perl-TermReadKey     \
+        perl-Test-CPAN-Meta  \
+        perl-WWW-Curl        \
+        postgresql13         \
+        postgresql13-devel &&\
+    yum clean all
 
 # Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=...  Can be local file or http://...
 ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
 ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
-        /$(basename $TRAFFIC_OPS_RPM) && \
-        rm /$(basename $TRAFFIC_OPS_RPM) && \
-    yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+    rm $(basename $TRAFFIC_OPS_RPM)

Review comment:
       Just for my own edification -- why was this change necessary? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] rawlinp commented on pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#issuecomment-904174769


   Uh oh, looks like the DB tests will fail perpetually until they're fixed to ignore the migration that was edited.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] zrhoffman commented on pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128#issuecomment-904173373


   Rebased to get the fix for #6127. Removing the `2021080408053529_fix_indices` migration is no longer necessary to test this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [trafficcontrol] rawlinp merged pull request #6128: Skip check for the past dbversion's migration existing if it is squashed

Posted by GitBox <gi...@apache.org>.
rawlinp merged pull request #6128:
URL: https://github.com/apache/trafficcontrol/pull/6128


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org