You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by dg4prez <gi...@git.apache.org> on 2017/02/22 17:56:36 UTC
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate
GitHub user dg4prez opened a pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299
Instant Invalidate
This code change adds a revalidate command line option to ort which will only check the regex_revalidate file. This option requires the use of the reval_pending value, which is enabled by setting the new parameter "use_reval_pending" in global parameters to "1" (the default is zero.) If this value is not found, this option will gracefully quit and syncds should be used. If reval_pending is in use then revalidate will verify that no parents are pending and run. Should syncds be run, it will also verify that no parents have reval_pending flagged in addition to the normal "upd_pending" flag, preventing revalidation in incorrect order.
Because this only checks and modifies the regex_revalidate.config file, this can be run via ort more frequently than normal syncds flags.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dg4prez/incubator-trafficcontrol regex_reval
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafficcontrol/pull/299.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #299
----
commit 3bd443e49deb93ebad22d0f3e0f7aa95947a397a
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T00:27:16Z
initial ORT changes
commit cacfd892e2f8e074e7574792f742b7780077c9a8
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T00:38:30Z
add upd_pending to server table
commit 766c226f7dad9e2167997fb75f11ee760ef87ede
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T01:21:25Z
add reval_pending and parent_reval_pending to server update page
commit f8b725571adae8fcf049046921a67b265628870e
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T02:21:48Z
one possible method of selecting between reval and upd
commit 58388bff7e85d84e71b9010217ec7599af068f1d
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T17:39:40Z
add reval_pending parameter and modify logic slightly
commit 86ceaba0c1e6f15f56ce9fe2a5340de15077f5ab
Author: Derek Gelinas <de...@cable.comcast.com>
Date: 2017-02-22T17:42:38Z
prevent syncds from running if parent_reval_pending is flagged
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106325223
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -646,18 +713,28 @@ sub update_trops {
}
}
elsif ( $script_mode == $BADASS || $script_mode == $SYNCDS ) {
- &send_update_to_trops($CLEAR);
+ if ( defined $reval_pending ) {
+ &send_update_to_trops($CLEAR, $reval_pending );
+ }
+ else {
+ &send_update_to_trops($CLEAR, $CLEAR);
+ }
+ }
+ elsif ( $script_mode = $REVALIDATE ) {
+ &send_update_to_trops($upd_pending, $CLEAR);
}
}
}
sub send_update_to_trops {
my $status = shift;
- my $url = "$traffic_ops_host\/update/$hostname_short";
+ my $reval_status = shift;
+ my $uri = "/update/$hostname_short";
( $log_level >> $DEBUG ) && print "DEBUG Setting update flag in Traffic Ops to $status.\n";
my %headers = ( 'Cookie' => $cookie );
- my $response = $lwp_conn->post( $url, [ 'updated' => $status ], %headers );
+ my $url = $traffic_ops_host . $uri;
+ my $response = $lwp_conn->post( $url, [ 'updated' => $status, 'reval_updated' => $reval_status ], %headers );
--- End diff --
is it ok if reval_status is null here? Didnt trace the code, I just assume that can happen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol issue #299: Instant Invalidate for ORT and Traffic ...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on the issue:
https://github.com/apache/incubator-trafficcontrol/pull/299
Ah brought this up with him the other day. IMO we should add a GLOBAL profile to the test DB via fixtures. I added it manually and it worked.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106294548
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -262,6 +316,7 @@ sub usage {
print "\t<Mode> = report - prints config differences and exits.\n";
print "\t<Mode> = badass - attempts to fix all config differences that it can.\n";
print "\t<Mode> = syncds - syncs delivery services with what is configured in Traffic Ops.\n";
+ print "\t<Mode> = revalidate - checks for updated revalidations in Traffic Ops and applies them. Requires Traffic Ops 1.2.\n";
--- End diff --
requires traffic ops 2.1 not 1.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106271956
--- Diff: traffic_ops/app/lib/API/Configs/ApacheTrafficServer.pm ---
@@ -76,7 +76,9 @@ sub get_config_metadata {
$data_obj->{'info'}->{'cdn_name'} = $cdn_name;
$data_obj->{'info'}->{'cdn_id'} = $server->cdn->id;
$data_obj->{'info'}->{'tm_url'} = $tm_url;
- $data_obj->{'info'}->{'tm_cache_url'} = $tm_cache_url;
+ if ( $tm_cache_url ) {
--- End diff --
Agreed. Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106265336
--- Diff: traffic_ops/app/lib/MojoPlugins/Job.pm ---
@@ -70,8 +70,16 @@ sub register {
}
);
- my $result = $update_server_bit_rs->update( { upd_pending => 1 } );
- &log( $self, "Set upd_pending = 1 for all applicable caches", "OPER" );
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
--- End diff --
isn't it
&& $use_reval_pending == '1'
but i dunno. maybe perl is smart enough to figure it out...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106412630
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -226,6 +263,7 @@
}
}
+
--- End diff --
Maybe we could use metal for the roof of the bikeshed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: WIP - Instant Invalidate
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r102823888
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -977,7 +1017,16 @@ sub postupdate {
my $update_server =
$self->db->resultset('Server')->search( { id => $serverid } );
- if ( defined $updated ) {
+
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
+ print STDERR "updating upd_pending and reval_pending\n";
--- End diff --
Definitely. Must have missed those.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106449659
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -646,18 +713,28 @@ sub update_trops {
}
}
elsif ( $script_mode == $BADASS || $script_mode == $SYNCDS ) {
- &send_update_to_trops($CLEAR);
+ if ( defined $reval_pending ) {
+ &send_update_to_trops($CLEAR, $reval_pending );
+ }
+ else {
+ &send_update_to_trops($CLEAR, $CLEAR);
+ }
+ }
+ elsif ( $script_mode = $REVALIDATE ) {
+ &send_update_to_trops($upd_pending, $CLEAR);
}
}
}
sub send_update_to_trops {
my $status = shift;
- my $url = "$traffic_ops_host\/update/$hostname_short";
+ my $reval_status = shift;
+ my $uri = "/update/$hostname_short";
( $log_level >> $DEBUG ) && print "DEBUG Setting update flag in Traffic Ops to $status.\n";
my %headers = ( 'Cookie' => $cookie );
- my $response = $lwp_conn->post( $url, [ 'updated' => $status ], %headers );
+ my $url = $traffic_ops_host . $uri;
+ my $response = $lwp_conn->post( $url, [ 'updated' => $status, 'reval_updated' => $reval_status ], %headers );
--- End diff --
\U0001f44d
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106326232
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -759,9 +967,9 @@ sub check_syncds_state {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no update is waiting to be applied.\n";
}
- my $stj = &lwp_get("$traffic_ops_host\/datastatus");
+ my $stj = &lwp_get("/datastatus");
--- End diff --
again, we should use the API endpoint here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106325936
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -671,31 +748,128 @@ sub get_print_current_client_connections {
( $log_level >> $DEBUG ) && print "DEBUG There are currently $current_connections connections.\n";
}
+sub check_revalidate_state {
+ my $sleep_override = shift;
+
+ my $syncds_update = 0;
+
+ ( $log_level >> $DEBUG ) && print "DEBUG Checking revalidate state.\n";
+ if ( $script_mode == $REVALIDATE || $sleep_override == 1 ) {
+ ## The herd is about to get /update/<hostname>
+
+ my $uri = "/update/$hostname_short";
+ my $upd_ref = &lwp_get($uri);
+ if ( $upd_ref =~ m/^\d{3}$/ ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri returned $upd_ref. Exiting, not sure what else to do.\n";
+ exit 1;
+ }
+
+ my $upd_json = decode_json($upd_ref);
+ my $reval_pending = ( defined( $upd_json->[0]->{'reval_pending'} ) ) ? $upd_json->[0]->{'reval_pending'} : undef;
+ if ( !defined($reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+
+ if ( $reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that a revalidation is waiting to be applied.\n";
+ $syncds_update = $UPDATE_TROPS_NEEDED;
+
+ my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
+ if ( !defined($parent_reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1. Unable to continue!\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ if ( $parent_reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need to revalidate.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ }
+ elsif ( $script_mode == $REVALIDATE && $reval_pending != 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR In revalidate mode, but no update needs to be applied. I'm outta here.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ else {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no revalidations are waiting to be applied.\n";
+ }
+
+ my $stj = &lwp_get("/datastatus");
--- End diff --
/datastatus will be depricated before too long, could you use /api/1.2/statuses instead?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106283866
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -980,7 +1018,14 @@ sub postupdate {
my $update_server =
$self->db->resultset('Server')->search( { id => $serverid } );
- if ( defined $updated ) {
+
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
--- End diff --
Adjusted per discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106326536
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -2151,13 +2453,22 @@ sub validate_result {
}
}
-sub set_url {
+sub set_uri {
my $filename = shift;
+
my $filepath = $cfg_file_tracker->{$filename}->{'location'};
+ my $URI;
+ if ( $api_in_use == 1 && defined($cfg_file_tracker->{$filename}->{'API_URI'}) ) {
+ $URI = $cfg_file_tracker->{$filename}->{'API_URI'};
+ }
+ else {
+ $URI = "\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
+ }
return if (!defined($cfg_file_tracker->{$filename}->{'fname-in-TO'}));
- return "$traffic_ops_host\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
+ #return "$traffic_ops_host\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
--- End diff --
do we need this commented out line or can it be removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106326442
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -1481,8 +1783,8 @@ sub get_header_comment {
my $to_host = shift;
my $toolname;
- my $url = "$to_host/api/1.1/system/info.json";
- my $result = &lwp_get($url);
+ my $uri = "/api/1.1/system/info.json";
--- End diff --
can we use the 1.2 version if the API instead? Not sure if it changed at all.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106412378
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -177,32 +184,62 @@
#### Start main flow
####-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-####
-#### First and foremost, if this is a syncds run, check to see if we can bail.
-my $syncds_update = undef;
-if ( defined $traffic_ops_host ) {
- ($syncds_update) = &check_syncds_state();
-}
-else {
- print "FATAL Could not resolve Traffic Ops host!\n";
- exit 1;
-}
#### Delete /tmp dirs older than one week
-if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS ) {
+if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS || $script_mode == $REVALIDATE ) {
&smart_mkdir($TMP_BASE);
&clean_tmp_dirs();
}
-( my $my_profile_name, $cfg_file_tracker, my $my_cdn_name ) = &get_cfg_file_list( $hostname_short, $traffic_ops_host );
my $header_comment = &get_header_comment($traffic_ops_host);
-&process_packages( $hostname_short, $traffic_ops_host );
-# get the ats user's UID after package installation in case this is the initial badass
my $ats_uid = getpwnam("ats");
-&process_chkconfig( $hostname_short, $traffic_ops_host );
+
+#### If this is a syncds run, check to see if we can bail.
+my $syncds_update = 0;
+if ( $script_mode == $REVALIDATE ) {
--- End diff --
Good call.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106264761
--- Diff: traffic_ops/app/lib/API/Configs/ApacheTrafficServer.pm ---
@@ -76,7 +76,9 @@ sub get_config_metadata {
$data_obj->{'info'}->{'cdn_name'} = $cdn_name;
$data_obj->{'info'}->{'cdn_id'} = $server->cdn->id;
$data_obj->{'info'}->{'tm_url'} = $tm_url;
- $data_obj->{'info'}->{'tm_cache_url'} = $tm_cache_url;
+ if ( $tm_cache_url ) {
--- End diff --
should this be called to_cache_url instead? it would be nice to get rid of all references to 'tm' at some point if possible...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106284146
--- Diff: traffic_ops/app/lib/MojoPlugins/Job.pm ---
@@ -70,8 +70,16 @@ sub register {
}
);
- my $result = $update_server_bit_rs->update( { upd_pending => 1 } );
- &log( $self, "Set upd_pending = 1 for all applicable caches", "OPER" );
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
--- End diff --
Adjusted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106414628
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -759,9 +967,9 @@ sub check_syncds_state {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no update is waiting to be applied.\n";
}
- my $stj = &lwp_get("$traffic_ops_host\/datastatus");
+ my $stj = &lwp_get("/datastatus");
--- End diff --
Agreed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106449223
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -177,32 +184,58 @@
#### Start main flow
####-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-####
-#### First and foremost, if this is a syncds run, check to see if we can bail.
-my $syncds_update = undef;
-if ( defined $traffic_ops_host ) {
- ($syncds_update) = &check_syncds_state();
-}
-else {
- print "FATAL Could not resolve Traffic Ops host!\n";
- exit 1;
-}
#### Delete /tmp dirs older than one week
-if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS ) {
+if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS || $script_mode == $REVALIDATE ) {
&smart_mkdir($TMP_BASE);
&clean_tmp_dirs();
}
-( my $my_profile_name, $cfg_file_tracker, my $my_cdn_name ) = &get_cfg_file_list( $hostname_short, $traffic_ops_host );
my $header_comment = &get_header_comment($traffic_ops_host);
-&process_packages( $hostname_short, $traffic_ops_host );
-# get the ats user's UID after package installation in case this is the initial badass
my $ats_uid = getpwnam("ats");
-&process_chkconfig( $hostname_short, $traffic_ops_host );
+
+if ( !defined $traffic_ops_host ) {
+ print "FATAL Could not resolve Traffic Ops host!\n";
+ exit 1;
+}
+
+#### If this is a syncds run, check to see if we can bail.
+my $syncds_update = 0;
+
+if ( $script_mode == $REVALIDATE ) {
+ ( $syncds_update ) = &check_revalidate_state();
+ if ( $syncds_update < 1 ) {
+ exit 1;
+ }
+}
+else {
+ ( $syncds_update ) = &check_syncds_state();
+ if ( $syncds_update < 1 ) {
--- End diff --
We aren't doing anything in this if{}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106263950
--- Diff: traffic_ops/app/db/migrations/20170221190747_add_reval_pending.sql ---
@@ -0,0 +1,26 @@
+/*
+
--- End diff --
this migration file needs a later timestamp on the name. see existing migrations - https://github.com/apache/incubator-trafficcontrol/tree/master/traffic_ops/app/db/migrations
it needs to be timestamped later than the add_client_steering migration...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: WIP - Instant Invalidate
Posted by mtorluemke <gi...@git.apache.org>.
Github user mtorluemke commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r102822502
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -977,7 +1017,16 @@ sub postupdate {
my $update_server =
$self->db->resultset('Server')->search( { id => $serverid } );
- if ( defined $updated ) {
+
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
+ print STDERR "updating upd_pending and reval_pending\n";
--- End diff --
Here too..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106414587
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -705,50 +879,84 @@ sub check_syncds_state {
$syncds_update = $UPDATE_TROPS_NEEDED;
my $parent_pending = ( defined( $upd_json->[0]->{'parent_pending'} ) ) ? $upd_json->[0]->{'parent_pending'} : undef;
+ my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
if ( !defined($parent_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $url did not have an parent_pending key.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_pending key.\n";
if ( $script_mode != $SYNCDS ) {
return $syncds_update;
}
else {
- ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $url. Exiting, not sure what else to do.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $uri. Exiting, not sure what else to do.\n";
exit 1;
}
}
- if ( $parent_pending == 1 && $wait_for_parents == 1) {
- ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need an update.\n";
- if ( $script_mode == $SYNCDS ) {
- if ( $dispersion > 0 ) {
- ( $log_level >> $WARN ) && print "WARN In syncds mode, sleeping for " . $dispersion . "s to see if the update my parents need is cleared.\n";
- for ( my $i = $dispersion; $i > 0; $i-- ) {
- ( $log_level >> $WARN ) && print ".";
- sleep 1;
+ if ( defined($parent_reval_pending) ) {
+ if ( ( $parent_pending == 1 || $parent_reval_pending == 1 ) && $wait_for_parents == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need an update.\n";
+ if ( $script_mode == $SYNCDS ) {
+ if ( $dispersion > 0 ) {
+ ( $log_level >> $WARN ) && print "WARN In syncds mode, sleeping for " . $dispersion . "s to see if the update my parents need is cleared.\n";
+ ( $dispersion > 0 ) && &sleep_timer($dispersion);
+ }
+
+ ( $log_level >> $WARN ) && print "\n";
--- End diff --
I'm a really big fan of carriage returns for some reason.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106269894
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -980,7 +1018,14 @@ sub postupdate {
my $update_server =
$self->db->resultset('Server')->search( { id => $serverid } );
- if ( defined $updated ) {
+
+ my $use_reval_pending = $self->db->resultset('Parameter')->search( { -and => [ 'name' => 'use_reval_pending', 'config_file' => 'global' ] } )->get_column('value')->single;
+
+ if ( defined($use_reval_pending) && $use_reval_pending == 1 ) {
--- End diff --
you can save a few lines of code here by getting rid of the else and moving
$update_server->update( { upd_pending => $updated } );
below the remaining if...and also leave it out of the if...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106415082
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -2151,13 +2453,22 @@ sub validate_result {
}
}
-sub set_url {
+sub set_uri {
my $filename = shift;
+
my $filepath = $cfg_file_tracker->{$filename}->{'location'};
+ my $URI;
+ if ( $api_in_use == 1 && defined($cfg_file_tracker->{$filename}->{'API_URI'}) ) {
+ $URI = $cfg_file_tracker->{$filename}->{'API_URI'};
+ }
+ else {
+ $URI = "\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
+ }
return if (!defined($cfg_file_tracker->{$filename}->{'fname-in-TO'}));
- return "$traffic_ops_host\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
+ #return "$traffic_ops_host\/genfiles\/view\/$hostname_short\/" . $cfg_file_tracker->{$filename}->{'fname-in-TO'};
--- End diff --
Let's get rid of it!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: WIP - Instant Invalidate
Posted by mtorluemke <gi...@git.apache.org>.
Github user mtorluemke commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r102822448
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -949,7 +985,11 @@ sub postupdate {
my $self = shift;
my $updated = $self->param("updated");
+ my $reval_updated = $self->param("reval_updated");
my $host_name = $self->param("host_name");
+ print STDERR Dumper($updated);
--- End diff --
Should take these out, yea?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol issue #299: Instant Invalidate for ORT and Traffic ...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on the issue:
https://github.com/apache/incubator-trafficcontrol/pull/299
\U0001f44d I checked ORT, looks good to me now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106414329
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -671,31 +748,128 @@ sub get_print_current_client_connections {
( $log_level >> $DEBUG ) && print "DEBUG There are currently $current_connections connections.\n";
}
+sub check_revalidate_state {
+ my $sleep_override = shift;
+
+ my $syncds_update = 0;
+
+ ( $log_level >> $DEBUG ) && print "DEBUG Checking revalidate state.\n";
+ if ( $script_mode == $REVALIDATE || $sleep_override == 1 ) {
+ ## The herd is about to get /update/<hostname>
+
+ my $uri = "/update/$hostname_short";
+ my $upd_ref = &lwp_get($uri);
+ if ( $upd_ref =~ m/^\d{3}$/ ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri returned $upd_ref. Exiting, not sure what else to do.\n";
+ exit 1;
+ }
+
+ my $upd_json = decode_json($upd_ref);
+ my $reval_pending = ( defined( $upd_json->[0]->{'reval_pending'} ) ) ? $upd_json->[0]->{'reval_pending'} : undef;
+ if ( !defined($reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+
+ if ( $reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that a revalidation is waiting to be applied.\n";
+ $syncds_update = $UPDATE_TROPS_NEEDED;
+
+ my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
+ if ( !defined($parent_reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1. Unable to continue!\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ if ( $parent_reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need to revalidate.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ }
+ elsif ( $script_mode == $REVALIDATE && $reval_pending != 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR In revalidate mode, but no update needs to be applied. I'm outta here.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ else {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no revalidations are waiting to be applied.\n";
+ }
+
+ my $stj = &lwp_get("/datastatus");
--- End diff --
yup.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106283782
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -915,20 +916,53 @@ sub readupdate {
{
$parent_pending{ $rs_servers->single->host_name } = 1;
}
+ if ( $prow->reval_pending == 1
+ && $prow->status->name ne "OFFLINE" )
+ {
+ $parent_reval_pending{ $rs_servers->single->host_name } = 1;
+ }
}
}
}
}
while ( my $row = $rs_servers->next ) {
- if ( $parent_pending{ $row->host_name } ) {
+ if ( $parent_pending{ $row->host_name } && $parent_reval_pending{ $row->host_name } ) {
--- End diff --
Agreed. Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106293453
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -177,32 +184,62 @@
#### Start main flow
####-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-####
-#### First and foremost, if this is a syncds run, check to see if we can bail.
-my $syncds_update = undef;
-if ( defined $traffic_ops_host ) {
- ($syncds_update) = &check_syncds_state();
-}
-else {
- print "FATAL Could not resolve Traffic Ops host!\n";
- exit 1;
-}
#### Delete /tmp dirs older than one week
-if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS ) {
+if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode == $SYNCDS || $script_mode == $REVALIDATE ) {
&smart_mkdir($TMP_BASE);
&clean_tmp_dirs();
}
-( my $my_profile_name, $cfg_file_tracker, my $my_cdn_name ) = &get_cfg_file_list( $hostname_short, $traffic_ops_host );
my $header_comment = &get_header_comment($traffic_ops_host);
-&process_packages( $hostname_short, $traffic_ops_host );
-# get the ats user's UID after package installation in case this is the initial badass
my $ats_uid = getpwnam("ats");
-&process_chkconfig( $hostname_short, $traffic_ops_host );
+
+#### If this is a syncds run, check to see if we can bail.
+my $syncds_update = 0;
+if ( $script_mode == $REVALIDATE ) {
--- End diff --
This seems like it can use some cleanup. Can you do the check to see if $traffic_ops_host is defined first and then check the script mode?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106414781
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -815,15 +1023,48 @@ sub check_syncds_state {
return ($syncds_update);
}
-sub sleep_rand {
- my $duration = int( rand(shift) );
-
- ( $log_level >> $WARN ) && print "WARN Sleeping for $duration seconds: ";
-
+sub sleep_timer {
+ my $duration = shift;
+ my $reval_clock = $reval_wait_time;
+
+ my $proper_script_mode = $script_mode;
+
+ if ( $reval_in_use == 1 ) {
+ ( $log_level >> $WARN ) && print "WARN Performing a revalidation check before sleeping... \n";
+ &revalidate_while_sleeping();
+ ( $log_level >> $WARN ) && print "WARN Revalidation check complete.\n";
+ }
+ if ( $duration < $reval_clock || $reval_in_use == 0 ) {
+ ( $log_level >> $WARN ) && print "WARN Sleeping for $duration seconds: ";
+ }
+ else {
+ ( $log_level >> $WARN ) && print "WARN $reval_clock seconds until next revalidation check.\n";
+ ( $log_level >> $WARN ) && print "WARN $duration seconds remaining in dispersion sleep period\n";
+ ( $log_level >> $WARN ) && print "WARN Sleeping for $reval_clock seconds: ";
+ }
for ( my $i = $duration; $i > 0; $i-- ) {
( $log_level >> $WARN ) && print ".";
sleep 1;
+ $reval_clock--;
+ if ($reval_clock < 1 && $script_mode != $BADASS && $reval_in_use == 1 ) {
+ ( $log_level >> $WARN ) && print "\n";
--- End diff --
Nope.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106294133
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -226,6 +263,7 @@
}
}
+
--- End diff --
is this new line necessary?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106326135
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -705,50 +879,84 @@ sub check_syncds_state {
$syncds_update = $UPDATE_TROPS_NEEDED;
my $parent_pending = ( defined( $upd_json->[0]->{'parent_pending'} ) ) ? $upd_json->[0]->{'parent_pending'} : undef;
+ my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
if ( !defined($parent_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $url did not have an parent_pending key.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_pending key.\n";
if ( $script_mode != $SYNCDS ) {
return $syncds_update;
}
else {
- ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $url. Exiting, not sure what else to do.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $uri. Exiting, not sure what else to do.\n";
exit 1;
}
}
- if ( $parent_pending == 1 && $wait_for_parents == 1) {
- ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need an update.\n";
- if ( $script_mode == $SYNCDS ) {
- if ( $dispersion > 0 ) {
- ( $log_level >> $WARN ) && print "WARN In syncds mode, sleeping for " . $dispersion . "s to see if the update my parents need is cleared.\n";
- for ( my $i = $dispersion; $i > 0; $i-- ) {
- ( $log_level >> $WARN ) && print ".";
- sleep 1;
+ if ( defined($parent_reval_pending) ) {
+ if ( ( $parent_pending == 1 || $parent_reval_pending == 1 ) && $wait_for_parents == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need an update.\n";
+ if ( $script_mode == $SYNCDS ) {
+ if ( $dispersion > 0 ) {
+ ( $log_level >> $WARN ) && print "WARN In syncds mode, sleeping for " . $dispersion . "s to see if the update my parents need is cleared.\n";
+ ( $dispersion > 0 ) && &sleep_timer($dispersion);
+ }
+
+ ( $log_level >> $WARN ) && print "\n";
--- End diff --
Is this meant to just print a carriage return?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r107960405
--- Diff: traffic_ops/app/db/migrations/20170226190747_add_reval_pending.sql ---
@@ -0,0 +1,26 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+-- +goose Up
+-- SQL in section 'Up' is executed when this migration is applied
+ALTER TABLE server ADD COLUMN reval_pending boolean NOT NULL DEFAULT false;
+INSERT INTO parameter (name, config_file, value, secure) VALUES ('use_reval_pending', 'global', '0', False);
+insert into profile_parameter (profile, parameter) values ((select id from profile where name = 'GLOBAL'), (select id from parameter where name = 'use_reval_pending'));
--- End diff --
@dg4prez - so this isn't going to work. when you run db/admin.pl --env=test updgrade to migrate the to_test database, this fails because there is no GLOBAL profile in the to_test db. I'm not exactly sure what the right answer is off the top of my head to fix this but this needs to be fixed because you can't run tests in the master branch currently. Maybe @dewrich or @dangogh have some ideas...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106416184
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -671,31 +748,128 @@ sub get_print_current_client_connections {
( $log_level >> $DEBUG ) && print "DEBUG There are currently $current_connections connections.\n";
}
+sub check_revalidate_state {
+ my $sleep_override = shift;
+
+ my $syncds_update = 0;
+
+ ( $log_level >> $DEBUG ) && print "DEBUG Checking revalidate state.\n";
+ if ( $script_mode == $REVALIDATE || $sleep_override == 1 ) {
+ ## The herd is about to get /update/<hostname>
+
+ my $uri = "/update/$hostname_short";
+ my $upd_ref = &lwp_get($uri);
+ if ( $upd_ref =~ m/^\d{3}$/ ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri returned $upd_ref. Exiting, not sure what else to do.\n";
+ exit 1;
+ }
+
+ my $upd_json = decode_json($upd_ref);
+ my $reval_pending = ( defined( $upd_json->[0]->{'reval_pending'} ) ) ? $upd_json->[0]->{'reval_pending'} : undef;
+ if ( !defined($reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+
+ if ( $reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that a revalidation is waiting to be applied.\n";
+ $syncds_update = $UPDATE_TROPS_NEEDED;
+
+ my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
+ if ( !defined($parent_reval_pending) ) {
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.1. Unable to continue!\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ if ( $parent_reval_pending == 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that my parents need to revalidate.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ }
+ elsif ( $script_mode == $REVALIDATE && $reval_pending != 1 ) {
+ ( $log_level >> $ERROR ) && print "ERROR In revalidate mode, but no update needs to be applied. I'm outta here.\n";
+ return($UPDATE_TROPS_NOTNEEDED);
+ }
+ else {
+ ( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no revalidations are waiting to be applied.\n";
+ }
+
+ my $stj = &lwp_get("/datastatus");
--- End diff --
The api response contains another json layer - 'response' which breaks the script. It's not a big deal, but I wonder which is "correct?"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106294076
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -213,7 +250,7 @@
&touch_file('remap.config');
last;
}
- }
+ }
--- End diff --
extra whitespace!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106412690
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -262,6 +316,7 @@ sub usage {
print "\t<Mode> = report - prints config differences and exits.\n";
print "\t<Mode> = badass - attempts to fix all config differences that it can.\n";
print "\t<Mode> = syncds - syncs delivery services with what is configured in Traffic Ops.\n";
+ print "\t<Mode> = revalidate - checks for updated revalidations in Traffic Ops and applies them. Requires Traffic Ops 1.2.\n";
--- End diff --
Fair point.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106413628
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -646,18 +713,28 @@ sub update_trops {
}
}
elsif ( $script_mode == $BADASS || $script_mode == $SYNCDS ) {
- &send_update_to_trops($CLEAR);
+ if ( defined $reval_pending ) {
+ &send_update_to_trops($CLEAR, $reval_pending );
+ }
+ else {
+ &send_update_to_trops($CLEAR, $CLEAR);
+ }
+ }
+ elsif ( $script_mode = $REVALIDATE ) {
+ &send_update_to_trops($upd_pending, $CLEAR);
}
}
}
sub send_update_to_trops {
my $status = shift;
- my $url = "$traffic_ops_host\/update/$hostname_short";
+ my $reval_status = shift;
+ my $uri = "/update/$hostname_short";
( $log_level >> $DEBUG ) && print "DEBUG Setting update flag in Traffic Ops to $status.\n";
my %headers = ( 'Cookie' => $cookie );
- my $response = $lwp_conn->post( $url, [ 'updated' => $status ], %headers );
+ my $url = $traffic_ops_host . $uri;
+ my $response = $lwp_conn->post( $url, [ 'updated' => $status, 'reval_updated' => $reval_status ], %headers );
--- End diff --
Yes. Traffic ops only looks for it if use_reval_pending is enabled, and reval_status won't be used if that's the case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dg4prez <gi...@git.apache.org>.
Github user dg4prez commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106412522
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -213,7 +250,7 @@
&touch_file('remap.config');
last;
}
- }
+ }
--- End diff --
I think we should use wood for the bike shed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by mitchell852 <gi...@git.apache.org>.
Github user mitchell852 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106267760
--- Diff: traffic_ops/app/lib/UI/Server.pm ---
@@ -915,20 +916,53 @@ sub readupdate {
{
$parent_pending{ $rs_servers->single->host_name } = 1;
}
+ if ( $prow->reval_pending == 1
+ && $prow->status->name ne "OFFLINE" )
+ {
+ $parent_reval_pending{ $rs_servers->single->host_name } = 1;
+ }
}
}
}
}
while ( my $row = $rs_servers->next ) {
- if ( $parent_pending{ $row->host_name } ) {
+ if ( $parent_pending{ $row->host_name } && $parent_reval_pending{ $row->host_name } ) {
--- End diff --
can't we collapse this if/elseif...seems like the only thing that is different is the
parent_pending => 0|1,
parent_reval_pending => 0|1
maybe a tertiary operator?
parent_pending => (condition) ? 1 : 0,
parent_reval_pending => (condition) ? 1 : 0,
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #299: Instant Invalidate for ORT and T...
Posted by dneuman64 <gi...@git.apache.org>.
Github user dneuman64 commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/299#discussion_r106326306
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -815,15 +1023,48 @@ sub check_syncds_state {
return ($syncds_update);
}
-sub sleep_rand {
- my $duration = int( rand(shift) );
-
- ( $log_level >> $WARN ) && print "WARN Sleeping for $duration seconds: ";
-
+sub sleep_timer {
+ my $duration = shift;
+ my $reval_clock = $reval_wait_time;
+
+ my $proper_script_mode = $script_mode;
+
+ if ( $reval_in_use == 1 ) {
+ ( $log_level >> $WARN ) && print "WARN Performing a revalidation check before sleeping... \n";
+ &revalidate_while_sleeping();
+ ( $log_level >> $WARN ) && print "WARN Revalidation check complete.\n";
+ }
+ if ( $duration < $reval_clock || $reval_in_use == 0 ) {
+ ( $log_level >> $WARN ) && print "WARN Sleeping for $duration seconds: ";
+ }
+ else {
+ ( $log_level >> $WARN ) && print "WARN $reval_clock seconds until next revalidation check.\n";
+ ( $log_level >> $WARN ) && print "WARN $duration seconds remaining in dispersion sleep period\n";
+ ( $log_level >> $WARN ) && print "WARN Sleeping for $reval_clock seconds: ";
+ }
for ( my $i = $duration; $i > 0; $i-- ) {
( $log_level >> $WARN ) && print ".";
sleep 1;
+ $reval_clock--;
+ if ($reval_clock < 1 && $script_mode != $BADASS && $reval_in_use == 1 ) {
+ ( $log_level >> $WARN ) && print "\n";
--- End diff --
again, do we mean to just print a new line?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---