You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2020/05/21 16:41:19 UTC
[GitHub] [trafficcontrol] ezelkow1 opened a new pull request #4713: Add csv as an output format option for astats via header
ezelkow1 opened a new pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713
This adds the ability to output CSV formatted stats via the `Accept` header. If set to `text/csv` then the output will be in csv format. Any other values will output in the old `text/json` format. The `Content-Type` field has also been fixed since previously it was reported as `application/json`
<!--
************ STOP!! ************
If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
-->
## What does this PR (Pull Request) do?
<!-- Explain the changes you made here. If this fixes an Issue, identify it by
replacing the text in the checkbox item with the Issue number e.g.
- [x] This PR fixes #9001 OR is not related to any Issue
^ This will automatically close Issue number 9001 when the Pull Request is
merged (The '#' is important).
Be sure you check the box properly, see the "The following criteria are ALL
met by this PR" section for details.
-->
- [x] This PR is related to the TM 2.0 issue but does not complete it <!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this
Pull Request. Also, feel free to add the name of a tool or script that is
affected but not on the list.
Additionally, if this Pull Request does NOT affect documentation, please
explain why documentation is not required. -->
- Traffic Monitor (will need future changes to support this)
- ATS
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your Pull Request. If
it includes tests (and most should), outline here the steps needed to run the
tests. If not, lay out the manual testing procedure and please explain why
tests are unnecessary for this Pull Request. -->
Build astats plugin and install on a cache. Hit the stats endpoint, verify the the old format is output with the new `Content-Type: text/json`. Add a header of `Accept: text/csv` to a new request and verify that the content comes back in csv format and with `Content-Type: text/csv`.
## If this is a bug fix, what versions of Traffic Control are affected?
<!-- If this PR fixes a bug, please list here all of the affected versions - to
the best of your knowledge. It's also pretty helpful to include a commit hash
of where 'master' is at the time this PR is opened (if it affects master),
because what 'master' means will change over time. For example, if this PR
fixes a bug that's present in master (at commit hash '1df853c8'), in v4.0.0,
and in the current 4.0.1 Release candidate (e.g. RC1), then this list would
look like:
- master (1df853c8)
- 4.0.0
- 4.0.1 (RC1)
If you don't know what other versions might have this bug, AND don't know how
to find the commit hash of 'master', then feel free to leave this section
blank (or, preferably, delete it entirely).
-->
## The following criteria are ALL met by this PR
<!-- Check the boxes to signify that the associated statement is true. To
"check a box", replace the space inside of the square brackets with an 'x'.
e.g.
- [ x] <- Wrong
- [x ] <- Wrong
- [] <- Wrong
- [*] <- Wrong
- [x] <- Correct!
-->
- [x] This PR includes tests OR I have explained why tests are unnecessary
- [x] This PR includes documentation OR I have explained why documentation is unnecessary
- [x] This PR includes an update to CHANGELOG.md OR such an update is not necessary
- [x] This PR includes any and all required license headers
- [x] This PR ensures that database migration sequence is correct OR this PR does not include a database migration
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
## Additional Information
<!-- If you would like to include any additional information on the PR for
potential reviewers please put it here.
Some examples of this would be:
- Before and after screenshots/gifs of the Traffic Portal if it is affected
- Links to other dependent Pull Requests
- References to relevant context (e.g. new/updates to dependent libraries,
mailing list records, blueprints)
Feel free to leave this section blank (or, preferably, delete it entirely).
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4713: Add csv as an output format option for astats via header
Posted by GitBox <gi...@apache.org>.
jhg03a commented on a change in pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713#discussion_r430496218
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -326,7 +383,40 @@ static void appendSystemState(stats_state *my_state) {
end = strstr(str, "\n");
if (end)
*end = 0;
- APPEND_STAT("proc.loadavg", "\"%s\"", str);
+ APPEND_STAT_JSON("proc.loadavg", "\"%s\"", str);
+ }
+}
+
+static void appendSystemStateCsv(stats_state *my_state) {
+ char *interface = my_state->interfaceName;
+ char buffer[16384];
+ char *str;
+ char *end;
+ int speed = 0;
+
+ APPEND_STAT_CSV("inf.name", "%s", interface);
+
+ speed = getSpeed(interface, buffer, sizeof(buffer));
+
+ APPEND_STAT_CSV("inf.speed", "%d", speed);
+
+ str = getFile("/proc/net/dev", buffer, sizeof(buffer));
+ if (str && interface) {
+ str = strstr(str, interface);
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.net.dev", "%s", str);
+ }
+ }
+
+ str = getFile("/proc/loadavg", buffer, sizeof(buffer));
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.loadavg", "%s", str);
Review comment:
This looks like a lot of copypasta just to change the output functions. Not a huge issue though.
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -193,14 +199,25 @@ stats_add_data_to_resp_buffer(const char *s, stats_state *my_state) {
return s_len;
}
-static const char RESP_HEADER[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/javascript\r\nCache-Control: no-cache\r\n\r\n";
+static const char RESP_HEADER_JSON[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/json\r\nCache-Control: no-cache\r\n\r\n";
+static const char RESP_HEADER_CSV[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/csv\r\nCache-Control: no-cache\r\n\r\n";
static void
stats_process_read(TSCont contp, TSEvent event, stats_state *my_state) {
TSDebug(PLUGIN_TAG, "stats_process_read(%d)", event);
if (event == TS_EVENT_VCONN_READ_READY) {
- my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER, my_state);
+ switch (my_state->output) {
+ case JSON_OUTPUT:
+ my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER_JSON, my_state);
+ break;
+ case CSV_OUTPUT:
+ my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER_CSV, my_state);
+ break;
+ default:
+ TSError("stats_process_read: Unknown output format\n");
+ break;
Review comment:
This might be an issue since it's a change to the default behavior. Just in case we've got malformed client requests and havn't noticed since it just worked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #4713: Add csv as an output format option for astats via header
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713#discussion_r430501584
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -326,7 +383,40 @@ static void appendSystemState(stats_state *my_state) {
end = strstr(str, "\n");
if (end)
*end = 0;
- APPEND_STAT("proc.loadavg", "\"%s\"", str);
+ APPEND_STAT_JSON("proc.loadavg", "\"%s\"", str);
+ }
+}
+
+static void appendSystemStateCsv(stats_state *my_state) {
+ char *interface = my_state->interfaceName;
+ char buffer[16384];
+ char *str;
+ char *end;
+ int speed = 0;
+
+ APPEND_STAT_CSV("inf.name", "%s", interface);
+
+ speed = getSpeed(interface, buffer, sizeof(buffer));
+
+ APPEND_STAT_CSV("inf.speed", "%d", speed);
+
+ str = getFile("/proc/net/dev", buffer, sizeof(buffer));
+ if (str && interface) {
+ str = strstr(str, interface);
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.net.dev", "%s", str);
+ }
+ }
+
+ str = getFile("/proc/loadavg", buffer, sizeof(buffer));
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.loadavg", "%s", str);
Review comment:
+1, to both
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ezelkow1 commented on a change in pull request #4713: Add csv as an output format option for astats via header
Posted by GitBox <gi...@apache.org>.
ezelkow1 commented on a change in pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713#discussion_r430499479
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -326,7 +383,40 @@ static void appendSystemState(stats_state *my_state) {
end = strstr(str, "\n");
if (end)
*end = 0;
- APPEND_STAT("proc.loadavg", "\"%s\"", str);
+ APPEND_STAT_JSON("proc.loadavg", "\"%s\"", str);
+ }
+}
+
+static void appendSystemStateCsv(stats_state *my_state) {
+ char *interface = my_state->interfaceName;
+ char buffer[16384];
+ char *str;
+ char *end;
+ int speed = 0;
+
+ APPEND_STAT_CSV("inf.name", "%s", interface);
+
+ speed = getSpeed(interface, buffer, sizeof(buffer));
+
+ APPEND_STAT_CSV("inf.speed", "%d", speed);
+
+ str = getFile("/proc/net/dev", buffer, sizeof(buffer));
+ if (str && interface) {
+ str = strstr(str, interface);
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.net.dev", "%s", str);
+ }
+ }
+
+ str = getFile("/proc/loadavg", buffer, sizeof(buffer));
+ if (str) {
+ end = strstr(str, "\n");
+ if (end)
+ *end = 0;
+ APPEND_STAT_CSV("proc.loadavg", "%s", str);
Review comment:
Agreed, just trying to leave as much of the original functionality alone as possible. We can look at refactoring after this if we want
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -193,14 +199,25 @@ stats_add_data_to_resp_buffer(const char *s, stats_state *my_state) {
return s_len;
}
-static const char RESP_HEADER[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/javascript\r\nCache-Control: no-cache\r\n\r\n";
+static const char RESP_HEADER_JSON[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/json\r\nCache-Control: no-cache\r\n\r\n";
+static const char RESP_HEADER_CSV[] = "HTTP/1.0 200 Ok\r\nContent-Type: text/csv\r\nCache-Control: no-cache\r\n\r\n";
static void
stats_process_read(TSCont contp, TSEvent event, stats_state *my_state) {
TSDebug(PLUGIN_TAG, "stats_process_read(%d)", event);
if (event == TS_EVENT_VCONN_READ_READY) {
- my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER, my_state);
+ switch (my_state->output) {
+ case JSON_OUTPUT:
+ my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER_JSON, my_state);
+ break;
+ case CSV_OUTPUT:
+ my_state->output_bytes = stats_add_data_to_resp_buffer(RESP_HEADER_CSV, my_state);
+ break;
+ default:
+ TSError("stats_process_read: Unknown output format\n");
+ break;
Review comment:
It also shouldnt even change based on client requests since everywhere else it will default to json output, on startup, on request with anything other than `text/csv`, so if we manage to get here and its some unknown type some memory probably got stomped on or something very very bad happened
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -343,25 +433,54 @@ static void json_out_stats(stats_state *my_state) {
if (my_state->recordTypes & SYSTEM_RECORD_TYPE) {
APPEND(",\n \"system\": {\n");
- appendSystemState(my_state);
- APPEND_STAT("configReloadRequests", "%d", configReloadRequests);
- APPEND_STAT("lastReloadRequest", "%" PRIu64, lastReloadRequest);
- APPEND_STAT("configReloads", "%d", configReloads);
- APPEND_STAT("lastReload", "%" PRIu64, lastReload);
- APPEND_STAT("astatsLoad", "%" PRIu64, astatsLoad);
+ appendSystemStateJson(my_state);
+ APPEND_STAT_JSON("configReloadRequests", "%d", configReloadRequests);
+ APPEND_STAT_JSON("lastReloadRequest", "%" PRIu64, lastReloadRequest);
+ APPEND_STAT_JSON("configReloads", "%d", configReloads);
+ APPEND_STAT_JSON("lastReload", "%" PRIu64, lastReload);
+ APPEND_STAT_JSON("astatsLoad", "%" PRIu64, astatsLoad);
APPEND("\"something\": \"here\"");
APPEND("\n }");
}
APPEND("\n}\n");
}
+static void csv_out_stats(stats_state *my_state) {
+ const char *version;
+ TSDebug(PLUGIN_TAG, "recordTypes: '0x%x'", my_state->recordTypes);
Review comment:
It can, but leaving it as 0x%x preserves the existing syntax used in the rest of the file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4713: Add csv as an output format option for astats via header
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713#discussion_r430706256
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -343,25 +433,54 @@ static void json_out_stats(stats_state *my_state) {
if (my_state->recordTypes & SYSTEM_RECORD_TYPE) {
APPEND(",\n \"system\": {\n");
- appendSystemState(my_state);
- APPEND_STAT("configReloadRequests", "%d", configReloadRequests);
- APPEND_STAT("lastReloadRequest", "%" PRIu64, lastReloadRequest);
- APPEND_STAT("configReloads", "%d", configReloads);
- APPEND_STAT("lastReload", "%" PRIu64, lastReload);
- APPEND_STAT("astatsLoad", "%" PRIu64, astatsLoad);
+ appendSystemStateJson(my_state);
+ APPEND_STAT_JSON("configReloadRequests", "%d", configReloadRequests);
+ APPEND_STAT_JSON("lastReloadRequest", "%" PRIu64, lastReloadRequest);
+ APPEND_STAT_JSON("configReloads", "%d", configReloads);
+ APPEND_STAT_JSON("lastReload", "%" PRIu64, lastReload);
+ APPEND_STAT_JSON("astatsLoad", "%" PRIu64, astatsLoad);
APPEND("\"something\": \"here\"");
APPEND("\n }");
}
APPEND("\n}\n");
}
+static void csv_out_stats(stats_state *my_state) {
+ const char *version;
+ TSDebug(PLUGIN_TAG, "recordTypes: '0x%x'", my_state->recordTypes);
Review comment:
This can be `'%#x'`
##########
File path: traffic_server/plugins/astats_over_http/astats_over_http.c
##########
@@ -343,25 +433,54 @@ static void json_out_stats(stats_state *my_state) {
if (my_state->recordTypes & SYSTEM_RECORD_TYPE) {
APPEND(",\n \"system\": {\n");
- appendSystemState(my_state);
- APPEND_STAT("configReloadRequests", "%d", configReloadRequests);
- APPEND_STAT("lastReloadRequest", "%" PRIu64, lastReloadRequest);
- APPEND_STAT("configReloads", "%d", configReloads);
- APPEND_STAT("lastReload", "%" PRIu64, lastReload);
- APPEND_STAT("astatsLoad", "%" PRIu64, astatsLoad);
+ appendSystemStateJson(my_state);
+ APPEND_STAT_JSON("configReloadRequests", "%d", configReloadRequests);
+ APPEND_STAT_JSON("lastReloadRequest", "%" PRIu64, lastReloadRequest);
+ APPEND_STAT_JSON("configReloads", "%d", configReloads);
+ APPEND_STAT_JSON("lastReload", "%" PRIu64, lastReload);
+ APPEND_STAT_JSON("astatsLoad", "%" PRIu64, astatsLoad);
APPEND("\"something\": \"here\"");
APPEND("\n }");
}
APPEND("\n}\n");
}
+static void csv_out_stats(stats_state *my_state) {
+ const char *version;
+ TSDebug(PLUGIN_TAG, "recordTypes: '0x%x'", my_state->recordTypes);
Review comment:
Fair enough
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c merged pull request #4713: Add csv as an output format option for astats via header
Posted by GitBox <gi...@apache.org>.
rob05c merged pull request #4713:
URL: https://github.com/apache/trafficcontrol/pull/4713
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org