You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by GitBox <gi...@apache.org> on 2020/06/19 22:33:00 UTC
[GitHub] [trafficserver] bneradt opened a new pull request #6931: AuTest: New log and When condition for ATS initialized
bneradt opened a new pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931
This aims to make the AuTests more reliable by providing a log message
from traffic server that the AutTests can use as a ready condition
before the tests are started.
In order to make use of this, however, many tests override the ready
condition. This was generally never correct in the first place because
MakeATSProcess sets the same ready condition anyway. Maybe this was
added later? In any case, to take advantage of the new Ready, the tests
have to be cleaned up to not set their own Ready. While doing this, I
added the enable_cache directive to make setting the cache or not
a more convenient mechanism for the MakeATSProcess caller.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445033763
##########
File path: tests/gold_tests/autest-site/when.test.ext
##########
@@ -19,18 +19,66 @@ When extensions.
from autest.api import AddWhenFunction
import hosts.output as host
+import os
+import re
-def FileContains(haystack, needle):
+def FileContains(haystack, needle, desired_count=1):
Review comment:
That's a good point. I should look into pre-compiling the regex.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] SolidWallOfCode commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
SolidWallOfCode commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444428892
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -2094,7 +2116,11 @@ main(int /* argc ATS_UNUSED */, const char **argv)
if (delay_p && ink_atomic_cas(&delay_listen_for_cache_p, 0, 1)) {
Debug("http_listen", "Delaying listen, waiting for cache initialization");
} else {
+ // Since the previous delay_listen_for_cache_p was not 0, then the
+ // cache must have been initialized already.
Review comment:
"or ATS is not waiting for the cache to make the ports ready"
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -173,10 +174,23 @@ static int poll_timeout = -1; // No value set.
static int cmd_disable_freelist = 0;
static bool signal_received[NSIG];
-// 1: delay listen, wait for cache.
-// 0: Do not delay, start listen ASAP.
+/// We use the following two variables to gate the emission of the "traffic
+/// server initialized" log message. This message can be helpful for users who
+/// want to know that Traffic Server is not just running but has had its cache
+/// initialized and has called accept() on its ports.
+
+/// The sockets are not only created, but traffic server has set our socket
+/// options on them, plugins have configured them as desired, and traffic
+/// server has called accept on them.
+static std::atomic<bool> ports_are_ready = false;
+
+/// Cache initialization has completed.
+static std::atomic<bool> cache_is_initialized = false;
+
+// 1: the main thread delayed accepting, start accepting.
+// 0: delay accept, wait for cache initialization.
// -1: cache is already initialized, don't delay.
-static int delay_listen_for_cache_p;
+static int delay_listen_for_cache_p = 0;
Review comment:
Should take the "_p" suffix off, it's not a `bool`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#issuecomment-648393333
> I'm not sure if all the extra
> 'proxy.config.http.cache.http': 1
> were removed.
> All my nits are about lines that perhaps should have been removed as part of the PR?
Thanks for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444887751
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
+ * 2. The ports are open and accept has been called upon them.
Review comment:
Do you mean listen rather than accept?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444459273
##########
File path: tests/gold_tests/pluginTest/slice/slice_regex.test.py
##########
@@ -126,7 +126,6 @@
ts.Disk.records_config.update({
# 'proxy.config.diags.debug.enabled': 1,
# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 0,
'proxy.config.http.wait_for_cache': 0,
Review comment:
Removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445059148
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
+ * 2. The ports are open and accept has been called upon them.
Review comment:
I'm adding a paragraph to flesh this out further. Let me know if it is helpful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
shinrich commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443555374
##########
File path: tests/gold_tests/headers/cachedIMSRange.test.py
##########
@@ -64,7 +64,6 @@
'proxy.config.diags.debug.tags': 'http',
'proxy.config.http.response_via_str': 3,
'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
})
Review comment:
What is the logic for removing wait_for_cache here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444911071
##########
File path: tests/gold_tests/autest-site/when.test.ext
##########
@@ -19,18 +19,66 @@ When extensions.
from autest.api import AddWhenFunction
import hosts.output as host
+import os
+import re
-def FileContains(haystack, needle):
+def FileContains(haystack, needle, desired_count=1):
Review comment:
Does Au Test allow this to be a functor? Seems like it should pre-compile the regular expression, and keep track of how much of the file it has already scanned. But perhaps beyond the scope of this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444956207
##########
File path: tests/gold_tests/pluginTest/slice/slice.test.py
##########
@@ -87,19 +87,16 @@
' @plugin=slice.so @pparam=--blockbytes-test={}'.format(block_bytes)
])
-# minimal configuration
-ts.Disk.records_config.update({
-# 'proxy.config.diags.debug.enabled': 1,
-# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
-})
+# ts.Disk.records_config.update({
+# 'proxy.config.diags.debug.enabled': 1,
Review comment:
Nitpick: You could uncomment this call, but set debug.enabled to 0. Then it would only be necessary to change one character to temporarily enable debug tracing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
shinrich commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443553338
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -737,6 +743,9 @@ CB_After_Cache_Init()
hook->invoke(TS_EVENT_LIFECYCLE_CACHE_READY, nullptr);
hook = hook->next();
}
+ if (ports_are_open) {
+ Note("traffic server initialized");
+ }
}
Review comment:
Is it possible that ports_are_open is not true when going through CB_After_Cache_Init()? If so, does the logic go through this callback multiple times?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443690300
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -737,6 +743,9 @@ CB_After_Cache_Init()
hook->invoke(TS_EVENT_LIFECYCLE_CACHE_READY, nullptr);
hook = hook->next();
}
+ if (ports_are_open) {
+ Note("traffic server initialized");
+ }
}
Review comment:
Yes, that is possible that the ports are not open when this callback is called, but the callback doesn't need to be called multiple times. In fact it will be the case that the ports will not have been opened when CB_After_Cache_Init is called whenever wait_for_cache is enabled. But in those cases we will set ports_are_open to true once we call start_HttpProxyServer in that callback.
In other words, ports_are_open is set to true in the two places we call start_HttpProxyServer:
1. In main if wait_for_cache is not set.
1. In CB_After_Cache_Init if wait_for_cache was set by the user, indicated they don't want the ports open until the cache is initialized.
So ports_are_open will have to be set in either of those two places.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445033310
##########
File path: tests/gold_tests/autest-site/trafficserver.test.ext
##########
@@ -281,10 +282,26 @@ def MakeATSProcess(obj, name, command='traffic_server', select_ports=True, enabl
get_port(p, "manager_port")
get_port(p, "admin_port")
- if enable_tls:
- p.Ready = When.PortsOpen([p.Variables.port, p.Variables.ssl_port])
+ if enable_cache:
+ # In records.config, the cache is enabled by default so there's nothing
+ # we have to do here to functionally enable it. However, the tests that
+ # rely upon the cache will not function correctly if ATS starts
+ # processing traffic before the cache is ready. Thus we set the
+ # wait_for_cache configuration.
+ p.Disk.records_config.update({
+ # Do not accept connections from clients until cache subsystem is
+ # operational.
+ 'proxy.config.http.wait_for_cache': 1,
+ })
else:
- p.Ready = When.PortOpen(p.Variables.port)
+ # The user wants the cache to be disabled.
+ p.Disk.records_config.update({
+ 'proxy.config.http.cache.http': 0
+ })
+
+ # The following message was added so that tests and users can know when
+ # Traffic Server is ready to both receive and optimize traffic.
Review comment:
What you describe is more specific and true, but I was trying to keep it generic. I'm trying to leave this implementation open to other state transitions that we may find later are helpful to include too. Thus I was trying to communicate that Traffic Server has done everything required to receive connections (bound ports and called accept on them) and everything required for optimization (initialized the cache being one of possibly other things).
That was my intention anyway. Think I should just keep it specific?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443801836
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -2125,6 +2136,9 @@ main(int /* argc ATS_UNUSED */, const char **argv)
ink_set_thread_name("[TS_MAIN]");
+ if (ports_are_open && cache_is_initialized) {
Review comment:
It should mean that the cache is initialized. Even if proxy.config.http.cache.http is disabled, the callback is called after cache initialization. Thus the "traffic server initialized" log will be emitted after cache initialization (even if http.cache.http is disabled) and after ports are open, regardless of whether wait_for_cache is enabled.
Let me know if I'm missing the point of your question.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445031635
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
Review comment:
Can you avoid cache initialization? I found that even if http.cache.http was disabled the cache was still initialized.
I can look into this further.
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
+ * 2. The ports are open and accept has been called upon them.
Review comment:
I actually do mean accept.
Here's some more context: Alan showed me that the ports may be opened and listen may have been called on the ports before cache is initialized and before accept is called on them, and that the difference in time between the opening of the ports and the accept can be large (over 10 ms, say).
Here's the sequence of events that Traffic Server follows with regard to configuring its ports:
1. Traffic server initializes data structures for the ports it listens on. It may bind upon those ports depending upon Traffic Server configuration. (Our docs say as much, see https://docs.trafficserver.apache.org/en/9.0.x/developer-guide/api/functions/TSLifecycleHookAdd.en.html?highlight=life%20cycle#_CPPv4N17TSLifecycleHookID35TS_LIFECYCLE_PORTS_INITIALIZED_HOOKE)
1. Traffic Server calls TS_LIFECYCLE_PORTS_INITIALIZED_HOOK on all plugins registered for it so they can do whatever they want to do to prepare for connections.
1. If wait_for_cache is called, we delay calling accept on the ports until cache is initialized, otherwise we directly...
1. Call start_HttpProxyServer(), which calls accept on the ports.
Before this patch, the AuTest extension considered Traffic Server ready to accept test traffic if it succeeded in opening a connection to the listening port. But this could happen in step one above, before cache is initialized and before accept is called on the ports, **even if wait_for_cache is configured**. This change adds a message to wait upon a message that is designed to guarantee that everything is done and optimization is ready.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bryancall merged pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bryancall merged pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445084881
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
Review comment:
I enabled the "cache" debug tag and ran the deny0 test which disables the cache. The cache does indeed get initialized with this output indicating cache initialization:
```
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3068 (ink_cache_init)> (cache_init) proxy.config.cache.ram_cache.size = -1 = 0Mb
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3076 (ink_cache_init)> (cache_init) proxy.config.cache.limits.http.max_alts = 5
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3080 (ink_cache_init)> (cache_init) cache_config_ram_cache_cutoff = 4194304 = 4Mb
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3083 (ink_cache_init)> (cache_init) proxy.config.cache.permit.pinning = 0
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3086 (ink_cache_init)> (cache_init) proxy.config.cache.dir.sync_frequency = 60
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3089 (ink_cache_init)> (cache_init) proxy.config.cache.select_alternate = 1
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3093 (ink_cache_init)> (cache_init) proxy.config.cache.max_doc_size = 0 = 0Mb
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3096 (ink_cache_init)> (cache_init) proxy.config.cache.mutex_retry_delay = 2ms
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3099 (ink_cache_init)> (cache_init) proxy.config.cache.read_while_writer.max_retries = 10
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3102 (ink_cache_init)> (cache_init) proxy.config.cache.read_while_writer_retry.delay = 50ms
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3105 (ink_cache_init)> (cache_init) proxy.config.cache.hit_evacuate_percent = 0
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3108 (ink_cache_init)> (cache_init) proxy.config.cache.hit_evacuate_size_limit = 0
[Jun 24 17:23:28.592] traffic_server DEBUG: <Cache.cc:3121 (ink_cache_init)> (cache_init) proxy.config.cache.max_disk_errors = 5
[Jun 24 17:23:28.593] traffic_server DEBUG: <Cache.cc:3124 (ink_cache_init)> (cache_init) proxy.config.cache.agg_write_backlog = 5242880
[Jun 24 17:23:28.593] traffic_server DEBUG: <Cache.cc:3127 (ink_cache_init)> (cache_init) proxy.config.cache.enable_checksum = 0
[Jun 24 17:23:28.593] traffic_server DEBUG: <Cache.cc:3130 (ink_cache_init)> (cache_init) proxy.config.cache.alt_rewrite_max_size = 4096
[Jun 24 17:23:28.593] traffic_server DEBUG: <Cache.cc:3135 (ink_cache_init)> (cache_init) proxy.config.cache.enable_read_while_writer = 1
[Jun 24 17:23:28.593] traffic_server DEBUG: <Store.cc:331 (read_config)> (cache_init) Store::read_config, fd = -1, "/tmp/sbdelay/deny0/ts/config/storage.config"
[Jun 24 17:23:28.593] traffic_server DEBUG: <Store.cc:364 (read_config)> (cache_init) Store::read_config: "storage"
[Jun 24 17:23:28.593] traffic_server DEBUG: <Store.cc:403 (read_config)> (cache_init) Store::read_config - ns = new Span; ns->init("/tmp/sbdelay/deny0/ts/storage",268435456), forced volume=-1
[Jun 24 17:23:28.593] traffic_server DEBUG: <Store.cc:594 (init)> (cache_init) initialized span '/tmp/sbdelay/deny0/ts/storage'
[Jun 24 17:23:28.593] traffic_server DEBUG: <Store.cc:596 (init)> (cache_init) hw_sector_size=4096, size=268435456, blocks=32768, disk_id=64514/197593927, file_pathname=0
[Jun 24 17:23:28.757] traffic_server DEBUG: <Cache.cc:688 (start_internal)> (cache_hosting) Disk: 0:/tmp/sbdelay/deny0/ts/storage/cache.db, blocks: 32767
[Jun 24 17:23:28.768] [ET_NET 9] DEBUG: <Cache.cc:2666 (cplist_reconfigure)> (cache_hosting) blocks = 32766
[Jun 24 17:23:28.769] [ET_NET 9] DEBUG: <Cache.cc:810 (diskInitialized)> (cache_hosting) Disk: 0:/tmp/sbdelay/deny0/ts/storage/cache.db: Vol Blocks: 1: Free space: 0
[Jun 24 17:23:28.769] [ET_NET 9] DEBUG: <Cache.cc:812 (diskInitialized)> (cache_hosting) Vol: 0 Size: 32766
[Jun 24 17:23:28.769] [ET_NET 9] DEBUG: <Cache.cc:816 (diskInitialized)> (cache_hosting) Block No: 0 Size: 32766 Free: 0
[Jun 24 17:23:28.769] [ET_NET 9] DEBUG: <Cache.cc:2065 (open)> (cache_init) Cache::open - proxy.config.cache.min_average_object_size = 8000
[Jun 24 17:23:28.778] [ET_NET 9] DEBUG: <Cache.cc:1256 (init)> (cache_init) Vol /tmp/sbdelay/deny0/ts/storage/cache.db 16384:32766: allocating 352256 directory bytes for a 268419072 byte volume (0.131234%)
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <CacheHosting.cc:433 (Init)> (cache_hosting) Host Record: 0x7f8f68028838, Volume: 0, size: 32766
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:1885 (build_vol_hash_table)> (cache_init) build_vol_hash_table index 0 mapped to 0 requested 32707 got 32707
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:863 (cacheInitialized)> (cache_init) CacheProcessor::cacheInitialized - theCache, total_size = 32766 = 255 MB
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:892 (cacheInitialized)> (cache_init) CacheProcessor::cacheInitialized - caches_ready=0x9, gnvol=1
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:911 (cacheInitialized)> (cache_init) CacheProcessor::cacheInitialized - cache_config_ram_cache_size == AUTO_SIZE_RAM_CACHE
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:919 (cacheInitialized)> (cache_init) CacheProcessor::cacheInitialized - ram_cache_bytes = 352256 = 0Mb
[Jun 24 17:23:28.782] [ET_NET 10] DEBUG: <Cache.cc:925 (cacheInitialized)> (cache_init) CacheProcessor::cacheInitialized - total_cache_bytes = 268066816 = 255Mb
```
Thanks for your thoughtful questions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445068345
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
+ * 2. The ports are open and accept has been called upon them.
Review comment:
Sorry, my bad, somehow I thought for TCP, listen blocked until it got the first SYN.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] traeak commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
traeak commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444438315
##########
File path: tests/gold_tests/pluginTest/slice/slice_regex.test.py
##########
@@ -126,7 +126,6 @@
ts.Disk.records_config.update({
# 'proxy.config.diags.debug.enabled': 1,
# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 0,
'proxy.config.http.wait_for_cache': 0,
Review comment:
Should this line go away with the enable_cache=False?
##########
File path: tests/gold_tests/pluginTest/slice/slice_error.test.py
##########
@@ -294,7 +294,6 @@
ts.Disk.records_config.update({
# 'proxy.config.diags.debug.enabled': 1,
# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 0,
'proxy.config.http.wait_for_cache': 0,
Review comment:
Should this line go away with the enable_cache setting?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445035335
##########
File path: tests/gold_tests/tls/tls_client_cert2.test.py
##########
@@ -66,7 +66,7 @@
'proxy.config.ssl.client.verify.server': 0,
'proxy.config.ssl.server.cipher_suite': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:RC4-MD5:AES128-SHA:AES256-SHA:DES-CBC3-SHA!SRP:!DSS:!PSK:!aNULL:!eNULL:!SSLv2',
'proxy.config.exec_thread.autoconfig.scale': 1.0,
- 'proxy.config.url_remap.pristine_host_hdr' : 1,
Review comment:
I use a vim plugin called ALE which highlights formatting and obvious syntax issues.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444454565
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -2094,7 +2116,11 @@ main(int /* argc ATS_UNUSED */, const char **argv)
if (delay_p && ink_atomic_cas(&delay_listen_for_cache_p, 0, 1)) {
Debug("http_listen", "Delaying listen, waiting for cache initialization");
} else {
+ // Since the previous delay_listen_for_cache_p was not 0, then the
+ // cache must have been initialized already.
Review comment:
I'm open to improving the comment. What are you fixing about it? Is it wrong to say that the cache must have been initialized by this point? As I read things, if we get to this branch then the value must have been -1 and thus the cache must be initialized and I'm trying to help the reader see that's the case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bryancall commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bryancall commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443724281
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -2125,6 +2136,9 @@ main(int /* argc ATS_UNUSED */, const char **argv)
ink_set_thread_name("[TS_MAIN]");
+ if (ports_are_open && cache_is_initialized) {
Review comment:
Does cache_is_initialized == true really mean that the cache is initialize? What happens if we are not waiting for cache initialized to open the ports and cache takes a long time to start up?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445069192
##########
File path: tests/gold_tests/autest-site/trafficserver.test.ext
##########
@@ -281,10 +282,26 @@ def MakeATSProcess(obj, name, command='traffic_server', select_ports=True, enabl
get_port(p, "manager_port")
get_port(p, "admin_port")
- if enable_tls:
- p.Ready = When.PortsOpen([p.Variables.port, p.Variables.ssl_port])
+ if enable_cache:
+ # In records.config, the cache is enabled by default so there's nothing
+ # we have to do here to functionally enable it. However, the tests that
+ # rely upon the cache will not function correctly if ATS starts
+ # processing traffic before the cache is ready. Thus we set the
+ # wait_for_cache configuration.
+ p.Disk.records_config.update({
+ # Do not accept connections from clients until cache subsystem is
+ # operational.
+ 'proxy.config.http.wait_for_cache': 1,
+ })
else:
- p.Ready = When.PortOpen(p.Variables.port)
+ # The user wants the cache to be disabled.
+ p.Disk.records_config.update({
+ 'proxy.config.http.cache.http': 0
+ })
+
+ # The following message was added so that tests and users can know when
+ # Traffic Server is ready to both receive and optimize traffic.
Review comment:
Yeah it's fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444956752
##########
File path: tests/gold_tests/pluginTest/slice/slice_selfhealing.test.py
##########
@@ -68,13 +68,10 @@
ts.Disk.plugin_config.AddLine('xdebug.so')
-# minimal configuration
-ts.Disk.records_config.update({
- 'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
-# 'proxy.config.diags.debug.enabled': 1,
-# 'proxy.config.diags.debug.tags': 'cache_range_requests|slice',
-})
+# ts.Disk.records_config.update({
Review comment:
ditto nitpick
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444970036
##########
File path: tests/gold_tests/tls/tls_client_cert2.test.py
##########
@@ -66,7 +66,7 @@
'proxy.config.ssl.client.verify.server': 0,
'proxy.config.ssl.server.cipher_suite': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:RC4-MD5:AES128-SHA:AES256-SHA:DES-CBC3-SHA!SRP:!DSS:!PSK:!aNULL:!eNULL:!SSLv2',
'proxy.config.exec_thread.autoconfig.scale': 1.0,
- 'proxy.config.url_remap.pristine_host_hdr' : 1,
Review comment:
Do you use a static checking tool to find these cases of non-standard use of whitespace?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445050944
##########
File path: tests/gold_tests/autest-site/trafficserver.test.ext
##########
@@ -281,10 +282,26 @@ def MakeATSProcess(obj, name, command='traffic_server', select_ports=True, enabl
get_port(p, "manager_port")
get_port(p, "admin_port")
- if enable_tls:
- p.Ready = When.PortsOpen([p.Variables.port, p.Variables.ssl_port])
+ if enable_cache:
+ # In records.config, the cache is enabled by default so there's nothing
+ # we have to do here to functionally enable it. However, the tests that
+ # rely upon the cache will not function correctly if ATS starts
+ # processing traffic before the cache is ready. Thus we set the
+ # wait_for_cache configuration.
+ p.Disk.records_config.update({
+ # Do not accept connections from clients until cache subsystem is
+ # operational.
+ 'proxy.config.http.wait_for_cache': 1,
Review comment:
I generally prefer keeping the comma for the last item as a convenience if another item is appended.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
shinrich commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443556044
##########
File path: tests/gold_tests/logging/all_headers.test.py
##########
@@ -36,9 +36,6 @@
server.addResponse("sessionlog.json", request_header, response_header)
ts.Disk.records_config.update({
- # Do not accept connections from clients until cache subsystem is operational.
- 'proxy.config.http.wait_for_cache': 1,
-
Review comment:
Here too, what is the rationale for not waiting for cache?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444459186
##########
File path: tests/gold_tests/pluginTest/slice/slice_error.test.py
##########
@@ -294,7 +294,6 @@
ts.Disk.records_config.update({
# 'proxy.config.diags.debug.enabled': 1,
# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 0,
'proxy.config.http.wait_for_cache': 0,
Review comment:
Good point.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
shinrich commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443704538
##########
File path: tests/gold_tests/headers/cachedIMSRange.test.py
##########
@@ -64,7 +64,6 @@
'proxy.config.diags.debug.tags': 'http',
'proxy.config.http.response_via_str': 3,
'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
})
Review comment:
Looks good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444899838
##########
File path: tests/gold_tests/autest-site/trafficserver.test.ext
##########
@@ -281,10 +282,26 @@ def MakeATSProcess(obj, name, command='traffic_server', select_ports=True, enabl
get_port(p, "manager_port")
get_port(p, "admin_port")
- if enable_tls:
- p.Ready = When.PortsOpen([p.Variables.port, p.Variables.ssl_port])
+ if enable_cache:
+ # In records.config, the cache is enabled by default so there's nothing
+ # we have to do here to functionally enable it. However, the tests that
+ # rely upon the cache will not function correctly if ATS starts
+ # processing traffic before the cache is ready. Thus we set the
+ # wait_for_cache configuration.
+ p.Disk.records_config.update({
+ # Do not accept connections from clients until cache subsystem is
+ # operational.
+ 'proxy.config.http.wait_for_cache': 1,
+ })
else:
- p.Ready = When.PortOpen(p.Variables.port)
+ # The user wants the cache to be disabled.
+ p.Disk.records_config.update({
+ 'proxy.config.http.cache.http': 0
+ })
+
+ # The following message was added so that tests and users can know when
+ # Traffic Server is ready to both receive and optimize traffic.
Review comment:
Hmm, isn't it "Trafficserver is ready to receive and process HTTP requests, and cache responses (if caching is configured)."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] traeak commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
traeak commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444437921
##########
File path: tests/gold_tests/pluginTest/slice/slice_error.test.py
##########
@@ -294,7 +294,6 @@
ts.Disk.records_config.update({
# 'proxy.config.diags.debug.enabled': 1,
# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 0,
'proxy.config.http.wait_for_cache': 0,
Review comment:
Should this line go away with the enable_cache=False setting?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] zwoop commented on pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
zwoop commented on pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#issuecomment-648331024
Too big for 9.0.x and not critical, moving out to 9.1.x for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445059148
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
+ * 2. The ports are open and accept has been called upon them.
Review comment:
I'm adding a paragraph in the comment to flesh this out further. Let me know if it is helpful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444480041
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -2094,7 +2116,11 @@ main(int /* argc ATS_UNUSED */, const char **argv)
if (delay_p && ink_atomic_cas(&delay_listen_for_cache_p, 0, 1)) {
Debug("http_listen", "Delaying listen, waiting for cache initialization");
} else {
+ // Since the previous delay_listen_for_cache_p was not 0, then the
+ // cache must have been initialized already.
Review comment:
Alan pointed out offline that we should take delay_p, tied to the value of wait_for_cache, into consideration with this comment. I've updated the comment accordingly.
Thanks for the review Alan!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444944088
##########
File path: tests/gold_tests/autest-site/trafficserver.test.ext
##########
@@ -281,10 +282,26 @@ def MakeATSProcess(obj, name, command='traffic_server', select_ports=True, enabl
get_port(p, "manager_port")
get_port(p, "admin_port")
- if enable_tls:
- p.Ready = When.PortsOpen([p.Variables.port, p.Variables.ssl_port])
+ if enable_cache:
+ # In records.config, the cache is enabled by default so there's nothing
+ # we have to do here to functionally enable it. However, the tests that
+ # rely upon the cache will not function correctly if ATS starts
+ # processing traffic before the cache is ready. Thus we set the
+ # wait_for_cache configuration.
+ p.Disk.records_config.update({
+ # Do not accept connections from clients until cache subsystem is
+ # operational.
+ 'proxy.config.http.wait_for_cache': 1,
Review comment:
Nitpick: unneeded comma. Or, maybe set cache.http to 1 here, to elbow-pad this in case of a default change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r443561573
##########
File path: tests/gold_tests/headers/cachedIMSRange.test.py
##########
@@ -64,7 +64,6 @@
'proxy.config.diags.debug.tags': 'http',
'proxy.config.http.response_via_str': 3,
'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
})
Review comment:
Thanks for the review!
I pushed wait_for_cache into MakeATSProcess. The logic being if you want the cache enabled, then you'll want to wait for it to be enabled before using ATS.
See:
https://github.com/apache/trafficserver/pull/6931/files#diff-bdc07bfafa700b29a03a4b96413198acR294
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444454739
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -173,10 +174,23 @@ static int poll_timeout = -1; // No value set.
static int cmd_disable_freelist = 0;
static bool signal_received[NSIG];
-// 1: delay listen, wait for cache.
-// 0: Do not delay, start listen ASAP.
+/// We use the following two variables to gate the emission of the "traffic
+/// server initialized" log message. This message can be helpful for users who
+/// want to know that Traffic Server is not just running but has had its cache
+/// initialized and has called accept() on its ports.
+
+/// The sockets are not only created, but traffic server has set our socket
+/// options on them, plugins have configured them as desired, and traffic
+/// server has called accept on them.
+static std::atomic<bool> ports_are_ready = false;
+
+/// Cache initialization has completed.
+static std::atomic<bool> cache_is_initialized = false;
+
+// 1: the main thread delayed accepting, start accepting.
+// 0: delay accept, wait for cache initialization.
// -1: cache is already initialized, don't delay.
-static int delay_listen_for_cache_p;
+static int delay_listen_for_cache_p = 0;
Review comment:
Makes sense.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
shinrich commented on pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#issuecomment-647520130
Looks like a lovely cleanup pass. Gets rid of a lot of repeating cruft.
My only question was about removing proxy.config.http.wait_for_cache. Is it set to 1 by default?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r444886546
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
Review comment:
Should this say "If enabled, the cache is initialized".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445034226
##########
File path: tests/gold_tests/pluginTest/slice/slice.test.py
##########
@@ -87,19 +87,16 @@
' @plugin=slice.so @pparam=--blockbytes-test={}'.format(block_bytes)
])
-# minimal configuration
-ts.Disk.records_config.update({
-# 'proxy.config.diags.debug.enabled': 1,
-# 'proxy.config.diags.debug.tags': 'slice',
- 'proxy.config.http.cache.http': 1,
- 'proxy.config.http.wait_for_cache': 1,
-})
+# ts.Disk.records_config.update({
+# 'proxy.config.diags.debug.enabled': 1,
Review comment:
That's a good point. I like that better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] ywkaras commented on a change in pull request #6931: AuTest: New log and When condition for ATS initialized
Posted by GitBox <gi...@apache.org>.
ywkaras commented on a change in pull request #6931:
URL: https://github.com/apache/trafficserver/pull/6931#discussion_r445067442
##########
File path: src/traffic_server/traffic_server.cc
##########
@@ -452,6 +453,33 @@ class MemoryLimit : public Continuation
struct rusage _usage;
};
+/** Gate the emission of the "Traffic Server is fuly initialized" log message.
+ *
+ * This message is intended to be helpful to users who want to know that
+ * Traffic Server is not just running but has become fully initialized and is
+ * ready to optimize traffic. This is in contrast to the "traffic server is
+ * running" message which can be printed before either of these conditions.
+ *
+ * This function is called on each initialization state transition. Currently,
+ * the two state transitions of interest are:
+ *
+ * 1. The cache is initialized.
Review comment:
I can believe it, it's trafficserver.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org