You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Nick Kew (Created) (JIRA)" <ji...@apache.org> on 2011/10/22 16:37:32 UTC
[jira] [Created] (TS-998) Broken ClientReq in TSAPI
Broken ClientReq in TSAPI
-------------------------
Key: TS-998
URL: https://issues.apache.org/jira/browse/TS-998
Project: Traffic Server
Issue Type: Bug
Affects Versions: 3.0.1
Environment: any
Reporter: Nick Kew
Assignee: Alan M. Carroll
Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
Observed behaviour: "http://" is prepended to the request URL:
GET /path/ HTTP/1.1
becomes
GET http:///path/ HTTP/1.1
(yes, that's three slashes)
Pseudo-code to reproduce from a PRE_REMAP hook:
TSHttpTxnClientReqGet(txnp, &buf, &hdr);
TSHttpHdrPrint(buf, hdr, iobuf);
reader = TSIOBufferReaderAlloc(iobuf);
block = TSIOBufferReaderStart(reader);
len = TSIOBufferBlockReadAvail(block, reader);
data = TSIOBufferBlockReadStart(block, reader, &len);
Now examine the contents of data.
Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187345#comment-13187345 ]
Leif Hedstrom commented on TS-998:
----------------------------------
I'm assuming that
{code}
if (must_copy && s_str) {
s_str = heap->duplicate_str(s_str, s_len);
}
{code}
will have to copy the string version of something (sounds like the entire request URL + header). Maybe that is cheap. As for the logging, that string is only used (afaik) if you use the cquuc tag in a custom log (it would not be used in the default logs, or error logs). We could perhaps change that if/where it makes sense.
What I'm saying is, what would make most sense to me personally would be to keep an immutable request object before remap happens, and then we can use normal APIs on that. This would remove the need for the PristineUrl API, and this string version of the request URL for logging (which is created solely for logging, if someone uses the cquuc tag in a custom log).
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187287#comment-13187287 ]
Nick Kew commented on TS-998:
-----------------------------
Hang on! All this patch is doing in my tests is to keep a pointer to the buf in url_parse. I'm not sure when copy_strings is required, but I haven't encountered it in any test. AFAICS the most expensive operation in the patch is the strstr !
My memory could be at fault here, but I think I encountered the same bug with PristineUrlGet as first reported: namely an incorrectly-reconstructed URL. Unfortunately an instant-test of it on reading your comment fails a sanity check ( *url_loc is NULL), and it's the time of evening when I'm going to run around in fruitless circles if I dig deeper.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom resolved TS-998.
------------------------------
Resolution: Won't Fix
Marking as wont fix, backed out with commit: ed0e0badf9562a8bd2bce5215c1eda7fdfe2b2cf
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.3.1
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "James Peach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421155#comment-13421155 ]
James Peach commented on TS-998:
--------------------------------
I #ifdef'ed ENABLE_SAVE_ORIGINAL_REQUEST and verified that you just get junk (ie. the buffer is modified in place). So as it stands, this API seems to be completely broken. I guess that Nick must have found a workaround for his original use case?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.3.1
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171992#comment-13171992 ]
Nick Kew commented on TS-998:
-----------------------------
Relatively simple to fix the narrow issue as reported at the level of http_hdr_print. Just returning the buffer from the heap arg and its length is exactly what my plugin wants.
But this is a broader bug. It manifests also in the error log, in messages like:
20111219.00h26m28s RESPONSE: sent 127.0.0.1 status 404 (Not Found on Accelerator) for 'http:///'
Here the critical traceback is:
#0 url_print (url=0xb6b64a08, buf_start=0xb6b24c15 "http://IVDIWF/F5W",
buf_length=8, buf_index_inout=0xb728f730,
buf_chars_to_skip_inout=0xb728f72c) at URL.cc:1534
#1 0x0823a7de in url_string_get (url=0xb6b64a08, arena=0xb1cf1ae0,
length=0x0, heap=0xb6b64808) at URL.cc:594
#2 0x08113e7e in URL::string_get (this=0xb1cf2280,
arena_or_null_for_malloc=0xb1cf1ae0, length=0x0) at ./hdrs/URL.h:405
#3 0x081ca4ea in HttpTransact::build_error_response (s=0xb1cf1ad8,
status_code=HTTP_STATUS_NOT_FOUND,
reason_phrase_or_null=0x832266c "Not Found on Accelerator",
error_body_type=0x8322656 "urlrouting#no_mapping",
format=0x8322634 "Your requested URL was not found.")
at HttpTransact.cc:8058
The logic of url_print eludes me completely: under what circumstances do we ever want to build a URL that includes pseudo-components such as username/password?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Alan M. Carroll
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187294#comment-13187294 ]
Leif Hedstrom commented on TS-998:
----------------------------------
Nick: So, looking back at our changes in the past, TS-96 implemented the new API TSHttpTxnPristineUrlGet(). Is that not what you ask for here? If not, what can we change to make it so that it works ?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187326#comment-13187326 ]
Nick Kew commented on TS-998:
-----------------------------
The URL used for logging (at least the error log) is broken in exactly the same way. That is to say, for example,
GET / HTTP/1.0
Host: something.unrecognised
logs an error with the URL as "http:///" , with the usual three slashes
I'm confused by the number of places we seem to store this URL that always reconstructs incorrectly. The correct buffer appears in many places, and my patch just attempts to store a reference to it at a point in processing when (AFAICT) it's known to be valid. I'm not sure what you mean by stringify: it sounds like a lot of work (as does copying it).
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187434#comment-13187434 ]
Leif Hedstrom edited comment on TS-998 at 1/17/12 4:22 AM:
-----------------------------------------------------------
Ok, I re-ran all the benchmarks (with and without the latest patch here):
Before:
{code}
48759110 fetches on 482907 conns, 300 max parallel, 4.875910E+09 bytes, in 300 seconds
100 mean bytes/fetch
162530.2 fetches/sec, 1.625302E+07 bytes/sec
msecs/connect: 0.266 mean, 9.225 max, 0.042 min
msecs/first-response: 1.329 mean, 294.947 max, 0.083 min
{code}
After:
{code}
43633453 fetches on 432166 conns, 300 max parallel, 4.363350E+09 bytes, in 300 seconds
100 mean bytes/fetch
145444.9 fetches/sec, 1.454449E+07 bytes/sec
msecs/connect: 0.188 mean, 10.545 max, 0.041 min
msecs/first-response: 1.528 mean, 234.828 max, 0.082 min
{code}
So yeah, pretty sure there's a noticeable cost of doing this "copy", no?
was (Author: zwoop):
Ok, I re-ran all the benchmarks (with and without the latest patch here):
Before:
{code}
tinkerballa (20:22) 256/0 $ ~/benchit.sh 100 300
48759110 fetches on 482907 conns, 300 max parallel, 4.875910E+09 bytes, in 300 seconds
100 mean bytes/fetch
162530.2 fetches/sec, 1.625302E+07 bytes/sec
msecs/connect: 0.266 mean, 9.225 max, 0.042 min
msecs/first-response: 1.329 mean, 294.947 max, 0.083 min
{code}
After:
{code}
http_load -parallel 100 -seconds 300 -keep_alive 100 /tmp/URL
43633453 fetches on 432166 conns, 300 max parallel, 4.363350E+09 bytes, in 300 seconds
100 mean bytes/fetch
145444.9 fetches/sec, 1.454449E+07 bytes/sec
msecs/connect: 0.188 mean, 10.545 max, 0.041 min
msecs/first-response: 1.528 mean, 234.828 max, 0.082 min
{code}
So yeah, pretty sure there's a noticeable cost of doing this "copy", no?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421263#comment-13421263 ]
Nick Kew commented on TS-998:
-----------------------------
Yes, we have a workaround, and that's even been simplified recently. I gave up on it when it became clear that this bug is considered a Feature by some established users, so a simple/clean fix wouldn't be accepted.
I do recollect this going through a couple of iterations to try and coexist with the 'Feature'. From your last comment it's evident it got left in an Abandon state. So I guess it's time to abandon the fix and mark the bug WONTFIX?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.3.1
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188539#comment-13188539 ]
Leif Hedstrom commented on TS-998:
----------------------------------
Looking into this some more, I don't think any URLs are properly "printed" until after TS_HTTP_READ_REQUEST_HDR_HOOK. Is that too late for your plugin to get this information? The issue, I suspect, is that as the hook name implies, "parsing" of the request headers isn't done until after this hook, and therefore, e.g. the "host" part of the URL simply isn't populated (since it comes from the Host: header).
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187434#comment-13187434 ]
Leif Hedstrom commented on TS-998:
----------------------------------
Ok, I re-ran all the benchmarks (with and without the latest patch here):
Before:
{code}
tinkerballa (20:22) 256/0 $ ~/benchit.sh 100 300
48759110 fetches on 482907 conns, 300 max parallel, 4.875910E+09 bytes, in 300 seconds
100 mean bytes/fetch
162530.2 fetches/sec, 1.625302E+07 bytes/sec
msecs/connect: 0.266 mean, 9.225 max, 0.042 min
msecs/first-response: 1.329 mean, 294.947 max, 0.083 min
{code}
After:
{code}
http_load -parallel 100 -seconds 300 -keep_alive 100 /tmp/URL
43633453 fetches on 432166 conns, 300 max parallel, 4.363350E+09 bytes, in 300 seconds
100 mean bytes/fetch
145444.9 fetches/sec, 1.454449E+07 bytes/sec
msecs/connect: 0.188 mean, 10.545 max, 0.041 min
msecs/first-response: 1.528 mean, 234.828 max, 0.082 min
{code}
So yeah, pretty sure there's a noticeable cost of doing this "copy", no?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-998:
-----------------------------
Fix Version/s: (was: 3.1.5)
3.3.0
Moving this out to 3.3.0, please move back to 3.1.4 if this will be worked on *soon*. Also, take a look at what can be closed here please!
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.3.0
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187240#comment-13187240 ]
Leif Hedstrom commented on TS-998:
----------------------------------
So, I'm not sure I like this approach. What are the advantages here of making a "stringified" copy of the pristine request headers, instead of making a copy of the request object? If we made a copy of the original request object, and leave it immutable ("static"), we can keep using the normal APIs for getting headers etc. out of it, no? In a similar fashion, we already have TSHttpTxnPristineUrlGet(). So, we'd add something like TSHttpTxnPristineReqGet() or some such.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187279#comment-13187279 ]
Leif Hedstrom commented on TS-998:
----------------------------------
Hmmm, maybe I'm smoking crack. A fresh build from trunk did not show the same performance degradation as my "incremental" patch. So, I'll withdraw that -1 for now, but I still think we should consider a different approach, that lets us use the standard set of hdr object APIs.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420865#comment-13420865 ]
Leif Hedstrom commented on TS-998:
----------------------------------
We should probably try to figure this out, since out of the box, the new APIs are broken. I still don't like the stringification and performance hits of doing this on every request, where most likely someone would never use this new heap string.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.3.1
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187275#comment-13187275 ]
Leif Hedstrom commented on TS-998:
----------------------------------
I did a quick benchmark on the impact of this change: I'm seeing about 15% less throughput, with a 20% increase in latency. It's pretty expensive. At a minimum, I think we need to make this optional, but ideally, find a solution that's less expensive (I'm still thinking avoiding the stringification is the way to go).
-1 from me on this patch in it's current incarnation.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175550#comment-13175550 ]
Nick Kew commented on TS-998:
-----------------------------
r1221229 fixes the original problem.
Logfile entries are also fixed in my 3.0.1, but are broken in 3.1.1, so there's evidently still a problem. This breakage to the error log entries is also what induced me to drop attempts to access the data more directly.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-998:
-----------------------------
Fix Version/s: (was: 3.1.2)
3.1.3
Moving this out for v3.1.3 for now.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.3
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Alan M. Carroll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150748#comment-13150748 ]
Alan M. Carroll commented on TS-998:
------------------------------------
This is a side effect of forward transparency. The original logic assumed that all of the parts of the URL are in the first line of the request, the URL portion of which is represented as the URL class, which contains a MIME like pointer to the host. In a request like this, that never gets set because there's no host in the URL. I have put in some hacks to get around this (when I was not so familiar with the codebase) but it was really just a bunch of patches in various places (e.g., the "effective URL" calls and related stuff). I wonder now if it might not work to have the HTTP header parser reach in to the URL object and set these when it finds the HOST field. That presumes that (1) the original ("pristine") URL is stored somewhere and (2) the MIME and URL memory is contemporaneous.
On a more local level, this output is created in proxy/hdrs/URL.cc - url_print(). It always prints out the scheme (e.g. "http") followed by "://". Tweaking this might well have other side effects, so it might be better to look at proxy/hdrs/HTTP.cc - http_hdr_print and fiddle there.
Anyway, those are places to start.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Alan M. Carroll
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nick Kew reassigned TS-998:
---------------------------
Assignee: Nick Kew (was: Alan M. Carroll)
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-998:
-----------------------------
Fix Version/s: 3.1.2
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Alan M. Carroll
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (TS-998) Broken ClientReq in TSAPI
Posted by "Nick Kew (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175550#comment-13175550 ]
Nick Kew edited comment on TS-998 at 12/23/11 7:36 PM:
-------------------------------------------------------
r1221229 fixes the original problem.
Logfile entries are also fixed in my 3.0.1, but are broken in 3.1.1, so there's evidently still a problem. This breakage to the error log entries is also what induced me to drop attempts to access the data more directly.
Something is calculating a bogus length to the URL to be logged!
was (Author: nick@webthing.com):
r1221229 fixes the original problem.
Logfile entries are also fixed in my 3.0.1, but are broken in 3.1.1, so there's evidently still a problem. This breakage to the error log entries is also what induced me to drop attempts to access the data more directly.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187728#comment-13187728 ]
Leif Hedstrom commented on TS-998:
----------------------------------
So, the issue is just that you get the http:// in the URL "printout", is that the case? That's an artifact in how we represent the URLs, I'm fairly certain, but I can check that.
I'm still very weary on having this expensive "feature" in there, just to make this copy of the request string that almost no one will ever use. At a minimum, I think this needs to be disabled by default, and turned on via either an API addition, or a records.config (either is fine with me). Let me know if you need help making it records.config'urable.
I will look into fixing the PristineURL, but I don't think that will help you anyways. The URL "print" (or stringify getter) would still include the http:// I'm fairly certain.
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-998) Broken ClientReq in TSAPI
Posted by "Leif Hedstrom (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187303#comment-13187303 ]
Leif Hedstrom commented on TS-998:
----------------------------------
To complicate things even further, in addition to the "copy" of the request URL that TSHttpTxnPristineUrlGet() creates, we also make a stringified copy of the original ("pristine") request URL. This is used for logging (cquuc), and is stored in t_state.unmapped_request_url. With the changes above, are we not getting a third copy of the "pristine" URL now?
Nothing is ever easy, is it?
> Broken ClientReq in TSAPI
> -------------------------
>
> Key: TS-998
> URL: https://issues.apache.org/jira/browse/TS-998
> Project: Traffic Server
> Issue Type: Bug
> Affects Versions: 3.0.1
> Environment: any
> Reporter: Nick Kew
> Assignee: Nick Kew
> Fix For: 3.1.2
>
>
> Extracting a Request using TSHttpTxnClientReqGet API yields a bogus Request line.
> Expected behaviour: In a PRE_REMAP hook it should return the client request line and headers, ideally verbatim.
> Observed behaviour: "http://" is prepended to the request URL:
> GET /path/ HTTP/1.1
> becomes
> GET http:///path/ HTTP/1.1
> (yes, that's three slashes)
> Pseudo-code to reproduce from a PRE_REMAP hook:
> TSHttpTxnClientReqGet(txnp, &buf, &hdr);
> TSHttpHdrPrint(buf, hdr, iobuf);
> reader = TSIOBufferReaderAlloc(iobuf);
> block = TSIOBufferReaderStart(reader);
> len = TSIOBufferBlockReadAvail(block, reader);
> data = TSIOBufferBlockReadStart(block, reader, &len);
> Now examine the contents of data.
> Assigned to AMC as suggested yesterday on-list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira