You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by "Sean Cosgrave (JIRA)" <ji...@apache.org> on 2010/02/11 03:16:29 UTC
[jira] Created: (TS-167) Support InkHttpFetchPage(s) in the TS API
Support InkHttpFetchPage(s) in the TS API
-----------------------------------------
Key: TS-167
URL: https://issues.apache.org/jira/browse/TS-167
Project: Traffic Server
Issue Type: New Feature
Components: InkAPI
Reporter: Sean Cosgrave
This is the original description from Anirban:
There are quite a few people that are attempting to write plugins that download data from an origin server (or other
HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
The input into the function would be
1. provide the headers passed into YTS as part of the request
- the header should obviously include the url to fetch
2. provide the moment when to wake up the plugin's continuation. Options include
0 - as soon as the request is made - no attempt is made to wait for the download to happen
1 - as soon as the headers are downloaded
2 - as soon as a portion of the message is downloaded
3 - after the full download is complete
3. ability to not cache the data (optional) (only used if one would like to override the data
4. ability to not run plugins (optional)
The output of the function would be
1. for each of the results
- error codes that define the download status of each of the requests
- IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
- time to download each object (not so critical)
Note: This fxn doesnt support streaming the answer back to the requesting client.
The fetches should be asynchronous and simultaneous.
A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-167) Support InkHttpFetchPage(s) in the TS API
Posted by "Sean Cosgrave (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Cosgrave updated TS-167:
-----------------------------
Attachment: FetchSM.h
FetchSM.cc
re-attaching these, they needed the license header and indenting.
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Attachments: FetchSM.cc, FetchSM.h, fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-167) Support InkHttpFetchPage(s) in the TS API
Posted by "Sean Cosgrave (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Cosgrave updated TS-167:
-----------------------------
Attachment: (was: FetchSM.cc)
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Attachments: fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TS-167) Support InkHttpFetchPage(s) in the TS
API
Posted by "Sean Cosgrave (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832364#action_12832364 ]
Sean Cosgrave commented on TS-167:
----------------------------------
I had a diff ready for this, until I realized there had been further changes made to the files to facilitate more functionality for this feature. I will add a diff in the morning.
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-167) Support InkHttpFetchPage(s) in the TS API
Posted by "Sean Cosgrave (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Cosgrave updated TS-167:
-----------------------------
Attachment: fp_fetch_diff.txt
FetchSM.h
FetchSM.cc
These are the 2 additional files that need to be added to TS, plus the diffs needed to implement the requested feature. It should be noted that the changes here are the first effort to implement the feature and have not been fully tested.
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Attachments: fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TS-167) Support InkHttpFetchPage(s) in the TS
API
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834904#action_12834904 ]
George Paul commented on TS-167:
--------------------------------
If possible it would be nice if a Regression test for this new INK API call could be added to 'proxy/InkAPITest.cc'.
-George
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Attachments: FetchSM.cc, FetchSM.h, fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-167) Support InkHttpFetchPage(s) in the TS API
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-167:
-----------------------------
Fix Version/s: 2.1.0
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Fix For: 2.1.0
>
> Attachments: FetchSM.cc, FetchSM.h, fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (TS-167) Support InkHttpFetchPage(s) in the TS API
Posted by "Sean Cosgrave (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Cosgrave updated TS-167:
-----------------------------
Attachment: (was: FetchSM.h)
> Support InkHttpFetchPage(s) in the TS API
> -----------------------------------------
>
> Key: TS-167
> URL: https://issues.apache.org/jira/browse/TS-167
> Project: Traffic Server
> Issue Type: New Feature
> Components: InkAPI
> Reporter: Sean Cosgrave
> Attachments: fp_fetch_diff.txt
>
>
> This is the original description from Anirban:
> There are quite a few people that are attempting to write plugins that download data from an origin server (or other
> HTTP sites) and then optionally modify the contents like the transformation plugins allow for.
> However, the downloading of the pages is cumbersome and a repeated task as this, should be wrapped in an API.
> The input into the function would be
> 1. provide the headers passed into YTS as part of the request
> - the header should obviously include the url to fetch
> 2. provide the moment when to wake up the plugin's continuation. Options include
> 0 - as soon as the request is made - no attempt is made to wait for the download to happen
> 1 - as soon as the headers are downloaded
> 2 - as soon as a portion of the message is downloaded
> 3 - after the full download is complete
> 3. ability to not cache the data (optional) (only used if one would like to override the data
> 4. ability to not run plugins (optional)
> The output of the function would be
> 1. for each of the results
> - error codes that define the download status of each of the requests
> - IOBuffer ptr (or char*) to the results (the caller of the fxn has to release references to the IOBuffers or if char* is used free the chunk)
> - time to download each object (not so critical)
> Note: This fxn doesnt support streaming the answer back to the requesting client.
> The fetches should be asynchronous and simultaneous.
> A possible model would be to have the user call INKHttpFetchPages which then in turn calls INKHttpFetchPage per request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.