You are viewing a plain text version of this content. The canonical link for it is here.
Posted to olio-dev@incubator.apache.org by "Shanti Subramanyam (JIRA)" <ji...@apache.org> on 2009/04/29 19:47:30 UTC
[jira] Created: (OLIO-90) Simulation of browser caches incorrect
Simulation of browser caches incorrect
--------------------------------------
Key: OLIO-90
URL: https://issues.apache.org/jira/browse/OLIO-90
Project: Olio
Issue Type: Bug
Components: generic
Affects Versions: 0.1
Reporter: Shanti Subramanyam
Assignee: Shanti Subramanyam
Priority: Minor
The driver simulates browser caches by not requesting the images included in a page 40% of the time. In reality, browsers make an 'if-modified-since' request to the server and the server checks if the file was modified later than the date specified in the request and if so it will return the file. Thus, this will incur an additional request/response cycle which we are not capturing in the driver/server interaction.
This bug will require fixes to both the driver and the server side. The driver can use a timestamp equivalent to when the run started. The server code (for PHP) must check for the existence of the if-modified-since flag. For rails, since the app does not service these files, the server will automatically take care of the request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (OLIO-90) Simulation of browser caches incorrect
Posted by "Shanti Subramanyam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shanti Subramanyam closed OLIO-90.
----------------------------------
> Simulation of browser caches incorrect
> --------------------------------------
>
> Key: OLIO-90
> URL: https://issues.apache.org/jira/browse/OLIO-90
> Project: Olio
> Issue Type: Bug
> Components: generic
> Affects Versions: 0.1
> Reporter: Shanti Subramanyam
> Assignee: Shanti Subramanyam
> Priority: Minor
> Fix For: 0.2
>
>
> The driver simulates browser caches by not requesting the images included in a page 40% of the time. In reality, browsers make an 'if-modified-since' request to the server and the server checks if the file was modified later than the date specified in the request and if so it will return the file. Thus, this will incur an additional request/response cycle which we are not capturing in the driver/server interaction.
> This bug will require fixes to both the driver and the server side. The driver can use a timestamp equivalent to when the run started. The server code (for PHP) must check for the existence of the if-modified-since flag. For rails, since the app does not service these files, the server will automatically take care of the request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (OLIO-90) Simulation of browser caches incorrect
Posted by "Shanti Subramanyam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shanti Subramanyam resolved OLIO-90.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.2
This issue was fixed by retrieving statics with a if-modified-since header when simulating browser caching behavior. For non-cached sessions, we still retrieve the statics only once per session. In this case, we assume that the application has set an expiry on the page that is well into the future. This is also true for the retrieval of all image files - we assume that these can be cached as long as the user session is valid.
On testing after the fix, there is a slight (0.5%) increase in network utilization at 500 users. So a large scale test is bound to show a more substantial increase in network traffic - these are all small packets as no real data is getting sent.
> Simulation of browser caches incorrect
> --------------------------------------
>
> Key: OLIO-90
> URL: https://issues.apache.org/jira/browse/OLIO-90
> Project: Olio
> Issue Type: Bug
> Components: generic
> Affects Versions: 0.1
> Reporter: Shanti Subramanyam
> Assignee: Shanti Subramanyam
> Priority: Minor
> Fix For: 0.2
>
>
> The driver simulates browser caches by not requesting the images included in a page 40% of the time. In reality, browsers make an 'if-modified-since' request to the server and the server checks if the file was modified later than the date specified in the request and if so it will return the file. Thus, this will incur an additional request/response cycle which we are not capturing in the driver/server interaction.
> This bug will require fixes to both the driver and the server side. The driver can use a timestamp equivalent to when the run started. The server code (for PHP) must check for the existence of the if-modified-since flag. For rails, since the app does not service these files, the server will automatically take care of the request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OLIO-90) Simulation of browser caches incorrect
Posted by "Shanti Subramanyam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718570#action_12718570 ]
Shanti Subramanyam commented on OLIO-90:
----------------------------------------
The fix is required only on the driver side. Since the requests we are talking about are all static (js, css, images), these are directly handled by the webserver and not by the server-side application.
Here is a proposed patch for the PHP driver. I do not plan to patch the other drivers at this point, as Kim is working on integrating the drivers and when that happens, this patch should be carried over to all the implementations.
--- src/org/apache/olio/workload/driver/UIDriver.java (revision 782811)
+++ src/org/apache/olio/workload/driver/UIDriver.java (working copy)
@@ -232,6 +232,7 @@
private HashSet<String> cachedURLs = new HashSet<String>();
private LinkedHashMap<String, String> loginHeaders =
new LinkedHashMap<String, String>();
+ private LinkedHashMap<String, String> cachedHeaders;
private UIDriverMetrics driverMetrics;
private long imgBytes = 0;
private int imagesLoaded = 0;
@@ -317,6 +318,10 @@
loginHeaders.put("Accept", "text/xml.application/xml,application/" +
"xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;" +
"q=0.5");
+
+ // We don't want the rest of the loginheaders in cachedHeaders
+ cachedHeaders = (LinkedHashMap)(loginHeaders.clone());
+
loginHeaders.put("Accept-Language", "en-us,en;q=0.5");
loginHeaders.put("Accept-Encoding", "gzip,deflate");
loginHeaders.put("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
@@ -325,7 +330,12 @@
loginHeaders.put("Referer", homepageURL);
isLoggedOn = false;
- isCached = cached();
+
+ // Create headers for if-modified-since
+ String ifmod = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss Z").format(BASE_DATE);
+ logger.finer("cachedhdr If-Modified-Since " + ifmod);
+ cachedHeaders.put("If-Modified-Since", ifmod);
+ isCached= cached();
}
@BenchmarkOperation (
@@ -345,12 +355,15 @@
throw new IOException("Received empty response");
Set<String> images = parseImages(responseBuffer);
+ /*****
if (!isCached) {
-
+ ****/
// Fetch the CSS/JS files
loadStatics(homepageStatics);
+ /****
}
+ ***/
loadImages(images);
selectedEvent = RandomUtil.randomEvent(random, responseBuffer);
logger.finer("Images loaded: " + imagesLoaded);
@@ -686,15 +699,22 @@
}
private void loadStatics(String[] urls) throws IOException {
-
+ /**
if (!isCached)
- for (String url : urls)
- if (cachedURLs.add(url)) {
- logger.finer("Loading URL " + url);
- http.readURL(url);
- } else {
+ **/
+ for (String url : urls) {
+ // If we are simulating browser caching, send if-modified-since header
+ // Don't bother checking return code as we really don't want to fetch page
+ if (isCached)
+ http.readURL(url, cachedHeaders);
+ else {
+ if (cachedURLs.add(url)) {
+ logger.finer("Loading URL " + url);
+ http.readURL(url);
+ } else
logger.finer("URL already cached: Not loading " + url);
- }
+ }
+ }
}
public DateFormat getDateFormat() {
> Simulation of browser caches incorrect
> --------------------------------------
>
> Key: OLIO-90
> URL: https://issues.apache.org/jira/browse/OLIO-90
> Project: Olio
> Issue Type: Bug
> Components: generic
> Affects Versions: 0.1
> Reporter: Shanti Subramanyam
> Assignee: Shanti Subramanyam
> Priority: Minor
>
> The driver simulates browser caches by not requesting the images included in a page 40% of the time. In reality, browsers make an 'if-modified-since' request to the server and the server checks if the file was modified later than the date specified in the request and if so it will return the file. Thus, this will incur an additional request/response cycle which we are not capturing in the driver/server interaction.
> This bug will require fixes to both the driver and the server side. The driver can use a timestamp equivalent to when the run started. The server code (for PHP) must check for the existence of the if-modified-since flag. For rails, since the app does not service these files, the server will automatically take care of the request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (OLIO-90) Simulation of browser caches incorrect
Posted by "Shanti Subramanyam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768489#action_12768489 ]
Shanti Subramanyam commented on OLIO-90:
----------------------------------------
The same fix has now gone into all 3 drivers. Since the drivers won't be merged in 0.2, it is essential to keep them synchronized.
> Simulation of browser caches incorrect
> --------------------------------------
>
> Key: OLIO-90
> URL: https://issues.apache.org/jira/browse/OLIO-90
> Project: Olio
> Issue Type: Bug
> Components: generic
> Affects Versions: 0.1
> Reporter: Shanti Subramanyam
> Assignee: Shanti Subramanyam
> Priority: Minor
> Fix For: 0.2
>
>
> The driver simulates browser caches by not requesting the images included in a page 40% of the time. In reality, browsers make an 'if-modified-since' request to the server and the server checks if the file was modified later than the date specified in the request and if so it will return the file. Thus, this will incur an additional request/response cycle which we are not capturing in the driver/server interaction.
> This bug will require fixes to both the driver and the server side. The driver can use a timestamp equivalent to when the run started. The server code (for PHP) must check for the existence of the if-modified-since flag. For rails, since the app does not service these files, the server will automatically take care of the request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.