You are viewing a plain text version of this content. The canonical link for it is here.
Posted to olio-dev@incubator.apache.org by "Mandy Waite (JIRA)" <ji...@apache.org> on 2009/01/16 12:05:59 UTC
[jira] Created: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Olio Rails driver causes no data to be added to the DB due to Session management issues
---------------------------------------------------------------------------------------
Key: OLIO-36
URL: https://issues.apache.org/jira/browse/OLIO-36
Project: Olio
Issue Type: Bug
Components: rails-driver
Environment: OpenSolaris 2008.11 for SUT, DB and Faban
Reporter: Mandy Waite
Assignee: Mandy Waite
This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
- doAddEvent() is always called in a new and separate session and so fails
- doPersonDetail() is always called in a new and separate session and so fails
- doAddAttendee() is always called in a new and separate session and so fails
- Olio Rails application requires use of authenticity_token exchange between it and client for many ops
- Post Parameters for event_image and event_document are incorrectly specified
- Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
- Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
- Phone numbers generated by RandomUtil don't work with the Rails App
- Scraping of event page results in some invalid event ids being used to view events
- Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
- Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
- Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
- EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
- Had to change Timing to manual in several ops because of the use of Apache HttpClient
- FileLoader.java uses wrong names for thumbnails
Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "Mandy Waite (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mandy Waite updated OLIO-36:
----------------------------
Attachment: Rails_RandomUtil_main.patch
Rails_UIDriver_main.patch
The fixes in the attached patches for com.sun.web20.driver.UIDriver and com.sun.web20.util.RandomUtil have been used in testing for a month now and they fix all of the problems associated with this issue. They may need some fine tuning, for example currently we use a single httpclient for each simulated user, as opposed to creating a new one every time a user logs in. The patches are based on svn revision 738549.
The fixes also require changes to the Rails application as the Apache httpclient can not generate useful content_type headers without including 'charset=<some encoding>" as part of the content type. The changes made include adding charset='8859-1' to the end of the 'image/jpeg' and 'application/pdf' which is the default if no encoding is specified when adding the files to the PostMethod in the code. The changes are:
diff -r -u olioapp/app/models/document.rb olioapp-working/app/models/document.rb
--- olioapp/app/models/document.rb 2008-10-20 12:32:04.000000000 -0700
+++ olioapp-working/app/models/document.rb 2008-12-30 10:09:28.372724723 -0
800
@@ -11,7 +11,7 @@
class Document < ActiveRecord::Base
- has_attachment :content_type => ['application/pdf', 'application/msword', 'text/plain'],
+ has_attachment :content_type => ['application/pdf', 'application/pdf; charset=ISO-8859-1', 'application/msword', 'text/plain'],
:storage => :file_system,
:path_prefix => DOCUMENT_STORE_PATH
=============================================================================
diff -r -u olioapp/vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu.rb olioapp-working/vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu.rb
--- olioapp/vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu.rb 2008-10-20 12:32:14.000000000 -0700
+++ olioapp-working/vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu.rb 2009-01-01 06:10:33.836562068 -0800
@@ -2,7 +2,7 @@
module AttachmentFu # :nodoc:
@@default_processors = %w(ImageScience Rmagick MiniMagick Gd2 CoreImage)
@@tempfile_path = File.join(RAILS_ROOT, 'tmp', 'attachment_fu')
- @@content_types = ['image/jpeg', 'image/pjpeg', 'image/gif', 'image/png', 'image/x-png', 'image/jpg']
+ @@content_types = ['image/jpeg; charset=ISO-8859-1', 'image/jpeg', 'image/pjpeg', 'image/gif', 'image/png', 'image/x-png', 'image/jpg']
mattr_reader :content_types, :tempfile_path, :default_processors
mattr_writer :tempfile_path
Apache HttpClient also doesn't handle redirects for Post (nor should it), but it emits a warning when a redirect was received. This can be suppressed by adding:
org.apache.commons.httpclient.HttpMethodBase.level=SEVERE
to $FABAN_HOME/config/logging.properties
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: Mandy Waite
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "William Sobel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
William Sobel resolved OLIO-36.
-------------------------------
Resolution: Fixed
Added the content types. In image.rb append the new content type to the existing list so we don't modify the vendor code.
Author: wsobel
Date: Wed Jan 28 18:23:01 2009
New Revision: 738575
URL: http://svn.apache.org/viewvc?rev=738575&view=rev
Log:
OLIO-36: Added charset to content types for attachment fu. (Mandy Waite)
Modified:
incubator/olio/webapp/rails/trunk/app/models/document.rb
incubator/olio/webapp/rails/trunk/app/models/image.rb
Modified: incubator/olio/webapp/rails/trunk/app/models/document.rb
URL: http://svn.apache.org/viewvc/incubator/olio/webapp/rails/trunk/app/models/document.rb?rev=738575&r1=738574&r2=738575&view=diff
==============================================================================
--- incubator/olio/webapp/rails/trunk/app/models/document.rb (original)
+++ incubator/olio/webapp/rails/trunk/app/models/document.rb Wed Jan 28 18:23:01 2009
@@ -11,7 +11,8 @@
class Document < ActiveRecord::Base
- has_attachment :content_type => ['application/pdf', 'application/msword', 'text/plain'],
+ has_attachment :content_type => ['application/pdf', 'application/pdf; charset=ISO-8859-1',
+ 'application/msword', 'text/plain'],
:storage => :file_system,
:path_prefix => DOCUMENT_STORE_PATH
Modified: incubator/olio/webapp/rails/trunk/app/models/image.rb
URL: http://svn.apache.org/viewvc/incubator/olio/webapp/rails/trunk/app/models/image.rb?rev=738575&r1=738574&r2=738575&view=diff
==============================================================================
--- incubator/olio/webapp/rails/trunk/app/models/image.rb (original)
+++ incubator/olio/webapp/rails/trunk/app/models/image.rb Wed Jan 28 18:23:01 2009
@@ -13,6 +13,10 @@
# thumbnail :string(255)
#
+unless Technoweenie::AttachmentFu.content_types.include? 'image/jpeg; charset=ISO-8859-1'
+ Technoweenie::AttachmentFu.content_types << 'image/jpeg; charset=ISO-8859-1'
+end
+
class Image < ActiveRecord::Base
has_attachment :content_type => :image,
:storage => :file_system,
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: William Sobel
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "William Sobel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
William Sobel closed OLIO-36.
-----------------------------
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: William Sobel
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "William Sobel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
William Sobel reassigned OLIO-36:
---------------------------------
Assignee: William Sobel (was: Mandy Waite)
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: William Sobel
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "William Sobel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
William Sobel reopened OLIO-36:
-------------------------------
Need to apply the driver patch as well.
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: William Sobel
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (OLIO-36) Olio Rails driver causes no data to be
added to the DB due to Session management issues
Posted by "William Sobel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OLIO-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
William Sobel resolved OLIO-36.
-------------------------------
Resolution: Fixed
Author: wsobel
Date: Wed Jan 28 18:30:44 2009
New Revision: 738580
URL: http://svn.apache.org/viewvc?rev=738580&view=rev
Log:
OLIO-36: Applied patches to Rails driver to fix session management issue (Mandy Waite)
Modified:
incubator/olio/workload/rails/trunk/src/com/sun/web20/driver/UIDriver.java
incubator/olio/workload/rails/trunk/src/com/sun/web20/util/RandomUtil.java
Modified: incubator/olio/workload/rails/trunk/src/com/sun/web20/driver/UIDriver.java
URL: http://svn.apache.org/viewvc/incubator/olio/workload/rails/trunk/src/com/sun/web20/driver/UIDriver.java?rev=738580&r1=738579&r2=738580&view=diff
==============================================================================
--- incubator/olio/workload/rails/trunk/src/com/sun/web20/driver/UIDriver.java (original)
+++ incubator/olio/workload/rails/trunk/src/com/sun/web20/driver/UIDriver.java Wed Jan 28 18:30:44 2009
@@ -40,6 +40,10 @@
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.logging.Logger;
+import org.apache.commons.httpclient.methods.GetMethod;
+import org.apache.commons.httpclient.methods.PostMethod;
+import org.apache.commons.httpclient.methods.multipart.FilePart;
+import org.apache.commons.httpclient.methods.multipart.Part;
@BenchmarkDefinition (
name = "Web20Bench",
@@ -59,11 +63,11 @@
operations = { "HomePage", "Login", "TagSearch", "EventDetail", "PersonDetail", "AddPerson", "AddEvent" },
mix = { @Row({ 0, 10, 50, 35, 0, 5, 0 }), // Home Page
- @Row({ 0, 0, 60, 20, 0, 0, 20 }), // Login
+ @Row({ 0, 0, 30, 20, 20, 0, 30 }), // Login
@Row({ 20, 5, 40, 30, 0, 5, 0 }), // Tag Search
@Row({ 70, 20, 0, 0, 0, 5, 0 }), // Event Detail
@Row({ 0, 0, 0, 30, 10, 0, 30 }), // Person Detail
- @Row({ 30, 65, 0, 0, 15, 0, 0 }), // Add Person
+ @Row({ 30, 65, 0, 0, 0, 0, 0 }), // Add Person
@Row({ 0, 0, 25, 75, 15, 0, 0 }) // Add Event
}
)
@@ -302,6 +306,15 @@
"/images/main_nav_hover_bg.gif"
};
+ /* Custom metric targets */
+ public static final double EVENT_VIEWS_WHERE_ATTENDEE_ATTENDED_TARGET = 8d;
+ public static final double AVG_IMAGE_REFS_PER_HOMEPAGE_TARGET = 10d;
+ public static final double AVG_IMAGES_LOADED_PER_HOMEPAGE_TARGET = 3d;
+ public static final double AVG_IMAGE_BYTES_PER_HOMEPAGE_TARGET = 20000d;
+ public static final double AVG_IMAGE_PER_TAG_SEARCH_TARGET = 10;
+ public static final double AVG_IMGS_PER_EVENT_DETAIL_TARGET = 0d;
+
+
// We just need today's date. java.sql.date does not have any time anyway.
public static final java.sql.Date BASE_DATE =
new java.sql.Date(System.currentTimeMillis());
@@ -338,6 +351,8 @@
private int imagesLoaded = 0;
private String tagCloudURL;
+ private HttpClient httpClient;
+
public UIDriver() throws XPathExpressionException {
ctx = DriverContext.getContext();
int scale = ctx.getScale();
@@ -420,6 +435,8 @@
isLoggedOn = false;
isCached = cached();
+ httpClient = new HttpClient();
+ httpClient.setConnectionTimeout(5000);
}
@BenchmarkOperation (
@@ -458,39 +475,50 @@
@BenchmarkOperation (
name = "Login",
max90th = 1,
- timing = Timing.AUTO
+ timing = Timing.MANUAL
)
public void doLogin() throws IOException, Exception {
+
+ httpClient = new HttpClient();
logger.finer("In doLogin");
int randomId = 0; //use as password
username = null;
- if (!isLoggedOn) {
+ if (isLoggedOn) {
+ //already logged in --> logout,then log in again
+ doLogout();
+ }
+
+ // http.fetchURL(loginURL);
+ // HttpClient _httpClient = new HttpClient();
+ // _httpClient.setConnectionTimeout(5000);
+
+ // httpClient = _httpClient;
+
randomId = selectUserID();
username = UserName.getUserName(randomId);
logger.fine("Logging in as " + username + ", " + randomId);
- http.readURL(loginURL, constructLoginPost(randomId), loginHeaders);
- // This redirects to home.
- http.fetchURL(homepageURL);
- int loginIdx = http.getResponseBuffer().indexOf("Login:");
+ PostMethod loginPost = constructLoginPost(randomId);
+
+ ctx.recordTime();
+
+ httpClient.executeMethod(loginPost);
+
+ // Should redirect automatically but doesn't seem to
+ GetMethod loginGet = new GetMethod(homepageURL);
+ httpClient.executeMethod(loginGet);
+
+ ctx.recordTime();
+
+ int loginIdx = loginGet.getResponseBodyAsString().indexOf("Login:");
if (loginIdx != -1)
throw new Exception(" Found login prompt at index " + loginIdx);
- /*if (http.getResponseBuffer().indexOf("Login:") != 1216) {
- logger.finest(http.getResponseBuffer().toString());
- //throw new RuntimeException("Index of LOGIN = " +http.getResponseBuffer().indexOf("Login:") );
- //logger.info(http.getResponseBuffer().toString());
- throw new RuntimeException("Login as " + username + ", " +
- randomId + " failed.");
- }*/
- //logger.fine("Login successful as " + username + ", " + randomId);
+ logger.fine("Login successful as " + username + ", " + randomId);
isLoggedOn=true;
- } else {
- //already logged in --> logout,then log in again
- doLogout();
- doLogin();
+
+
}
- }
@BenchmarkOperation (
@@ -501,11 +529,11 @@
public void doLogout() throws IOException {
if (isLoggedOn){
logger.finer("Logging off = " + isLoggedOn);
- http.fetchURL(logoutURL);
+ httpClient.executeMethod(new GetMethod(logoutURL));
cachedURLs.clear();
isCached = cached();
isLoggedOn=false;
- http = new HttpTransport(); // clear all state
+ // httpClient = new HttpClient(); // clear all state
}
}
@@ -539,14 +567,20 @@
timing = Timing.MANUAL
)
public void doAddEvent() throws IOException {
- logger.finer("doAddEvent");
+ logger.finer("entering doAddEvent()");
+ if(!isLoggedOn)
+ throw new IOException("User not logged when trying to add an event");
+
ctx.recordTime();
- http.readURL(addEventURL);
loadStatics(addEventStatics);
MultipartPostMethod post = new MultipartPostMethod(addEventResultURL);
- if(isLoggedOn) {
+ GetMethod eventForm = new GetMethod(addEventURL);
+
+ // TODO: Implement prepareEvent() for Rails form data
StringBuilder buffer = new StringBuilder(256);
+ post.addParameter("commit", "Create");
+ // post.addParameter("controller", "events");
post.addParameter("event[title]", RandomUtil.randomText(random, 15, 20));
post.addParameter("event[summary]", RandomUtil.randomText(random, 50, 200));
post.addParameter("event[description]", RandomUtil.randomText(random, 100, 495));
@@ -557,15 +591,27 @@
post.addParameter("event[event_timestamp(4i)]", "20");
post.addParameter("event[event_timestamp(5i)]", "10");
- // We do the images last, not to split the fields into parts
- post.addParameter("event[image]", eventImg);
- post.addParameter("event[document]", eventPdf);
- post.addParameter("tag_list", random.makeCString(4, 14));
+ Part imagePart = new FilePart("event_image", eventImg, "image/jpeg", null);
+ Part docPart = new FilePart("event_document", eventPdf, "application/pdf", null);
+
+ post.addPart(imagePart);
+ post.addPart(docPart);
+ post.addParameter("tag_list", "tag1");
addAddress(post);
+ // GET the new event form within a user session
+ httpClient.executeMethod(eventForm);
+ String responseBuffer = eventForm.getResponseBodyAsString();
+ if (responseBuffer.length() == 0)
+ throw new IOException("Received empty response");
+
+ // Parse the authenticity_token from the response
+ String token = parseAuthToken(responseBuffer);
+
+ post.addParameter("authenticity_token", token);
+
doMultiPartPost(post);
- }
ctx.recordTime();
++driverMetrics.addEventTotal;
@@ -581,12 +627,11 @@
if (isLoggedOn)
doLogout();
- ctx.recordTime();
+ ctx.recordTime(); // Start critical section
http.readURL(addPersonURL);
- loadStatics(addPersonStatics);
-
MultipartPostMethod post = new MultipartPostMethod(addPersonResultURL);
+ // TODO: Implement preparePerson() for Rails form data
String fields[] = new String[8];
StringBuilder b = new StringBuilder(256);
int id = loadedUsers + personsAdded++ * ScaleFactors.activeUsers +
@@ -609,11 +654,14 @@
post.addParameter("user[summary]", RandomUtil.randomText(random, 50, 200));
post.addParameter("user[timezone]", RandomUtil.randomTimeZone(random));
- // Images
- post.addParameter("user_image", personImg);
+ Part imagePart = new FilePart("user_image", personImg, "image/jpeg", null);
+ post.addPart(imagePart);
addAddress(post);
+
+ loadStatics(addPersonStatics);
doMultiPartPost(post);
+
ctx.recordTime();
++driverMetrics.addPersonTotal;
}
@@ -666,17 +714,19 @@
@BenchmarkOperation (
name = "PersonDetail",
max90th = 2,
- timing = Timing.AUTO
+ timing = Timing.MANUAL
)
public void doPersonDetail() throws IOException {
logger.finer("doPersonDetail");
StringBuilder buffer = new StringBuilder(fileServiceURL.length() + 20);
//buffer.append(fileServiceURL).append("file=p");
+ ctx.recordTime();
if (isLoggedOn) {
int id = random.random(1, ScaleFactors.users);
- http.fetchURL(personDetailURL + id);
- StringBuilder responseBuffer = http.getResponseBuffer();
+ GetMethod personDetailGet = new GetMethod(personDetailURL + id);
+ httpClient.executeMethod(personDetailGet);
+ StringBuilder responseBuffer = new StringBuilder(personDetailGet.getResponseBodyAsString());
if (responseBuffer.length() == 0)
throw new IOException("Received empty response");
@@ -696,18 +746,45 @@
logger.warning("Trying to view user, but not logged in");
http.fetchURL(homepageURL);
}
+ ctx.recordTime();
}
public void doAddAttendee() throws IOException {
//can only add yourself (one attendee) to party
- http.readURL(addAttendeeURL + selectedEvent + "/attend", "");
+ PostMethod attendeePost = new PostMethod(addAttendeeURL + selectedEvent + "/attend");
+ int status = httpClient.executeMethod(attendeePost);
+
+ switch (status) {
+ case HttpStatus.SC_ACCEPTED:
+ logger.finer("Status = SC_ACCEPTED");
+ break;
+ case HttpStatus.SC_MOVED_TEMPORARILY:
+ logger.finer("Status = SC_MOVED_TEMPORARILY");
+ // System.out.println(post.getResponseBodyAsString());
+ // for (Header header: post.getResponseHeaders()) {
+ // logger.info(header.getName() + ":" + header.getValue());
+ // }
+ httpClient.executeMethod(new GetMethod(attendeePost.getResponseHeader("Location").getValue()));
+ break;
+ case HttpStatus.SC_OK:
+ logger.finer("Status = SC_OK");
+ break;
+ default:
+ String statusHeaderStr = attendeePost.getResponseHeader("Status").getValue();
+ logger.finer("Status = " + statusHeaderStr);
+ }
+ if (status != HttpStatus.SC_OK && status != HttpStatus.SC_MOVED_TEMPORARILY) {
+ throw new IOException("GET operation failed");
+ }
+
// http.readURL(updatePageURL, "list=attendees");
}
public Set<String> parseImages(StringBuilder buffer) {
LinkedHashSet<String> urlSet = new LinkedHashSet<String>();
- String elStart = "<img ";
- String attrStart = " src=\"";
+ //String elStart = "<img ";
+ String elStart = "background: ";
+ String attrStart = " url(";
int elStartLen = elStart.length() - 1; // Don't include the trailing space
int attrStartLen = attrStart.length();
int idx = 0;
@@ -719,10 +796,11 @@
if (idx == -1)
break;
idx += elStartLen;
- int endIdx = buffer.indexOf("/>", idx);
+ int endIdx = buffer.indexOf(")", idx) + 1; // +1 to include the '('
if (endIdx == -1)
break;
String elText = buffer.substring(idx, endIdx);
+ logger.finest(elText);
idx = endIdx + 1;
// Find the attribute
@@ -731,34 +809,67 @@
logger.finer("No img src attribute. Weird! " + elText);
continue;
}
- endIdx = elText.indexOf("\"", idx2 + attrStartLen);
+ endIdx = elText.indexOf(")", idx2 + attrStartLen);
if (endIdx == -1) {
logger.warning("No img src attribute ending. Weird! " + elText);
continue;
}
String link = elText.substring(idx2 + attrStartLen, endIdx);
- if (link.startsWith("fileService.")) {
+ if (link.startsWith("/filestore")) {
String url = baseURL + '/' + link;
- logger.finest("Adding " + url + " from idx " + idx);
+ logger.finer("Adding " + url + " from idx " + idx);
urlSet.add(url);
}
}
return urlSet;
}
+ private String parseAuthToken(String responseBuffer) throws IOException {
+
+ int idx = responseBuffer.indexOf("authenticity_token");
+
+ if (idx == -1)
+ throw new IOException("Trying to add event but authenticity token not found");
+
+ int endIdx = responseBuffer.indexOf("\" />", idx);
+
+ if (endIdx == -1)
+ throw new IOException("Invalid authenticity_token element. Buffer(100 chars) = " +
+ responseBuffer.substring(idx, idx + 100));
+
+ String tmpString = responseBuffer.substring(idx, endIdx);
+ String[] splitStr = tmpString.split("value=\"");
+
+ if (splitStr.length < 2)
+ throw new IOException("Invalid authenticity_token element. Buffer(100 chars) = " +
+ responseBuffer.substring(idx, idx + 100));
+
+ String token = splitStr[1];
+
+ logger.finer("authenticity_token = " + token);
+
+ return token;
+
+ }
+
private void loadImages(Set<String> images) throws IOException {
+ logger.finer("loadImages()");
+ logger.finest("No. images = " + images.size());
+
if (images != null)
for (String image : images)
// Loads image only if not cached, means we can add to cache.
if (cachedURLs.add(image)) {
- logger.finer("Loading image " + image);
+ logger.finest("Loading image " + image);
imgBytes += http.readURL(image);
++imagesLoaded;
} else {
- logger.finer("Image already cached: Not loading " + image);
+ logger.finest("Image already cached: Not loading " + image);
}
+ else
+ logger.finest("images == null");
}
private boolean cached() {
@@ -814,20 +925,53 @@
return returnList;
}
- private String constructLoginPost(int randomId) {
- return "users[username]=" + username + "&users[password]=" +
- String.valueOf(randomId) + "&submit=Login";
+ private PostMethod constructLoginPost(int randomId) {
+ PostMethod loginPost = new PostMethod(loginURL);
+
+ loginPost.setFollowRedirects(true);
+ loginPost.addParameter("users[username]", username);
+ loginPost.addParameter("users[password]", String.valueOf(randomId));
+ loginPost.addParameter("submit", "Login");
+
+ return loginPost;
}
public void doMultiPartPost(MultipartPostMethod post) throws IOException {
+ logger.finer("In doMultiPartPost()");
- HttpClient client = new HttpClient();
- client.setConnectionTimeout(5000);
- int status = client.executeMethod(post);
- if(status != HttpStatus.SC_OK)
+ if (httpClient == null) {
+ logger.warning("HttpClient is null, this shouldn't happen");
+ httpClient = new HttpClient();
+ httpClient.setConnectionTimeout(5000);
+ }
+
+ post.setFollowRedirects(false);
+ int status = httpClient.executeMethod(post);
+ switch (status) {
+ case HttpStatus.SC_ACCEPTED:
+ logger.finer("Status = SC_ACCEPTED");
+ break;
+ case HttpStatus.SC_MOVED_TEMPORARILY:
+ logger.finer("Status = SC_MOVED_TEMPORARILY");
+ // System.out.println(post.getResponseBodyAsString());
+ // for (Header header: post.getResponseHeaders()) {
+ // logger.info(header.getName() + ":" + header.getValue());
+ // }
+ httpClient.executeMethod(new GetMethod(post.getResponseHeader("Location").getValue()));
+ break;
+ case HttpStatus.SC_OK:
+ logger.finer("Status = SC_OK");
+ break;
+ default:
+ String statusHeaderStr = post.getResponseHeader("Status").getValue();
+ logger.finer("Status = " + statusHeaderStr);
+ }
+ if (status != HttpStatus.SC_OK && status != HttpStatus.SC_MOVED_TEMPORARILY) {
throw new IOException("Multipart Post did not work");
}
+ }
+ // TODO: implement prepareAddress() for Rails form data
public void addAddress(MultipartPostMethod post)
{
//add the address
@@ -988,7 +1132,7 @@
if (total > 0) {
double avgImgs = eventDetailImages / (double) total;
el[6].result = String.format("%.2f", avgImgs);
- if (avgImgs >= 9d)
+ if (avgImgs >= 1d)
el[6].passed = Boolean.TRUE;
else
el[6].passed = Boolean.FALSE;
Modified: incubator/olio/workload/rails/trunk/src/com/sun/web20/util/RandomUtil.java
URL: http://svn.apache.org/viewvc/incubator/olio/workload/rails/trunk/src/com/sun/web20/util/RandomUtil.java?rev=738580&r1=738579&r2=738580&view=diff
==============================================================================
--- incubator/olio/workload/rails/trunk/src/com/sun/web20/util/RandomUtil.java (original)
+++ incubator/olio/workload/rails/trunk/src/com/sun/web20/util/RandomUtil.java Wed Jan 28 18:30:44 2009
@@ -174,21 +174,21 @@
"W-SU", "WET", "Zulu"};
- // Phone comes as +1 888 999 0000 or +77 66 999 0000 for non-US.
+ // Phone is 0018889990000 or 0077669990000 for non-US (no spaces for Rails app)
// 50% of the time, do US, 50% non-us.
public static String randomPhone(Random r, StringBuilder b) {
String v = r.makeNString(1, 2);
if (v.length() == 1) {
- b.append("+1 ");
+ b.append("001"); // removed space
v = r.makeNString(3, 3);
- b.append(v).append(' ');
+ b.append(v); // removed space
} else {
- b.append("+").append(v);
+ b.append("00").append(v);
v = r.makeNString(2, 2);
- b.append(' ').append(v).append(' ');
+ b.append(v); //removed spaces
}
v = r.makeNString(3, 3);
- b.append(v).append(' ');
+ b.append(v); // removed space
v = r.makeNString(4, 4);
b.append(v);
return b.toString();
@@ -291,6 +291,7 @@
String search1 = "<a href=\"/events/";
int idx = 0;
HashSet<String> eventIdSet = new HashSet<String>();
+ String eventItem = "";
for (;;) {
idx = eventListPage.indexOf(search1, idx);
if (idx == -1)
@@ -300,6 +301,9 @@
int endIdx = eventListPage.indexOf("\"", idx);
if (endIdx == -1)
break;
+
+ eventItem = eventListPage.substring(idx, endIdx).trim();
+ if (!eventItem.contains("tagged") && !eventItem.contains("new") )
eventIdSet.add(eventListPage.substring(idx, endIdx).trim());
idx = endIdx;
}
> Olio Rails driver causes no data to be added to the DB due to Session management issues
> ---------------------------------------------------------------------------------------
>
> Key: OLIO-36
> URL: https://issues.apache.org/jira/browse/OLIO-36
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Environment: OpenSolaris 2008.11 for SUT, DB and Faban
> Reporter: Mandy Waite
> Assignee: William Sobel
> Attachments: Rails_RandomUtil_main.patch, Rails_UIDriver_main.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This by necessity is a meta-bug as all of the described issues were fixed in order to make the driver work correctly during testing.
> When running tests on the Olio Rails Driver, the following issues were observed (some were masked by others):
> - doAddEvent() is always called in a new and separate session and so fails
> - doPersonDetail() is always called in a new and separate session and so fails
> - doAddAttendee() is always called in a new and separate session and so fails
> - Olio Rails application requires use of authenticity_token exchange between it and client for many ops
> - Post Parameters for event_image and event_document are incorrectly specified
> - Post Parameters added with the correct 'content_type' so as to be acceptable to the Rails app
> - Rails app needs changing to accept content_typeS with '; charset=..." (Side effect of using Apache HttpClient Parts)
> - Phone numbers generated by RandomUtil don't work with the Rails App
> - Scraping of event page results in some invalid event ids being used to view events
> - Timing for doAddEvent() is measured even if the op isn't run (no-op'd because user not logged on)
> - Parsing of images from responseBuffer fails as it expects the responseBuffer to be the same as with the PHP app
> - Error message when trying to view user but not logged on due to state transition from addPerson to PersonDetail
> - EventDetailImages metric target is set to > 9 when there is only 1 image on each event page
> - Had to change Timing to manual in several ops because of the use of Apache HttpClient
> - FileLoader.java uses wrong names for thumbnails
> Most of the fixes involve using the Apache HttpClient instead of the Faban HttpTransport which allows better support for Session management and for the uploading of files within a User Session. Others come about because parts of the driver are still looking for the app to behave as per the PHP App. Particular in terms of what data the app will accept and what data it returns.
> I'm willing to break this out in to separate bugs, but it would be difficult to address them all separately so it's better to deliver the fixes as a single patch. There are also some additions required on the app side, as the Apache HttpClient insists on adding 'charset=' to the content_type field
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.