You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-commits@lucene.apache.org by ry...@apache.org on 2007/05/05 19:14:23 UTC

svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Author: ryan
Date: Sat May  5 10:14:22 2007
New Revision: 535548

URL: http://svn.apache.org/viewvc?view=rev&rev=535548
Log:
fixing a request parsing bug.  When posting "application/x-www-form-urlencoded" and including a charset, application/x-www-form-urlencoded; charset=utf-8, the parser would treat the body as raw content.

This includes tests to repeat the error.

It also adds easymock.jar to lib, and excludes it from distribution.

Added:
    lucene/solr/trunk/lib/easymock.jar   (with props)
Modified:
    lucene/solr/trunk/build.xml
    lucene/solr/trunk/src/test/org/apache/solr/servlet/SolrRequestParserTest.java
    lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Modified: lucene/solr/trunk/build.xml
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/build.xml?view=diff&rev=535548&r1=535547&r2=535548
==============================================================================
--- lucene/solr/trunk/build.xml (original)
+++ lucene/solr/trunk/build.xml Sat May  5 10:14:22 2007
@@ -380,6 +380,7 @@
          manifest="${dest}/META-INF/MANIFEST.MF">
        <lib dir="${lib}">
          <exclude name="servlet-api*.jar" />
+         <exclude name="easymock.jar" />
        </lib>
        <lib dir="${dist}">
          <include name="${fullnamever}.jar" />
@@ -478,4 +479,5 @@
   </target>
 
 </project>
+
 

Added: lucene/solr/trunk/lib/easymock.jar
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/lib/easymock.jar?view=auto&rev=535548
==============================================================================
Binary file - no diff available.

Propchange: lucene/solr/trunk/lib/easymock.jar
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Modified: lucene/solr/trunk/src/test/org/apache/solr/servlet/SolrRequestParserTest.java
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/src/test/org/apache/solr/servlet/SolrRequestParserTest.java?view=diff&rev=535548&r1=535547&r2=535548
==============================================================================
--- lucene/solr/trunk/src/test/org/apache/solr/servlet/SolrRequestParserTest.java (original)
+++ lucene/solr/trunk/src/test/org/apache/solr/servlet/SolrRequestParserTest.java Sat May  5 10:14:22 2007
@@ -17,6 +17,10 @@
 
 package org.apache.solr.servlet;
 
+import static org.easymock.EasyMock.createMock;
+import static org.easymock.EasyMock.expect;
+import static org.easymock.EasyMock.replay;
+
 import java.net.URL;
 import java.util.ArrayList;
 import java.util.Collections;
@@ -24,6 +28,8 @@
 import java.util.List;
 import java.util.Map;
 
+import javax.servlet.http.HttpServletRequest;
+
 import org.apache.commons.io.IOUtils;
 import org.apache.solr.core.SolrConfig;
 import org.apache.solr.core.SolrCore;
@@ -125,6 +131,37 @@
     for( String[] tst : teststr ) {
       MultiMapSolrParams params = SolrRequestParsers.parseQueryString( "val="+tst[1] );
       assertEquals( tst[0], params.get( "val" ) );
+    }
+  }
+  
+  public void testStandardParseParamsAndFillStreams() throws Exception
+  {
+    ArrayList<ContentStream> streams = new ArrayList<ContentStream>();
+    Map<String,String[]> params = new HashMap<String, String[]>();
+    params.put( "q", new String[] { "hello" } );
+    
+    // Set up the expected behavior
+    String[] ct = new String[] {
+        "application/x-www-form-urlencoded",
+        "Application/x-www-form-urlencoded",
+        "application/x-www-form-urlencoded; charset=utf-8",
+        "application/x-www-form-urlencoded;"
+    };
+    
+    for( String contentType : ct ) {
+      HttpServletRequest request = createMock(HttpServletRequest.class);
+      expect(request.getMethod()).andReturn("POST").anyTimes();
+      expect(request.getContentType()).andReturn( contentType ).anyTimes();
+      expect(request.getParameterMap()).andReturn(params).anyTimes();
+      replay(request);
+      
+      MultipartRequestParser multipart = new MultipartRequestParser( 1000000 );
+      RawRequestParser raw = new RawRequestParser();
+      StandardRequestParser standard = new StandardRequestParser( multipart, raw );
+      
+      SolrParams p = standard.parseParamsAndFillStreams( request, streams );
+      
+      assertEquals( "contentType: "+contentType, "hello", p.get("q") );
     }
   }
 }

Modified: lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java?view=diff&rev=535548&r1=535547&r2=535548
==============================================================================
--- lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java (original)
+++ lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java Sat May  5 10:14:22 2007
@@ -367,6 +367,10 @@
     if( "POST".equals( method ) ) {
       String contentType = req.getContentType();
       if( contentType != null ) {
+        int idx = contentType.indexOf( ';' );
+        if( idx > 0 ) { // remove the charset definition "; charset=utf-8"
+          contentType = contentType.substring( 0, idx );
+        }
         if( "application/x-www-form-urlencoded".equals( contentType.toLowerCase() ) ) {
           return new ServletSolrParams(req); // just get the params from parameterMap
         }
@@ -379,6 +383,7 @@
     throw new SolrException( 400, "Unsuported method: "+method );
   }
 }
+
 
 
 



Re: svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Posted by Chris Hostetter <ho...@fucit.org>.
: I think we need something...  without some mock framework, testing
: HttpServlets is difficult.  This accounts for a serious lack of tests in
: that area.  I have avoided writing some test (like this one) because
: there is too much overhead.

: got it.  I was trying to follow the "fix bugs with a test" guideline was
: quick to commit since this is in the testing side, but i see your point.

i don't disagree with your choice, i was just pointing out for the
purposes of discussion that this is an area where if i were in your shoes
i would have fixed the bug without a test, and then started a seperate
issue/discussion about hte need for better servlet tests and using a mock
tool to do so ... but then i'm a slow moving worry wart, and you're a more
get up and go type person :)



-Hoss


Re: svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On May 6, 2007, at 2:13 AM, Ryan McKinley wrote:
>> As for the "do we want it" questions, i'm not really familiar with
>> easymock or jMock, so i don't have an opinion ...
>
> I think we need something...  without some mock framework, testing  
> HttpServlets is difficult.


+1

Testing through Java container API's is hard.  Back in the day* we  
used things like Cactus, and StrutsMockTestCase, and HttpUnit,  
CanooWebTest and such.  IMO, though, simple mocks are all that's  
needed.  In RubyLand, we mock Solr itself for unit tests, but have a  
suite of functional tests that work against a live server too.


> It is unclear if the jmock license is compatible:
> http://www.jmock.org/license.html

I think so.  Looks like Struts uses it.  I'd follow their lead and  
see how they note it.

	Erik


Re: svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Posted by Ryan McKinley <ry...@gmail.com>.
Chris Hostetter wrote:
> : I included the jar in lib, but exclude it from distribution, is this ok?
> 
> excluding it from the war isn't the same as excluding it from distribution
> (it looks like it would still be included in both the source and binary
> tgz/zip releases).
> 

oh right...


> assuming this is what you mean by easymock...
> 
> 	http://www.easymock.org/
> 	http://www.easymock.org/License.html
> 
> ...then before any real questions about wether or not it's useful and do
> we want to use it can be answered we have to figure out wether we re
> allowed to use it -- it's the MIT License, which seems to (currently)
> be acceptible for 3rd party libraries included in apache releases...
> 
>    http://people.apache.org/~cliffs/3party.html
> 
> ... but it would definitely needs to be mentioned in the NOTICE.txt
> 

I added a line in rev 535581.  We can remove it (with the .jar and 
tests) if that is a better choice.


> 
> As for the "do we want it" questions, i'm not really familiar with
> easymock or jMock, so i don't have an opinion ... 

I think we need something...  without some mock framework, testing 
HttpServlets is difficult.  This accounts for a serious lack of tests in 
that area.  I have avoided writing some test (like this one) because 
there is too much overhead.

It is unclear if the jmock license is compatible:
http://www.jmock.org/license.html


> 
> 
> For the record, i think adding a new 3rd party jar is a weighty enough
> change that it should have warranted opening a Jirra issue and asking for
> comments prior to commiting.  I also think that the bug which motivatied
> this test probably should have been filed in jira, so that if anyone else
> ever asked about it, we would have refrence URL to cite when discussing
> the bug, the fix, and any potential complications (or future bugs) that
> may arrise from it.
> 

got it.  I was trying to follow the "fix bugs with a test" guideline was 
quick to commit since this is in the testing side, but i see your point.

Is it still worth opening an issue?

ryan



Re: svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Posted by Chris Hostetter <ho...@fucit.org>.
: I included the jar in lib, but exclude it from distribution, is this ok?

excluding it from the war isn't the same as excluding it from distribution
(it looks like it would still be included in both the source and binary
tgz/zip releases).

assuming this is what you mean by easymock...

	http://www.easymock.org/
	http://www.easymock.org/License.html

...then before any real questions about wether or not it's useful and do
we want to use it can be answered we have to figure out wether we re
allowed to use it -- it's the MIT License, which seems to (currently)
be acceptible for 3rd party libraries included in apache releases...

   http://people.apache.org/~cliffs/3party.html

... but it would definitely needs to be mentioned in the NOTICE.txt


As for the "do we want it" questions, i'm not really familiar with
easymock or jMock, so i don't have an opinion ... and i don't really have
the energy to investigate further at the moment.  (this cold is kicking my
ass, and checking mail in fits and spurts is both helping keep me sane,
and making my sinus headache worse ... so i'm dmned if i do damned if i
don't)


For the record, i think adding a new 3rd party jar is a weighty enough
change that it should have warranted opening a Jirra issue and asking for
comments prior to commiting.  I also think that the bug which motivatied
this test probably should have been filed in jira, so that if anyone else
ever asked about it, we would have refrence URL to cite when discussing
the bug, the fix, and any potential complications (or future bugs) that
may arrise from it.


-Hoss


Re: svn commit: r535548 - in /lucene/solr/trunk: build.xml lib/easymock.jar src/test/org/apache/solr/servlet/SolrRequestParserTest.java src/webapp/src/org/apache/solr/servlet/SolrRequestParsers.java

Posted by Ryan McKinley <ry...@gmail.com>.
> 
> Added:
>     lucene/solr/trunk/lib/easymock.jar   (with props)

Are we ok with this?

I like easymock because unlike jmock it does not require a special base 
test class, and I don't need to type in the function prototypes as strings.

I included the jar in lib, but exclude it from distribution, is this ok?

ryan