You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@oodt.apache.org by "Verma, Rishi (317I)" <Ri...@jpl.nasa.gov> on 2010/12/09 19:38:24 UTC

Webgrid OFSNGetHandler customization

Hello everyone -

I've created a webgrid query handler (implements OFSNGetHandler) which returns some processed data when invoked from an HTTP request. The data returned takes the form of a file, with name and extension identical to the original (unprocessed) input file my query handler receives as input.

I wanted to ask two questions regarding an OFSNGetHandler:
[1] Is it possible to customize the name and type extension of byte[] file data returned by an OFSNGetHandler? If so, where could this be done in the scope of OFSNGetHandler methods or related config files?
[2] Is it possible to ensure the OFSNGetHandler "retrieveChunk"method is only invoked once (ie. the offset is set at zero and the length is the total byte size of the file)? The processing my query handler performs is time intensive and dependent on reading the entire file at once - not in chunks. Therefore, I want to ensure the processing happens only once.

Would appreciate any advice, thanks!
Rishi

Re: Webgrid OFSNGetHandler customization

Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.

Hey Rishi,

> I think my question is whether there exists some specification allowing for processing of an entire product to occur before bytes of a product are returned. The processing should occur before the product is returned via "retrieveChunk" and only occur once.  The only way I can think of enabling this kind of feature to have special code within "retreiveChunk" which 'processes' the product only during the first invocation of 'retrieveChunk', and subsequently returns the processed bytes in any following invocations. Do you have any recommendations on a better technique or does the webgrid infrastructure support for something like this?

Yeah, actually one thing you *can* assume is that there will always be a first call to query(XMLQuery q), even in the case of the LargeProductQueryHandler. So, one thing that I've done in the past is to realize what the user is asking for on query(XMLQuery q), then cook up some id to shove into the LargeResult returned as part of XMLQuery.getResults().add(LargeResult...). That id can then be used (and is actually used later) when retrieveChunk is called. 

So, then you'll know at the query(XMLQuery q) time what the user is going to next ask for in a series of subsequent calls to retrieveChunk, so in the initial query(XMLQuery q) method, you may try doing the initial (and 1-time only) processing required for the query *first*, then cache the result or something that you make available in retrieveChunk next. Of course, if you cache it you'll have to clean it up after the last retrieveChunk call, but the good news is, you've got one more method call in there to take care of that (the close(id) method that's part of the interface). You can take care of the release of resources in that method and you're good!

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Re: Webgrid OFSNGetHandler customization

Posted by "Verma, Rishi (317I)" <Ri...@jpl.nasa.gov>.

Hi Chris,

Thanks for your responses!

Wanted to ask a follow-up on your second point:

> http://svn.apache.org/repos/asf/oodt/trunk/product/src/main/java/org/apache/oodt/product/LargeProductQueryHandler.java
> 
> Basically the way it works is that you can't assume that the retrieveChunk method is only called 1x. It will be called as many times as needed i order to retrieve the bytes of the product.

I think my question is whether there exists some specification allowing for processing of an entire product to occur before bytes of a product are returned. The processing should occur before the product is returned via "retrieveChunk" and only occur once.  The only way I can think of enabling this kind of feature to have special code within "retreiveChunk" which 'processes' the product only during the first invocation of 'retrieveChunk', and subsequently returns the processed bytes in any following invocations. Do you have any recommendations on a better technique or does the webgrid infrastructure support for something like this?

Thanks so much!
Rishi

On Dec 9, 2010, at 9:06 PM, Mattmann, Chris A (388J) wrote:

> Hi Rishi,
> 
> Thanks for your question! My comments below:
> 
>> I wanted to ask two questions regarding an OFSNGetHandler:
>> [1] Is it possible to customize the name and type extension of byte[] file data returned by an OFSNGetHandler? If so, where could this be done in the scope of OFSNGetHandler methods or related config files?
> 
> Yep. One way to do this would be customize in Tomcat the URL path (and filter) to the web-grid container serving up the OFSN product handler. Right now if you look at web-grid, I think it assumes a default filename (that it sets via the response header in the javax.servlet.HttpServletResponse class I believe).
> 
> One way to customize this might be to extend Web-Grid's product service here:
> 
> http://svn.apache.org/repos/asf/oodt/trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java
> 
> and override the characterize method, which suggests the filename, and allow the ability of the underlying handler to help set the name. Then in turn you could have your handler call this callback.
> 
>> [2] Is it possible to ensure the OFSNGetHandler "retrieveChunk"method is only invoked once (ie. the offset is set at zero and the length is the total byte size of the file)? The processing my query handler performs is time intensive and dependent on reading the entire file at once - not in chunks. Therefore, I want to ensure the processing happens only once.
> 
> Check out the LargeProductQueryHandler interface, here:
> 
> http://svn.apache.org/repos/asf/oodt/trunk/product/src/main/java/org/apache/oodt/product/LargeProductQueryHandler.java
> 
> Basically the way it works is that you can't assume that the retrieveChunk method is only called 1x. It will be called as many times as needed i order to retrieve the bytes of the product.
> 
> HTH!
> 
> Cheers,
> Chris
> 
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Re: Webgrid OFSNGetHandler customization

Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.

Hi Rishi,

Thanks for your question! My comments below:

> I wanted to ask two questions regarding an OFSNGetHandler:
> [1] Is it possible to customize the name and type extension of byte[] file data returned by an OFSNGetHandler? If so, where could this be done in the scope of OFSNGetHandler methods or related config files?

Yep. One way to do this would be customize in Tomcat the URL path (and filter) to the web-grid container serving up the OFSN product handler. Right now if you look at web-grid, I think it assumes a default filename (that it sets via the response header in the javax.servlet.HttpServletResponse class I believe).

One way to customize this might be to extend Web-Grid's product service here:

http://svn.apache.org/repos/asf/oodt/trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java

and override the characterize method, which suggests the filename, and allow the ability of the underlying handler to help set the name. Then in turn you could have your handler call this callback.

> [2] Is it possible to ensure the OFSNGetHandler "retrieveChunk"method is only invoked once (ie. the offset is set at zero and the length is the total byte size of the file)? The processing my query handler performs is time intensive and dependent on reading the entire file at once - not in chunks. Therefore, I want to ensure the processing happens only once.

Check out the LargeProductQueryHandler interface, here:

http://svn.apache.org/repos/asf/oodt/trunk/product/src/main/java/org/apache/oodt/product/LargeProductQueryHandler.java

Basically the way it works is that you can't assume that the retrieveChunk method is only called 1x. It will be called as many times as needed i order to retrieve the bytes of the product.

HTH!

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++