You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tapestry.apache.org by René Bernhardsgrütter <re...@gmail.com> on 2013/01/24 12:49:50 UTC
X-Sendfile / mod_xsendfile with Tapestry 5
Hi all,
in my current project I need to _send_ large files to authorized users.
The server mustn't execute/display the files, even if it's a index.html.
The environment will be an Apache2-frontent, via mod_jk to a Tomcat
instance where Tapestry runs.
After some googleing I believe the best way to solve my problem this is
with mod_xsendfile. I've never used this before and since there seems to
be no documentation about this in Tapestry, I'm not sure how to
implement it the right way.
What I have now:
The component part:
[...]
@Property
private FileReference fileReference; // from DB with paths, size, etc.
@Inject
private RequestGlobals requestGlobals;
private void onActionFromSend() throws IOException {
[... security checks ...]
requestGlobals.getHTTPServletResponse().setHeader("X-Sendfile",
fileReference.absolutePath());
requestGlobals.getHTTPServletResponse().flushBuffer();
}
The template part for this component:
<t:actionlink t:id="send"
target="_blank">${fileReference.fileName}</t:actionlink>
Unfortunately my Apache environment isn't set up yet so I couldn't try
it and don't know if it works. But the main question is: Is it OK to use
requestGlobals the way as it's done in the code above?
I'm not sure what this would mean for the request process flow
(https://tapestry.apache.org/request-processing.html). I think it would
be disrupted somewhere between ComponentRequestHandler and
ComponentRequestHandlerTerminator.
A confirmation/correction/hit would be great :-)
René
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Sat, 26 Jan 2013 11:20:56 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
> You are right, it doesn't seem to be a problem.
>
> So, it it the common way to serve user content files (uploaded images,
> etc.) via a StreamResponse in a page that reads the files from the
> harddrive?
You can also use events for that, but I think is only the best solution
when you only need to server files in one page. The page-based solution is
more reusable and yields better URLs.
> For example: /content/[fileid] returns an image that's embedded into the
> page. And in the template, it's just '<img src="/content/1234" />'.
You can do that. It'll work, but I think there are better ways of doing
that.
> Or how should that be done?
Right now, out of the box, Tapestry doesn't provide a component for the
<img> tag, but you could easily do that. Please file a JIRA about it.
Meanwhile, use the PageRenderLinkSource service to generate the urls and
pass them to the src attribute of <img>. Or, better yet, create a
component that encapsulates this logic.
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by René Bernhardsgrütter <re...@gmail.com>.
You are right, it doesn't seem to be a problem.
So, it it the common way to serve user content files (uploaded images,
etc.) via a StreamResponse in a page that reads the files from the
harddrive?
For example: /content/[fileid] returns an image that's embedded into the
page. And in the template, it's just '<img src="/content/1234" />'.
Or how should that be done?
On 25.01.2013 12:21, Thiago H de Paula Figueiredo wrote:
> On Thu, 24 Jan 2013 14:07:31 -0200, René Bernhardsgrütter
> <re...@gmail.com> wrote:
>
>>> I wouldn't say that. File transfers are affected by IO, not CPU.
>> I've read this somewhere several months ago and yesterday again here:
>> https://blogs.warwick.ac.uk/chrismay/entry/mod_x_sendfile/
>
> Have you seen the second comment in that post? It links to another
> post in the same blog saying this:
>
> "We have two in day-to-day production at the moment. One serves about
> 60GB/day of dynamic web pages and it’s CPU sits around 10% – 15% all
> day long. To be honest, a box with half as many CPUs would do the job
> just fine – but it’s nice to know that should we get a sudden increase
> in load on the app, we have more than enough headroom!"
>
> The site serves tens of gigabytes per day with less than 15% CPU
> usage, so I still think file transfers are basically IO-bound, not
> CPU-bound.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Thu, 24 Jan 2013 14:07:31 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
>> I wouldn't say that. File transfers are affected by IO, not CPU.
> I've read this somewhere several months ago and yesterday again here:
> https://blogs.warwick.ac.uk/chrismay/entry/mod_x_sendfile/
Have you seen the second comment in that post? It links to another post in
the same blog saying this:
"We have two in day-to-day production at the moment. One serves about
60GB/day of dynamic web pages and it’s CPU sits around 10% – 15% all day
long. To be honest, a box with half as many CPUs would do the job just
fine – but it’s nice to know that should we get a sudden increase in load
on the app, we have more than enough headroom!"
The site serves tens of gigabytes per day with less than 15% CPU usage, so
I still think file transfers are basically IO-bound, not CPU-bound.
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Sat, 26 Jan 2013 11:28:29 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
> You are -- of course -- right :-D
Unless I'm wrong, and that happens a lot. :P
> Before the url rewriting I wanted to do the separate page, but something
> made it impossible. It had something to do with the currently loaded
> page...
> During a little refactoring of my FileReference I solved the problem by
> the way. Now the page solution is just perfect. Man, 5 mins and it was
> done...
That's a nice suprise we usually get when implementing something in
Tapestry: it ends up being simpler than we expected. :)
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by René Bernhardsgrütter <re...@gmail.com>.
You are -- of course -- right :-D
Before the url rewriting I wanted to do the separate page, but something
made it impossible. It had something to do with the currently loaded page...
During a little refactoring of my FileReference I solved the problem by
the way. Now the page solution is just perfect. Man, 5 mins and it was
done...
On 25.01.2013 12:15, Thiago H de Paula Figueiredo wrote:
> On Thu, 24 Jan 2013 14:07:31 -0200, René Bernhardsgrütter
> <re...@gmail.com> wrote:
>
>> Hi,
>
> Hi!
>
>>
>>> I wouldn't say that. File transfers are affected by IO, not CPU.
>> I've read this somewhere several months ago and yesterday again here:
>> https://blogs.warwick.ac.uk/chrismay/entry/mod_x_sendfile/
>
>>> I'd like to see how you implemented it.
>> Basically like the HowTo
>> [https://wiki.apache.org/tapestry/Tapestry5HowToStreamAnExistingBinaryFile]
>>
>> but I shortened it a little bit:
>
> Quite simple, no? :)
>
>> The only ugly thing for users who want to copy the direct file link was
>> the event request:
>> [server]/index.ajaxuploadarea.download/61/nEWEaMExZ5xbJzSmNQQe?t:ac=5tAnW,
>>
>> where 61 is the fileReferenceId and nEW.. the folderName.
>> I've now also integrated url rewriting (as
>> http://blog.tapestry5.de/index.php/2010/09/06/new-url-rewriting-api/).
>
> Event URLs were never meant to be seen by end users. You can have a
> way better URL without URL rewriting by creating a page just for
> returning the StreamResponse on its onActivate() method. You'll use
> PageLink instead of ActionLink or EventLink. Just put all the info
> you'll need in the context parameter.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Thu, 24 Jan 2013 14:07:31 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
> Hi,
Hi!
>
>> I wouldn't say that. File transfers are affected by IO, not CPU.
> I've read this somewhere several months ago and yesterday again here:
> https://blogs.warwick.ac.uk/chrismay/entry/mod_x_sendfile/
>> I'd like to see how you implemented it.
> Basically like the HowTo
> [https://wiki.apache.org/tapestry/Tapestry5HowToStreamAnExistingBinaryFile]
> but I shortened it a little bit:
Quite simple, no? :)
> The only ugly thing for users who want to copy the direct file link was
> the event request:
> [server]/index.ajaxuploadarea.download/61/nEWEaMExZ5xbJzSmNQQe?t:ac=5tAnW,
> where 61 is the fileReferenceId and nEW.. the folderName.
> I've now also integrated url rewriting (as
> http://blog.tapestry5.de/index.php/2010/09/06/new-url-rewriting-api/).
Event URLs were never meant to be seen by end users. You can have a way
better URL without URL rewriting by creating a page just for returning the
StreamResponse on its onActivate() method. You'll use PageLink instead of
ActionLink or EventLink. Just put all the info you'll need in the context
parameter.
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by René Bernhardsgrütter <re...@gmail.com>.
Hi,
> I wouldn't say that. File transfers are affected by IO, not CPU.
I've read this somewhere several months ago and yesterday again here:
https://blogs.warwick.ac.uk/chrismay/entry/mod_x_sendfile/
> I'd like to see how you implemented it.
Basically like the HowTo
[https://wiki.apache.org/tapestry/Tapestry5HowToStreamAnExistingBinaryFile]
but I shortened it a little bit:
AttachmentStreamResponse.java:
public class AttachmentStreamResponse implements StreamResponse {
private InputStream inputStream = null;
private String contentType = "text/plain";
private FileReference fileReference;
public AttachmentStreamResponse(FileReference fileReference)
throws FileNotFoundException {
this.fileReference = fileReference;
this.inputStream = new BufferedInputStream(new
FileInputStream(fileReference.getAbsolutePath()));
}
[getter for contentType and inputStream]
@Override
public void prepareResponse(Response response) {
response.setHeader("Content-Disposition", "attachment;
filename=\"" + fileReference.getFileName() + "\"");
response.setContentLength((int) fileReference.getSizeInBytes());
response.setHeader("Expires", "0");
response.setHeader("Cache-Control", "must-revalidate,
post-check=0, pre-check=0");
response.setHeader("Pragma", "public");
}
}
And in the Component, where the Actionlink points to:
private StreamResponse onActionFromDownload(long fileReferenceId,
String folderName) throws IOException {
FileReference fileToSend = new FileReference(fileReferenceId);
if (folderName != null
&& [... is file existing and allowed to download? ...]) {
return new AttachmentStreamResponse(fileToSend);
}
return null;
}
The only ugly thing for users who want to copy the direct file link was
the event request:
[server]/index.ajaxuploadarea.download/61/nEWEaMExZ5xbJzSmNQQe?t:ac=5tAnW,
where 61 is the fileReferenceId and nEW.. the folderName.
I've now also integrated url rewriting (as
http://blog.tapestry5.de/index.php/2010/09/06/new-url-rewriting-api/).
This took the most time :-D
public class FileDownloadLinkTransformer implements
ComponentEventLinkTransformer {
private enum ManuallyTransformedRequest {
FILE_DOWNLOAD, NOT_MANUALLY_TRANSFORMED;
}
private final String FILE_DOWNLOAD_COMPONENT_PATH = "/" +
Index.class.getSimpleName().toLowerCase() + ".ajaxuploadarea.download";
private final String FILE_DOWNLOAD_COMPONENT_PATH_REGEX = "/" +
Index.class.getSimpleName().toLowerCase() +
"\\.ajaxuploadarea\\.download.*";
private final String FILE_DOWNLOAD_COMPONENT_PATH_TRANSFORMATION =
"/download";
private final String ACTIVATION_CONTEXT_PARAMETER_NAME = "t:ac";
@Inject
private TypeCoercer typeCoercer;
@Override
public Link transformComponentEventLink(Link defaultLink,
ComponentEventRequestParameters parameters) {
switch (getTypeOfPage(defaultLink)) {
case FILE_DOWNLOAD:
String context = exctractFileIdAndFolderName(defaultLink);
if (context != null) {
defaultLink.removeParameter(ACTIVATION_CONTEXT_PARAMETER_NAME);
return
defaultLink.copyWithBasePath(getDevelopeModeExtension() +
FILE_DOWNLOAD_COMPONENT_PATH_TRANSFORMATION + context);
}
break;
case NOT_MANUALLY_TRANSFORMED:
break;
}
return defaultLink;
}
@Override
public ComponentEventRequestParameters
decodeComponentEventRequest(Request request) {
String requestedPath = request.getPath();
if
(requestedPath.startsWith(FILE_DOWNLOAD_COMPONENT_PATH_TRANSFORMATION)) {
return new ComponentEventRequestParameters(
Index.class.getSimpleName(),
Index.class.getSimpleName(),
FILE_DOWNLOAD_COMPONENT_PATH.replaceFirst("/" +
Index.class.getSimpleName().toLowerCase() + "\\.", ""),
EventConstants.ACTION,
new EmptyEventContext(),
toEventContext(requestedPath.substring(FILE_DOWNLOAD_COMPONENT_PATH_TRANSFORMATION.length()
+ 1)));
}
return null;
}
private ManuallyTransformedRequest getTypeOfPage(Link link) {
if (link.getBasePath().matches(getDevelopeModeExtension() +
FILE_DOWNLOAD_COMPONENT_PATH_REGEX)) {
return ManuallyTransformedRequest.FILE_DOWNLOAD;
}
return ManuallyTransformedRequest.NOT_MANUALLY_TRANSFORMED;
}
private String exctractFileIdAndFolderName(Link link) {
if (link.getBasePath().matches(getDevelopeModeExtension() +
FILE_DOWNLOAD_COMPONENT_PATH + "/[0-9]+/[0-9a-zA-Z].+")) {
return
link.getBasePath().substring((getDevelopeModeExtension() +
FILE_DOWNLOAD_COMPONENT_PATH).length());
}
return null;
}
private EventContext toEventContext(String value) {
return value == null ? new EmptyEventContext() : new
ArrayEventContext(typeCoercer, (Object[]) value.split("/"));
}
private String getDevelopeModeExtension() {
return PlainTraySetting.PRODUCTION_MODE ? "" : "/webapp";
}
}
I know, it got a bit ugly, but at least it works and the most parts are
strongly typed.
René
On 24.01.2013 15:29, Thiago H de Paula Figueiredo wrote:
> On Thu, 24 Jan 2013 11:49:05 -0200, René Bernhardsgrütter
> <re...@gmail.com> wrote:
>
>> Hi Thiago,
>
> Hi!
>
>> thank you for your answer!
>
> ;)
>
>> I thought that Java/Tomcat wouldn't be optimal to handle large/many file
>> transfers in a web environment (too high cpu usage) and that Apache
>> would handle it better -- correct me, if I'm wrong.
>
> I wouldn't say that. File transfers are affected by IO, not CPU.
>
>> I've now implemented the StreamResponse solution you suggested and it
>> works fine.
>
> I'd like to see how you implemented it.
>
>> Anyway, I'll see if the performance will be a problem.
>
> That's exactly how you should do: instead of just guessing how it'll
> perform, implement and test. :)
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Thu, 24 Jan 2013 11:49:05 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
> Hi Thiago,
Hi!
> thank you for your answer!
;)
> I thought that Java/Tomcat wouldn't be optimal to handle large/many file
> transfers in a web environment (too high cpu usage) and that Apache
> would handle it better -- correct me, if I'm wrong.
I wouldn't say that. File transfers are affected by IO, not CPU.
> I've now implemented the StreamResponse solution you suggested and it
> works fine.
I'd like to see how you implemented it.
> Anyway, I'll see if the performance will be a problem.
That's exactly how you should do: instead of just guessing how it'll
perform, implement and test. :)
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by René Bernhardsgrütter <re...@gmail.com>.
Hi Thiago,
thank you for your answer!
I thought that Java/Tomcat wouldn't be optimal to handle large/many file
transfers in a web environment (too high cpu usage) and that Apache
would handle it better -- correct me, if I'm wrong.
I've now implemented the StreamResponse solution you suggested and it
works fine.
Anyway, I'll see if the performance will be a problem.
For the moment, it works.
Thanks again for your support!
René
On 24.01.2013 13:00, Thiago H de Paula Figueiredo wrote:
> On Thu, 24 Jan 2013 09:49:50 -0200, René Bernhardsgrütter
> <re...@gmail.com> wrote:
>
>> Hi all,
>
> Hi!
>
>> in my current project I need to _send_ large files to authorized users.
>> The server mustn't execute/display the files, even if it's a index.html.
>> The environment will be an Apache2-frontent, via mod_jk to a Tomcat
>> instance where Tapestry runs.
>
> Before trying some non-Tapestry solution, why don't you try having
> Tapestry serving these large files for you using a page specific for
> that (so you can have your authorization logic there) that returns a
> StreamResponse wrapping the files? This way, doing a good
> StreamResponde implementation, must probably using a
> BufferedInputStream, you can easily avoid loading the whole file in
> memory for serving it (actually, you can even control the size of the
> buffer.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: X-Sendfile / mod_xsendfile with Tapestry 5
Posted by Thiago H de Paula Figueiredo <th...@gmail.com>.
On Thu, 24 Jan 2013 09:49:50 -0200, René Bernhardsgrütter
<re...@gmail.com> wrote:
> Hi all,
Hi!
> in my current project I need to _send_ large files to authorized users.
> The server mustn't execute/display the files, even if it's a index.html.
> The environment will be an Apache2-frontent, via mod_jk to a Tomcat
> instance where Tapestry runs.
Before trying some non-Tapestry solution, why don't you try having
Tapestry serving these large files for you using a page specific for that
(so you can have your authorization logic there) that returns a
StreamResponse wrapping the files? This way, doing a good StreamResponde
implementation, must probably using a BufferedInputStream, you can easily
avoid loading the whole file in memory for serving it (actually, you can
even control the size of the buffer.
--
Thiago H. de Paula Figueiredo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org