Blocking during/after upload

[Dan <da...@tipjarawards.com>]

We would like to perform some file processing once a file has been uploaded
but it seems that if we use an Ftplet and implement onUploadEnd() then that
won't block (i.e. the client will think the file has been successfully
uploaded)

We need something like:

1. Client uploads a file
2. File upload complete, processing starts
3. Processing finished, OK result sent back to client

One problem is that the processing in #2 can take some time (minutes even)
and FTP clients assume a failure has happened since no reply has been
received.

Any thoughts on how you can block, do your processing, and then commit a
response back to the client?

Thanks,

Dan

RE: Blocking during/after upload

[DevNull43 <de...@gmail.com>]

On Wed, 2010-03-03 at 17:32 -0500, Dan wrote:
> That isn't exactly what we need since our clients aren't expecting any delay
> and won't know to look for a .ok file (or other file trick). Also, right
> after the upload they might want to download but of course the processing is
> still running.
> 
> We tried creating our own OutputStream that overrides the close() method to
> do the processing but that triggers the client to upload again as it
> timeouts waiting for a response
> 
> On the surface it doesn't look like you can block effectively after an
> upload since the client will timeout waiting in any case and then retry.
> 
> Dan
> 

You only have 2 choices, sync or async processing.

In sync processing you should start a 226 Reply, then do your procesing
and end the 226 reply, for example:

200 Command PORT okay.
150 File status okay; about to open data connection.
226- Starting processing for file filename
... time consumes here
Processing ok/Failed
226 Transfer complete.
5866 bytes received in 0.02 secs (365.6 kB/s)

If you choose for async, you delegate to a thread who does the
processing, and delete file if result was bad, but you loose control
with a default FTP client.

I haven't try yet but I will need it in a near future, and probably will
choose both options depending on filesize, so if you find a solution let
me know ;)

Thanks.

RE: Blocking during/after upload

[Dan <da...@tipjarawards.com>]

That isn't exactly what we need since our clients aren't expecting any delay
and won't know to look for a .ok file (or other file trick). Also, right
after the upload they might want to download but of course the processing is
still running.

We tried creating our own OutputStream that overrides the close() method to
do the processing but that triggers the client to upload again as it
timeouts waiting for a response

On the surface it doesn't look like you can block effectively after an
upload since the client will timeout waiting in any case and then retry.

Dan

-----Original Message-----
From: Vinicius Carvalho [mailto:vinicius.carvalho@sambatech.com.br] 
Sent: March-03-10 5:27 PM
To: ftpserver-users@mina.apache.org
Subject: Re: Blocking during/after upload

We have the same scenario here. What we do is delegate to a ExecutorService
the execution (pardon me) of the given task. Since it's an asynchronous job,
you can reply to the client and process it on a separate thread. The client
then gets the response from the ftp, and since the execution is still
running, we later add a file with a .ok extension to inform the client that
the process went right

Regards

On Wed, Mar 3, 2010 at 7:12 PM, Dan <da...@tipjarawards.com> wrote:

> We would like to perform some file processing once a file has been 
> uploaded but it seems that if we use an Ftplet and implement 
> onUploadEnd() then that won't block (i.e. the client will think the 
> file has been successfully
> uploaded)
>
> We need something like:
>
> 1. Client uploads a file
> 2. File upload complete, processing starts 3. Processing finished, OK 
> result sent back to client
>
> One problem is that the processing in #2 can take some time (minutes 
> even) and FTP clients assume a failure has happened since no reply has 
> been received.
>
> Any thoughts on how you can block, do your processing, and then commit 
> a response back to the client?
>
> Thanks,
>
> Dan
>
>

Re: Blocking during/after upload

[Vinicius Carvalho <vi...@sambatech.com.br>]

We have the same scenario here. What we do is delegate to a ExecutorService
the execution (pardon me) of the given task. Since it's an asynchronous job,
you can reply to the client and process it on a separate thread. The client
then gets the response from the ftp, and since the execution is still
running, we later add a file with a .ok extension to inform the client that
the process went right

Regards

On Wed, Mar 3, 2010 at 7:12 PM, Dan <da...@tipjarawards.com> wrote:

> We would like to perform some file processing once a file has been uploaded
> but it seems that if we use an Ftplet and implement onUploadEnd() then that
> won't block (i.e. the client will think the file has been successfully
> uploaded)
>
> We need something like:
>
> 1. Client uploads a file
> 2. File upload complete, processing starts
> 3. Processing finished, OK result sent back to client
>
> One problem is that the processing in #2 can take some time (minutes even)
> and FTP clients assume a failure has happened since no reply has been
> received.
>
> Any thoughts on how you can block, do your processing, and then commit a
> response back to the client?
>
> Thanks,
>
> Dan
>
>