You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@commons.apache.org by Kirill Safonov <ki...@gmail.com> on 2010/05/12 17:07:19 UTC

potential problem: FtpFileObject caches children when not attached

Hello,

 

While digging a bit into FTP performance, I've noticed the following:

 

When FtpFileObject.refresh() is called, FtpFileObject.children field is
reset to null in FtpFileObject.doDetach(), and AbstractFileObject.attached
flag is set to false (this is OK)

But, if then FtpFileObject.getChildFile() is called (this may happen when
getType() is invoked for one of the child files - see below), current
implementation will ignore 'attached' state and silently fill children cache
by calling doGetChildren().

 

The result is that FtpFileObject remains in inconsistent state: it is not
attached with AbstractFileObject.type field = null, child names cache
(AbstractFileObject.children)  = null, but has children cache bulit.

 

I'm not currently sure which consequences it may cause.

 

What I've found up to date is that when I load list of children I see that
two FTP LIST commands are issued: 

-          first one is to list the children themselves

-          then parent file is refreshed since CacheStrategy is ON_RESOLVE
(this is actually another issue I'd like to discuss, since it gives a
performance drop clearing the children cache right after is was build)

-          and then another LIST call that comes from child.getType(). If
there was no inconsistency I described above, this LIST would have been
called for each of the children.

 

Hope all this makes sense,

 Regards,

 Kirill

RE: potential problem: FtpFileObject caches children when not attached

Posted by Kirill Safonov <ki...@gmail.com>.

Hi Mario,

 

File structure on server:

- Root

     - Folder 'parent'

            - file 'child1.txt'

            - file 'child2.txt'

 

Code:

 

            FileObject parent =
VFS.getManager().resolveFile("ftp://localhost:9789/parent", fops);

            FileObject[] children = parent.getChildren();

            assert children.length > 0;

            children[0].getType();

 

 

Complete FTP log:

 

220 Service ready for new user.

> USER test

331 User name okay, need password for test.

> PASS ******

230 User logged in, proceed.

> TYPE I

200 Command TYPE okay.

> CWD /

250 Directory changed to /

> SYST

215 UNIX Type: Apache FtpServer

> PORT 127,0,0,1,211,6

200 Command PORT okay.

> LIST

150 File status okay; about to open data connection.

226 Closing data connection.

> PORT 127,0,0,1,211,8

200 Command PORT okay.

> LIST parent

150 File status okay; about to open data connection.

226 Closing data connection.

> PORT 127,0,0,1,211,10

200 Command PORT okay.

> LIST parent

150 File status okay; about to open data connection.

226 Closing data connection.

 

So I see two 'LIST' commands issued for 'parent' folder.

 

Do you get the same behavior?

 

Thanks,

 Kirill

 

From: Mario Ivankovits [mailto:mario@ops.co.at] 
Sent: Friday, May 14, 2010 2:52 PM
To: 'Kirill Safonov'; 'Commons Users List'
Subject: AW: potential problem: FtpFileObject caches children when not
attached

 

Hi!

 

Hmmm . I tried to reproduce it like this:

 

                               FileObject fos =
VFS.getManager().resolveFile("ftp://XYZ.com/public/ test with space", fops);

                               fos.getType();

                               fos.getChildren();

 

                               fos.refresh();

                               fos.getType();

                               fos.getChildren();

 

After the refresh() I just get one ftp list . can you please outline with
code how to reproduce the problem.

Thanks!

 

Ciao,

Mario

 

 

Von: Kirill Safonov [mailto:kirill.safonov@gmail.com] 
Gesendet: Mittwoch, 12. Mai 2010 17:07
An: 'Commons Users List'
Cc: Mario Ivankovits
Betreff: potential problem: FtpFileObject caches children when not attached

 

Hello,

 

While digging a bit into FTP performance, I've noticed the following:

 

When FtpFileObject.refresh() is called, FtpFileObject.children field is
reset to null in FtpFileObject.doDetach(), and AbstractFileObject.attached
flag is set to false (this is OK)

But, if then FtpFileObject.getChildFile() is called (this may happen when
getType() is invoked for one of the child files - see below), current
implementation will ignore 'attached' state and silently fill children cache
by calling doGetChildren().

 

The result is that FtpFileObject remains in inconsistent state: it is not
attached with AbstractFileObject.type field = null, child names cache
(AbstractFileObject.children)  = null, but has children cache bulit.

 

I'm not currently sure which consequences it may cause.

 

What I've found up to date is that when I load list of children I see that
two FTP LIST commands are issued: 

-          first one is to list the children themselves

-          then parent file is refreshed since CacheStrategy is ON_RESOLVE
(this is actually another issue I'd like to discuss, since it gives a
performance drop clearing the children cache right after is was build)

-          and then another LIST call that comes from child.getType(). If
there was no inconsistency I described above, this LIST would have been
called for each of the children.

 

Hope all this makes sense,

 Regards,

 Kirill

AW: potential problem: FtpFileObject caches children when not attached

Posted by Mario Ivankovits <ma...@ops.co.at>.

Hi!

 

Hmmm … I tried to reproduce it like this:

 

                               FileObject fos =
VFS.getManager().resolveFile("ftp://XYZ.com/public/ test with space", fops);

                               fos.getType();

                               fos.getChildren();

 

                               fos.refresh();

                               fos.getType();

                               fos.getChildren();

 

After the refresh() I just get one ftp list … can you please outline with
code how to reproduce the problem.

Thanks!

 

Ciao,

Mario

 

 

Von: Kirill Safonov [mailto:kirill.safonov@gmail.com] 
Gesendet: Mittwoch, 12. Mai 2010 17:07
An: 'Commons Users List'
Cc: Mario Ivankovits
Betreff: potential problem: FtpFileObject caches children when not attached

 

Hello,

 

While digging a bit into FTP performance, I’ve noticed the following:

 

When FtpFileObject.refresh() is called, FtpFileObject.children field is
reset to null in FtpFileObject.doDetach(), and AbstractFileObject.attached
flag is set to false (this is OK)

But, if then FtpFileObject.getChildFile() is called (this may happen when
getType() is invoked for one of the child files – see below), current
implementation will ignore ‘attached’ state and silently fill children cache
by calling doGetChildren().

 

The result is that FtpFileObject remains in inconsistent state: it is not
attached with AbstractFileObject.type field = null, child names cache
(AbstractFileObject.children)  = null, but has children cache bulit.

 

I’m not currently sure which consequences it may cause.

 

What I’ve found up to date is that when I load list of children I see that
two FTP LIST commands are issued: 

-          first one is to list the children themselves

-          then parent file is refreshed since CacheStrategy is ON_RESOLVE
(this is actually another issue I’d like to discuss, since it gives a
performance drop clearing the children cache right after is was build)

-          and then another LIST call that comes from child.getType(). If
there was no inconsistency I described above, this LIST would have been
called for each of the children.

 

Hope all this makes sense,

 Regards,

 Kirill