[jira] [Comment Edited] (GUACAMOLE-818) Missing some files in the sftp file system

["Mike Jumper (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GUACAMOLE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925323#comment-16925323 ] 

Mike Jumper edited comment on GUACAMOLE-818 at 2/25/20 1:19 AM:
----------------------------------------------------------------

I have analyzed the causes for the above cases.

If the number of files and directories is large, it seems to be caused by exceeding the buffer size. (more than 4096 bytes).

If the buffer size is exceeded, the buffer is flushed, at the same time the blob_written value is changed to 1.

As the blob_written value changes 1, the next file or directory seems to be missing without being true in the while conditional statement.

(For example, if the file test_049 is missing, the size up to test_048 will exceed 4096 bytes and then the next file test_049 will be missing.)

Please refer to the sources below and consider them in the next patch.

{code:c|title=src/common/json.c}
/* Flush if more room is needed */
if (json_state->size + blob_length > sizeof(json_state->buffer)) {
    guac_common_json_flush(user, stream, json_state); 
    blob_written = 1; 
}
{code}

{code:c|title=src/common-ssh/sftp.c}
/* While directory entries remain */
while ((bytes_read = libssh2_sftp_readdir(list_state->directory,
            filename, sizeof(filename), &attributes)) > 0
        && !blob_written) {
    ...
}

/* Write entry */
blob_written |= guac_common_json_write_property(user, stream,
        &list_state->json_state, absolute_path, mimetype);
{code}



was (Author: kwanghee83.lee):
I have analyzed the causes for the above cases.

If the number of files and directories is large, it seems to be caused by exceeding the buffer size. (more than 4096 bytes).

If the buffer size is exceeded, the buffer is flushed, at the same time the blob_written value is changed to 1.

As the blob_written value changes 1, the next file or directory seems to be missing without being true in the while conditional statement.

(For example, if the file test_049 is missing, the size up to test_048 will exceed 4096 bytes and then the next file test_049 will be missing.)

Please refer to the sources below and consider them in the next patch.

- src/common/json.c

-> /* Flush if more room is needed */
if (json_state->size + blob_length > sizeof(json_state->buffer)) { 
 guac_common_json_flush(user, stream, json_state); 
blob_written = 1; 
}

- src/common-ssh/sftp.c

-> /* While directory entries remain */
 while ((bytes_read = libssh2_sftp_readdir(list_state->directory,
 filename, sizeof(filename), &attributes)) > 0
 && !blob_written) \{ ... }

-> /* Write entry */
 blob_written |= guac_common_json_write_property(user, stream,
 &list_state->json_state, absolute_path, mimetype);

 

> Missing some files in the sftp file system
> ------------------------------------------
>
>                 Key: GUACAMOLE-818
>                 URL: https://issues.apache.org/jira/browse/GUACAMOLE-818
>             Project: Guacamole
>          Issue Type: Bug
>          Components: guacamole
>    Affects Versions: 1.0.0
>            Reporter: kwangheelee
>            Assignee: Mike Jumper
>            Priority: Minor
>             Fix For: 1.2.0
>
>         Attachments: blob-20190621.json, filesystem1.png, filesystem2.png, location-of-20190419-within-json.png, missing1.png, missing2.png
>
>
> I am currently developing using Guacamole version 1.0.0. There are some bugs in uploading/downloading files through the sftp file system.
> I checked the source and found that when importing the sftp file structure, if the number of files and directories is large, it is obtained by dividing buffer. It does not matter when the number of files is small, but when the number of files is large, it is confirmed that some files are missing.
> For example, when loading a file system with three buffers, two files are missing. 
> =============================================================
> Client : 3.get,1.0,31./home/guacd/guacamole/recording;
>  Server : 4.body,1.0,1.0,53.application/vnd.glyptodon.guacamole.stream-index+json,31./home/guacd/guacamole/recording;
>  Client : 3.ack,1.0,5.Ready,1.0;
>  Server : 4.blob,1.0,5416.SOME DATA;
>  Client : 3.ack,1.0,8.Received,1.0;
>  Server : 4.blob,1.0,3168.SOME DATA;3.end,1.0;
>  client : 3.ack,1.0,8.Received,1.0;
> =============================================================
> I do not know if it is a guacamole issue or a libssh2 issue, and I would like to make a correction.
> Attachment can not be transferred because of security problem in company system.
> (Please write your email address in Comments and I will send you an attachment.)
> Thanks.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)