[jira] [Closed] (GUACAMOLE-1717) Cursor Improperly Initialized when Connecting to Gnome Remote Desktop via RDP

["Mike Jumper (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GUACAMOLE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Jumper closed GUACAMOLE-1717.
----------------------------------
    Fix Version/s:     (was: 1.6.0)
       Resolution: Fixed

> Cursor Improperly Initialized when Connecting to Gnome Remote Desktop via RDP
> -----------------------------------------------------------------------------
>
>                 Key: GUACAMOLE-1717
>                 URL: https://issues.apache.org/jira/browse/GUACAMOLE-1717
>             Project: Guacamole
>          Issue Type: Bug
>          Components: RDP
>    Affects Versions: 1.4.0
>            Reporter: Mike Powers
>            Assignee: Mike Jumper
>            Priority: Minor
>             Fix For: 1.5.2
>
>         Attachments: image-2022-11-20-20-14-11-333.png
>
>
> When connecting to a [Gnome Remote Desktop|https://gitlab.gnome.org/GNOME/gnome-remote-desktop] RDP instance the mouse cursor is created but the image data is never copied into the cursor's pixel buffer resulting in a garbled bitmap containing uninitialized data:
> !image-2022-11-20-20-14-11-333.png|width=656,height=355!
> This appears to happen because of this [guard condition|https://github.com/apache/guacamole-server/blob/3ca6bb0a61715d17dc263e565e6ce0de7354606c/src/protocols/rdp/pointer.c#L50] in src/protocols/rdp/pointer.c (line 49):
> {code:java}
> if (pointer->andMaskData && pointer->xorMaskData) {code}
>  
> It appears when Gnome Remote Desktop [initializes cursors|https://gitlab.gnome.org/GNOME/gnome-remote-desktop/-/blob/master/src/grd-session-rdp.c#L591] when starting the session it sets `andMaskData` to NULL with a length of zero:
> {code:java}
>       pointer_color->lengthAndMask = 0;
>       pointer_color->lengthXorMask = xor_mask_length;
>       pointer_color->andMaskData = NULL; {code}
> This causes the guard condition in the RDP plugin to fail to copy bitmap data from the server. It appears as if FreeRDP [correctly handles|https://github.com/FreeRDP/FreeRDP/blob/master/libfreerdp/codec/color.c#L430] the image data in {{freerdp_image_copy_from_pointer_data()}} if {{lengthAndMask}} == 0:
> {code:java}
>     if (andMask)
>     {
>         if (andStep * nHeight > andMaskLength)
>             return FALSE;
>     }{code}
> I believe this guard condition is probably not necessary; removing it resolves the issue for me. I tested against both Gnome Remote Desktop and a Windows 11 Professional instance.
> Please let me know if you'd like any additional information or testing done. This was tested with Guacamole 1.4.0 from Docker Hub (guacamole/guacamole:1.4.0 and guacamole/guacd:1.4.0). Testing against a fresh install of Fedora 37/Gnome 43.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)