[jira] [Updated] (GUACAMOLE-1717) Cursor Improperly Initialized when Connecting to Gnome Remote Desktop via RDP

["Mike Powers (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GUACAMOLE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Powers updated GUACAMOLE-1717:
-----------------------------------
    Attachment: image-2022-11-20-20-14-11-333.png

> Cursor Improperly Initialized when Connecting to Gnome Remote Desktop via RDP
> -----------------------------------------------------------------------------
>
>                 Key: GUACAMOLE-1717
>                 URL: https://issues.apache.org/jira/browse/GUACAMOLE-1717
>             Project: Guacamole
>          Issue Type: Bug
>          Components: RDP
>    Affects Versions: 1.4.0
>            Reporter: Mike Powers
>            Priority: Minor
>         Attachments: image-2022-11-20-20-14-11-333.png
>
>
> When connecting to a [Gnome Remote Desktop|https://gitlab.gnome.org/GNOME/gnome-remote-desktop] RDP instance the mouse cursor is created but the image data is never copied into the cursor's pixel buffer resulting in a garbled bitmap containing uninitialized data:
> !image-2022-11-20-19-59-57-478.png|width=664,height=311!
> This appears to happen because of this guard condition in src/protocols/rdp/pointer.c:
> ```c
>     if (pointer->andMaskData && pointer->xorMaskData)
> ```
> It appears when Gnome Remote Desktop [initializes cursors|https://gitlab.gnome.org/GNOME/gnome-remote-desktop/-/blob/master/src/grd-session-rdp.c#L591] when starting the session it sets `andMaskData` to NULL with a length of zero:
> ```c
>       pointer_color->lengthAndMask = 0;
>       pointer_color->lengthXorMask = xor_mask_length;
>       pointer_color->andMaskData = NULL;
> ```
> This causes the guard condition in the RDP plugin to fail to copy bitmap data from the server. It appears as if FreeRDP [correctly handles|https://github.com/FreeRDP/FreeRDP/blob/eb7adaec46eceaa4b247dc664dcce2d65a5edead/libfreerdp/codec/color.c#L322] the image data in `freerdp_image_copy_from_pointer_data` if `lengthAndMask` == 0:
> ```c
> 	if (!xorMask || (xorMaskLength == 0))
> 		return FALSE;
> 	if (!andMask || (andMaskLength == 0))
> 		return FALSE;
> ```
> I believe this guard condition is probably not necessary; removing it resolves the issue for me. I tested against both Gnome Remote Desktop and a Windows 11 Professional instance.
> Please let me know if you'd like any additional information or testing done. This was tested with Guacamole 1.4.0 from Docker Hub (guacamole/guacamole:1.4.0 and guacamole/guacd:1.4.0). Testing against a fresh install of Fedora 37/Gnome 43.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)