You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@guacamole.apache.org by Zujian YU <fi...@anz.com.INVALID> on 2025/05/12 06:34:32 UTC
Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
Hello,
We are running Gucamole solution on AWS environment, and it works for years, recently we have issue to access it. The Gucamole EC2 server join AWS managed AD, and the AWS managed AD setup on-way trust to On-Prem DC AD which host the end user credentials.
Env:
OS: RHEL8.10
Gucamole: 1.5.5
Apache-tomcat: 9.0.100
Configuration:
# cat /etc/guacamole/guacd.conf
[daemon]
pid_file = /var/run/guacd.pid
log_level = info
[server]
bind_host = 127.0.0.1
bind_port = 4822
# cat /usr/share/tomcat/.guacamole/guacamole.properties
guacd-hostname: 127.0.0.1
guacd-port: 4822
auth-provider: net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider
mysql-hostname: rds-url
mysql-port: 3306
mysql-database: dbname
mysql-username: username
mysql-password: passwd
mysql-default-max-connections-per-user: 0
mysql-default-max-group-connections-per-user: 0
# LDAP properties
ldap-hostname: onprem-ad-***.com
ldap-port: 636
ldap-encryption-method: ssl
ldap-search-bind-dn: CN=**,OU=**,OU=**,OU=**,DC=**,DC=**,DC=com
ldap-search-bind-password: ********
ldap-user-base-dn: DC=**,DC=**,DC=com
ldap-username-attribute: sAMAccountName
ldap-user-search-filter: (memberOf=CN=**,OU=**,OU=**,OU=**,OU=**,DC=**,DC=**,DC=com)
# TOTP properties
totp-issuer AWS 3.0 (MAIN) - **
totp-mode sha1
Symptom:
1. Some users login Gucamole portal with timeout error, and found below logs:
#cat /var/log/tomcat/catalina.out
…
00:36:00.468 [NioProcessor-160] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:01.011 [http-nio-8080-exec-21] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:01.012 [http-nio-8080-exec-21] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:11.072 [NioProcessor-162] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:13.754 [NioProcessor-163] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 9891 milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 17734 milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 16344 milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
…
1. I can login Gucamole portal but when click the AWS EC2 server to ssh, error log show:
1:28:06.312 [NioProcessor-183] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.tunnel.TunnelRequestService - User “***” connected to connection "1874".
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
01:31:14.621 [http-nio-8080-exec-5] INFO o.a.g.tunnel.TunnelRequestService - User “***” disconnected from connection "1874". Duration: 75177 milliseconds
01:31:14.635 [http-nio-8080-exec-5] ERROR o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request failed: Connection to guacd timed out.
01:31:14.808 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
01:31:14.814 [http-nio-8080-exec-4] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
Verfication:
* AWS subnet NACLs / Guacamole EC2 security group rules are all good to whitelist required traffic.
* The Gucamole EC2 join domain successfully
# net ads info
LDAP server: 10.*.*.*
LDAP server name: * (AD server of AWS managed AD)
Realm: *.com (the domain of AWS managed AD)
Bind Path: dc=*,dc=*,dc=*,dc=*,dc=*,dc=COM
LDAP port: 389
Server time: Mon, 12 May 2025 06:17:04 UTC
KDC server: 10.*.*.* (same as LDAP server)
Server time offset: 0
Last machine account password change: Mon, 05 May 2025 06:03:02 UTC
* Tomcat and guacd status ( also restarted it but not works)
# systemctl status tomcat guacd
● tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/etc/systemd/system/tomcat.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2025-05-12 03:31:12 UTC; 2h 58min ago
Process: 1780229 ExecStop=/usr/share/tomcat/bin/shutdown.sh (code=exited, status=0/SUCCESS)
Process: 1780349 ExecStart=/usr/share/tomcat/bin/startup.sh (code=exited, status=0/SUCCESS)
Main PID: 1780359 (java)
Tasks: 40 (limit: 99182)
Memory: 772.6M
CGroup: /system.slice/tomcat.service
└─1780359 /usr/lib/jvm/jre-openjdk/bin/java -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.head>
May 12 03:31:12 ac152d4d39a4432 systemd[1]: Starting Apache Tomcat Web Application Container...
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Existing PID file found during start.
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Removing/clearing stale PID file.
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Tomcat started.
May 12 03:31:12 ac152d4d39a4432 systemd[1]: Started Apache Tomcat Web Application Container.
● guacd.service - LSB: Guacamole proxy daemon
Loaded: loaded (/etc/rc.d/init.d/guacd; generated)
Active: active (running) since Mon 2025-05-12 03:31:18 UTC; 2h 57min ago
Docs: man:systemd-sysv-generator(8)
Process: 1780442 ExecStop=/etc/rc.d/init.d/guacd stop (code=exited, status=0/SUCCESS)
Process: 1780445 ExecStart=/etc/rc.d/init.d/guacd start (code=exited, status=0/SUCCESS)
Tasks: 1 (limit: 99182)
Memory: 10.5M
CGroup: /system.slice/guacd.service
└─1780450 /usr/local/sbin/guacd -p /var/run/guacd.pid
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending normalization specified. Defaulting to preserving the format of all line endings.
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will be saved to "/opt/guacamole/recording/screen/992382480584 - mx-asg-db-connect-j13555-j13555-ap-southeast-3 - i-0b77f582013cbc860//202505>
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "en-us-qwerty"
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused connection: Disconnected.
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected
May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.
Any suggestion is appreciated.
thanks
This e-mail and any attachments to it (the "Communication") is, unless otherwise stated,confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication.
Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
Posted by Zujian YU <fi...@anz.com.INVALID>.
Hello Community,
Has anyone encountered similar issue?
Kind regards
From: Zujian YU <fi...@anz.com.INVALID>
Date: Monday, 12 May 2025 at 10:14 pm
To: user@guacamole.apache.org <us...@guacamole.apache.org>
Subject: Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
CAUTION: This email originated from outside of ANZ. Verify the sender is authentic and that the content is safe before taking action, clicking links, opening attachments, and/or following instructions (e.g. payment requests).
Hello Nick,
Thank you for the comment.
In our installation, end user (On-Prem) <> AWS load balancer <> AWS Guacamole EC2, and AWS Guacamole EC2 joined AWS managed AD with one-way trust to On-Prem AD.
It’s unlikely any issue between AWS load balancer <> AWS Guacamole EC2 and AWS Guacamole EC2 <> AWS managed AD which we managed subnet NACL rules and security group rules. Can you suggest which path to be checked?
And as below error logs show:
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 9891 milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 17734 milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 16344 milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
What’s those GuacamoleHTTPTunnel error means? Is that something inside guaca service?
Kind regards
From: Nick Couchman <vn...@apache.org>
Date: Monday, 12 May 2025 at 9:23 pm
To: user@guacamole.apache.org <us...@guacamole.apache.org>
Subject: Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
CAUTION: This email originated from outside of ANZ. Verify the sender is authentic and that the content is safe before taking action, clicking links, opening attachments, and/or following instructions (e.g. payment requests).
On Mon, May 12, 2025 at 2:37 AM Zujian YU <fi...@anz.com.invalid> wrote:
Hello,
We are running Gucamole solution on AWS environment, and it works for years, recently we have issue to access it. The Gucamole EC2 server join AWS managed AD, and the AWS managed AD setup on-way trust to On-Prem DC AD which host the end user credentials.
Env:
OS: RHEL8.10
Gucamole: 1.5.5
Apache-tomcat: 9.0.100
Configuration:
# cat /etc/guacamole/guacd.conf
[daemon]
pid_file = /var/run/guacd.pid
log_level = info
[server]
bind_host = 127.0.0.1
bind_port = 4822
# cat /usr/share/tomcat/.guacamole/guacamole.properties
guacd-hostname: 127.0.0.1
guacd-port: 4822
auth-provider: net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider
Please note that the "auth-provider" property was removed a long time ago and has absolutely no effect on the configuration.
Symptom:
1. Some users login Gucamole portal with timeout error, and found below logs:
#cat /var/log/tomcat/catalina.out
…
00:36:00.468 [NioProcessor-160] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:01.011 [http-nio-8080-exec-21] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:01.012 [http-nio-8080-exec-21] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:11.072 [NioProcessor-162] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:13.754 [NioProcessor-163] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 9891 milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 17734 milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 16344 milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
…
1. I can login Gucamole portal but when click the AWS EC2 server to ssh, error log show:
1:28:06.312 [NioProcessor-183] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.tunnel.TunnelRequestService - User “***” connected to connection "1874".
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
01:31:14.621 [http-nio-8080-exec-5] INFO o.a.g.tunnel.TunnelRequestService - User “***” disconnected from connection "1874". Duration: 75177 milliseconds
01:31:14.635 [http-nio-8080-exec-5] ERROR o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request failed: Connection to guacd timed out.
01:31:14.808 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
01:31:14.814 [http-nio-8080-exec-4] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
Verfication:
* AWS subnet NACLs / Guacamole EC2 security group rules are all good to whitelist required traffic.
* The Gucamole EC2 join domain successfully
[...]
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending normalization specified. Defaulting to preserving the format of all line endings.
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will be saved to "/opt/guacamole/recording/screen/992382480584 - mx-asg-db-connect-j13555-j13555-ap-southeast-3 - i-0b77f582013cbc860//202505>
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "en-us-qwerty"
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused connection: Disconnected.
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected
May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.
Any suggestion is appreciated.
I think some more network-level troubleshooting is going to be required - all of the messages you posted and symptoms you describe appear to be network-related. In the immediate message above, the reason for the failure is given: RDP server closed/refused connection: Disconnected. This indicates that, for one reason or another, Guacamole could not establish the connection to the RDP server. Could be a VPC-level issue (routing, ACL, Security Groups, etc.), or it could be a configuration issue (wrong security level, for example). Combine this with the LDAP issues you mention earlier, and I would definitely say something network-related is going on - Guacamole is having trouble consistently connecting to or maintaining a connection to the LDAP server.
Aside from network issues, you might check resources utilization - depending on how many users you have logging in concurrently, it could be that network connections are failing or getting dropped because the system running Guacamole does not have the RAM or CPU to handle the number of connections. Also, since you're running RHEL, you might want to make sure that SELinux is not interfering with the connections. I don't recommend disabling SELinux entirely, but you can use audit2why to examine the /var/log/audit/audit.log file and see if SELinux is dropping or preventing any connections.
-Nick
This e-mail and any attachments to it (the "Communication") is, unless otherwise stated,confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication.
This e-mail and any attachments to it (the "Communication") is, unless otherwise stated,confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication.
Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
Posted by Nick Couchman <vn...@apache.org>.
On Mon, May 12, 2025 at 8:14 AM Zujian YU <fi...@anz.com.invalid> wrote:
> Hello Nick,
>
>
>
> Thank you for the comment.
>
>
>
> In our installation, end user (On-Prem) <> AWS load balancer <> AWS
> Guacamole EC2, and AWS Guacamole EC2 joined AWS managed AD with one-way
> trust to On-Prem AD.
>
If you're using a load balancer, please be aware that Guacamole does not
currently have any native support for coordinating sessions among multiple
instances, so your load balancer will need to be configured to consistently
send users to the same Guacamole instance based on something
deterministic about that user. I haven't used AWS load balancer in a while,
but I'm sure there are some various options to configure session
"stickiness" that you can employ.
>
>
> It’s unlikely any issue between AWS load balancer <> AWS Guacamole EC2 and
> AWS Guacamole EC2 <> AWS managed AD which we managed subnet NACL rules and
> security group rules. Can you suggest which path to be checked?
>
All of the things I already mentioned:
* Validate resource utilization on the EC2 systems - make sure you're not
running out of RAM, using excessively high CPU, etc.
* Validate traffic flow in the VPC - verify that all of the traffic you
expect is making it from client to AWS EC2, and from AWS EC2 to RDP
servers. Check for dropped packets, routing issues, traffic not making it
to the destination, replies not making it back to the source, etc.
* Valid traffic flow between the VPC and your on-premise environment -
similar to above - is all the traffic making it between the two
environments, is anything being dropped, etc.?
* Validate connection parameters, particularly with respect to security
config. This is less about LDAP and more about the RDP error you were
seeing - something is blocking that communication, either network or
security-related, and you'll need to dig in to figure out what it is. I'd
suggest putting guacd into debug logging mode and reproduce the issue, and
see if you get any more information.
>
>
> And as below error logs show:
> *00:36:16.119 [http-nio-8080-exec-26] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to
> connection "1787".*
>
> *00:36:16.119 [http-nio-8080-exec-26] INFO
> o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
> WebSocket). Performance may be sub-optimal.*
>
> *00:36:26.010 [http-nio-8080-exec-27] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from
> connection "1787". Duration: 9891 milliseconds*
>
> *00:36:32.609 [http-nio-8080-exec-19] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to
> connection "1787".*
>
> *00:36:32.609 [http-nio-8080-exec-19] INFO
> o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
> WebSocket). Performance may be sub-optimal.*
>
> *00:36:50.343 [http-nio-8080-exec-2] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from
> connection "1787". Duration: 17734 milliseconds*
>
> *00:36:50.555 [http-nio-8080-exec-23] WARN
> o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such
> tunnel.*
>
> *00:37:33.522 [http-nio-8080-exec-27] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to
> connection "1787".*
>
> *00:37:33.522 [http-nio-8080-exec-27] INFO
> o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
> WebSocket). Performance may be sub-optimal.*
>
> *00:37:49.866 [http-nio-8080-exec-2] INFO
> o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from
> connection "1787". Duration: 16344 milliseconds*
>
> *00:37:50.071 [http-nio-8080-exec-21] WARN
> o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such
> tunnel.*
>
> What’s those GuacamoleHTTPTunnel error means? Is that something inside
> guaca service?
>
There are a couple of things going on, here. First, it looks like the AWS
load balancer either does not support or is not configured to support
WebSocket tunnels. As the log messages say, this can impact performance -
the HTTP tunnels should work, but the performance will be lower than it
would if WebSockets worked. I don't know if the AWS load balancer supports
WebSockets at all - probably it does if you're using a Layer3/4 method, but
it's possible the Layer7/HTTP load balancer just does not support it.
Second, I suspect you're actually seeing the results of what I mentioned
above - that the load balancer is sending sessions to different Guacamole
Client back-ends (Tomcat instances) and not persistently assigning sessions
to the correct server. So a user logs in with one Guacamole Client
instance, opens a connection, and then gets sent to another instance that
has no knowledge of the connection that has been opened. Again, you need to
check the session persistence (stickiness) configuration of the load
balancer and adjust so that clients consistently get sent to the same
Guacamole Client back-end.
-Nick
>
Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
Posted by Zujian YU <fi...@anz.com.INVALID>.
Hello Nick,
Thank you for the comment.
In our installation, end user (On-Prem) <> AWS load balancer <> AWS Guacamole EC2, and AWS Guacamole EC2 joined AWS managed AD with one-way trust to On-Prem AD.
It’s unlikely any issue between AWS load balancer <> AWS Guacamole EC2 and AWS Guacamole EC2 <> AWS managed AD which we managed subnet NACL rules and security group rules. Can you suggest which path to be checked?
And as below error logs show:
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 9891 milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 17734 milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 16344 milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
What’s those GuacamoleHTTPTunnel error means? Is that something inside guaca service?
Kind regards
From: Nick Couchman <vn...@apache.org>
Date: Monday, 12 May 2025 at 9:23 pm
To: user@guacamole.apache.org <us...@guacamole.apache.org>
Subject: Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
CAUTION: This email originated from outside of ANZ. Verify the sender is authentic and that the content is safe before taking action, clicking links, opening attachments, and/or following instructions (e.g. payment requests).
On Mon, May 12, 2025 at 2:37 AM Zujian YU <fi...@anz.com.invalid> wrote:
Hello,
We are running Gucamole solution on AWS environment, and it works for years, recently we have issue to access it. The Gucamole EC2 server join AWS managed AD, and the AWS managed AD setup on-way trust to On-Prem DC AD which host the end user credentials.
Env:
OS: RHEL8.10
Gucamole: 1.5.5
Apache-tomcat: 9.0.100
Configuration:
# cat /etc/guacamole/guacd.conf
[daemon]
pid_file = /var/run/guacd.pid
log_level = info
[server]
bind_host = 127.0.0.1
bind_port = 4822
# cat /usr/share/tomcat/.guacamole/guacamole.properties
guacd-hostname: 127.0.0.1
guacd-port: 4822
auth-provider: net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider
Please note that the "auth-provider" property was removed a long time ago and has absolutely no effect on the configuration.
Symptom:
1. Some users login Gucamole portal with timeout error, and found below logs:
#cat /var/log/tomcat/catalina.out
…
00:36:00.468 [NioProcessor-160] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:01.011 [http-nio-8080-exec-21] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:01.012 [http-nio-8080-exec-21] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:11.072 [NioProcessor-162] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully authenticated by LDAP server "onprem-ad-***.com".
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.r.auth.AuthenticationService - User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28, 127.0.0.1].
00:36:13.754 [NioProcessor-163] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 9891 milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 17734 milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService - User "userid-***" disconnected from connection "1787". Duration: 16344 milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
…
1. I can login Gucamole portal but when click the AWS EC2 server to ssh, error log show:
1:28:06.312 [NioProcessor-183] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.tunnel.TunnelRequestService - User “***” connected to connection "1874".
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.
01:31:14.621 [http-nio-8080-exec-5] INFO o.a.g.tunnel.TunnelRequestService - User “***” disconnected from connection "1874". Duration: 75177 milliseconds
01:31:14.635 [http-nio-8080-exec-5] ERROR o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request failed: Connection to guacd timed out.
01:31:14.808 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
01:31:14.814 [http-nio-8080-exec-4] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel.
Verfication:
* AWS subnet NACLs / Guacamole EC2 security group rules are all good to whitelist required traffic.
* The Gucamole EC2 join domain successfully
[...]
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending normalization specified. Defaulting to preserving the format of all line endings.
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will be saved to "/opt/guacamole/recording/screen/992382480584 - mx-asg-db-connect-j13555-j13555-ap-southeast-3 - i-0b77f582013cbc860//202505>
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "en-us-qwerty"
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused connection: Disconnected.
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected
May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection "$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.
Any suggestion is appreciated.
I think some more network-level troubleshooting is going to be required - all of the messages you posted and symptoms you describe appear to be network-related. In the immediate message above, the reason for the failure is given: RDP server closed/refused connection: Disconnected. This indicates that, for one reason or another, Guacamole could not establish the connection to the RDP server. Could be a VPC-level issue (routing, ACL, Security Groups, etc.), or it could be a configuration issue (wrong security level, for example). Combine this with the LDAP issues you mention earlier, and I would definitely say something network-related is going on - Guacamole is having trouble consistently connecting to or maintaining a connection to the LDAP server.
Aside from network issues, you might check resources utilization - depending on how many users you have logging in concurrently, it could be that network connections are failing or getting dropped because the system running Guacamole does not have the RAM or CPU to handle the number of connections. Also, since you're running RHEL, you might want to make sure that SELinux is not interfering with the connections. I don't recommend disabling SELinux entirely, but you can use audit2why to examine the /var/log/audit/audit.log file and see if SELinux is dropping or preventing any connections.
-Nick
This e-mail and any attachments to it (the "Communication") is, unless otherwise stated,confidential, may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication.
Re: Problem with Guacamole portal login and AWS EC2 SSH - HTTP Tunnel error
Posted by Nick Couchman <vn...@apache.org>.
On Mon, May 12, 2025 at 2:37 AM Zujian YU <fi...@anz.com.invalid> wrote:
> Hello,
>
>
>
> We are running Gucamole solution on AWS environment, and it works for
> years, recently we have issue to access it. The Gucamole EC2 server join
> AWS managed AD, and the AWS managed AD setup on-way trust to On-Prem DC AD
> which host the end user credentials.
>
>
>
> *Env*:
>
> OS: RHEL8.10
>
> Gucamole: 1.5.5
>
> Apache-tomcat: 9.0.100
>
>
>
> *Configuration:*
>
> # cat /etc/guacamole/guacd.conf
>
> *[daemon]*
>
> *pid_file = /var/run/guacd.pid*
>
> *log_level = info*
>
>
>
> *[server]*
>
> *bind_host = 127.0.0.1*
>
> *bind_port = 4822*
>
>
>
> # cat /usr/share/tomcat/.guacamole/guacamole.properties
>
>
>
> *guacd-hostname: 127.0.0.1*
>
> *guacd-port: 4822*
>
> *auth-provider:
> net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider*
>
>
>
Please note that the "auth-provider" property was removed a long time ago
and has absolutely no effect on the configuration.
>
>
> *Symptom:*
>
> 1. Some users login Gucamole portal with timeout error, and found
> below logs:
>
> #cat /var/log/tomcat/catalina.out
>
> …
>
> *00:36:00.468 [NioProcessor-160] WARN* *o.a.d.l.c.api.LdapNetworkConnection
> - null*
>
> *org.apache.mina.core.write.WriteToClosedSessionException: null*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)*
>
> *at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)*
>
> *at java.base/java.lang.Thread.run(Thread.java:829)*
>
> *00:36:01.011 [http-nio-8080-exec-21] INFO* *o.a.g.a.l.AuthenticationProviderService
> - User "userid-***" was successfully authenticated by LDAP server
> "onprem-ad-***.com".*
>
> *00:36:01.012 [http-nio-8080-exec-21] INFO* *o.a.g.r.auth.AuthenticationService
> - User "userid-***" successfully authenticated from [10.139.12.175,
> 10.73.192.28, 127.0.0.1].*
>
> *00:36:11.072 [NioProcessor-162] WARN* *o.a.d.l.c.api.LdapNetworkConnection
> - null*
>
> *org.apache.mina.core.write.WriteToClosedSessionException: null*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)*
>
> *at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)*
>
> *at java.base/java.lang.Thread.run(Thread.java:829)*
>
> *00:36:11.606 [http-nio-8080-exec-26] INFO* *o.a.g.a.l.AuthenticationProviderService
> - User "userid-***" was successfully authenticated by LDAP server
> "onprem-ad-***.com".*
>
> *00:36:11.606 [http-nio-8080-exec-26] INFO* *o.a.g.r.auth.AuthenticationService
> - User "userid-***" successfully authenticated from [10.139.12.175,
> 10.73.192.28, 127.0.0.1].*
>
> *00:36:13.754 [NioProcessor-163] WARN* *o.a.d.l.c.api.LdapNetworkConnection
> - null*
>
> *org.apache.mina.core.write.WriteToClosedSessionException: null*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)*
>
> *at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)*
>
> *at java.base/java.lang.Thread.run(Thread.java:829)*
>
> *00:36:16.119 [http-nio-8080-exec-26] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" connected to connection "1787".*
>
> *00:36:16.119 [http-nio-8080-exec-26] INFO* *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet
> - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.*
>
> *00:36:26.010 [http-nio-8080-exec-27] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" disconnected from connection "1787". Duration: 9891
> milliseconds*
>
> *00:36:32.609 [http-nio-8080-exec-19] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" connected to connection "1787".*
>
> *00:36:32.609 [http-nio-8080-exec-19] INFO* *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet
> - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.*
>
> *00:36:50.343 [http-nio-8080-exec-2] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" disconnected from connection "1787". Duration: 17734
> milliseconds*
>
> *00:36:50.555 [http-nio-8080-exec-23] WARN* *o.a.g.s.GuacamoleHTTPTunnelServlet
> - HTTP tunnel request rejected: No such tunnel.*
>
> *00:37:33.522 [http-nio-8080-exec-27] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" connected to connection "1787".*
>
> *00:37:33.522 [http-nio-8080-exec-27] INFO* *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet
> - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.*
>
> *00:37:49.866 [http-nio-8080-exec-2] INFO* *o.a.g.tunnel.TunnelRequestService
> - User "userid-***" disconnected from connection "1787". Duration: 16344
> milliseconds*
>
> *00:37:50.071 [http-nio-8080-exec-21] WARN* *o.a.g.s.GuacamoleHTTPTunnelServlet
> - HTTP tunnel request rejected: No such tunnel.*
>
>
>
> …
>
>
>
>
>
> 2. I can login Gucamole portal but when click the AWS EC2 server to
> ssh, error log show:
>
>
>
> *1:28:06.312 [NioProcessor-183] WARN* *o.a.d.l.c.api.LdapNetworkConnection
> - null*
>
> *org.apache.mina.core.write.WriteToClosedSessionException: null*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)*
>
> *at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)*
>
> *at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)*
>
> *at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)*
>
> *at java.base/java.lang.Thread.run(Thread.java:829)*
>
> *01:29:59.444 [http-nio-8080-exec-23] INFO* *o.a.g.tunnel.TunnelRequestService
> - User “***” connected to connection "1874".*
>
> *01:29:59.444 [http-nio-8080-exec-23] INFO* *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet
> - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.*
>
> *01:31:14.621 [http-nio-8080-exec-5] INFO* *o.a.g.tunnel.TunnelRequestService
> - User “***” disconnected from connection "1874". Duration: 75177
> milliseconds*
>
> *01:31:14.635 [http-nio-8080-exec-5] ERROR
> o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request failed: Connection
> to guacd timed out.*
>
> *01:31:14.808 [http-nio-8080-exec-21] WARN* *o.a.g.s.GuacamoleHTTPTunnelServlet
> - HTTP tunnel request rejected: No such tunnel.*
>
> *01:31:14.814 [http-nio-8080-exec-4] WARN* *o.a.g.s.GuacamoleHTTPTunnelServlet
> - HTTP tunnel request rejected: No such tunnel.*
>
>
>
> *Verfication:*
>
> - AWS subnet NACLs / Guacamole EC2 security group rules are all good
> to whitelist required traffic.
> - The Gucamole EC2 join domain successfully
>
> *[...]*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending
> normalization specified. Defaulting to preserving the format of all line
> endings.*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User
> "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection
> "$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will
> be saved to "/opt/guacamole/recording/screen/992382480584 -
> mx-asg-db-connect-j13555-j13555-ap-southeast-3 -
> i-0b77f582013cbc860//202505>*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"*
>
> *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap
> "en-us-qwerty"*
>
> *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused
> connection: Disconnected.*
>
> *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User
> "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)*
>
> *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection
> "$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected*
>
> *May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection
> "$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.*
>
>
>
> Any suggestion is appreciated.
>
>
>
I think some more network-level troubleshooting is going to be required -
all of the messages you posted and symptoms you describe appear to be
network-related. In the immediate message above, the reason for the failure
is given: RDP server closed/refused connection: Disconnected. This
indicates that, for one reason or another, Guacamole could not establish
the connection to the RDP server. Could be a VPC-level issue (routing, ACL,
Security Groups, etc.), or it could be a configuration issue (wrong
security level, for example). Combine this with the LDAP issues you mention
earlier, and I would definitely say something network-related is going on -
Guacamole is having trouble consistently connecting to or maintaining a
connection to the LDAP server.
Aside from network issues, you might check resources utilization -
depending on how many users you have logging in concurrently, it could be
that network connections are failing or getting dropped because the system
running Guacamole does not have the RAM or CPU to handle the number of
connections. Also, since you're running RHEL, you might want to make sure
that SELinux is not interfering with the connections. I don't recommend
disabling SELinux entirely, but you can use audit2why to examine the
/var/log/audit/audit.log file and see if SELinux is dropping or preventing
any connections.
-Nick
>