xrdp: Windows 10 client presents certificate, then fails to connect

I’m using xrdp as a server on a Archlinux box, and was able to connect successfully from my Win10 machine until April 1 (strangely enough). As far as I’m aware, nothing changed on either machine at the time; I have since updated the server box and rebooted both machines. The issue has persisted.

The Win10 “Remote Desktop Connection” client shows a name mismatch on the security certificate, as usual–I’m connecting by ip address, and it says the name on the certificate is www.xrdp.org --but when I hit ‘Yes’ to complete the connection, I get an error “This computer can’t connect to the remote computer. Try connecting again. If the problem continues, contact the owner of the remote computer or your network administrator.” When I hit ‘OK’ there, I’m back to the initial RDC screen.

systemctl status xrdp-sesman

● xrdp-sesman.service - xrdp session manager
     Loaded: loaded (/usr/lib/systemd/system/xrdp-sesman.service; enabled; vendor preset: disabled)
     Active: active (running) since Sat 2021-04-10 15:42:04 EDT; 2min 43s ago
       Docs: man:xrdp-sesman(8)
             man:sesman.ini(5)
    Process: 569 ExecStart=/usr/bin/xrdp-sesman (code=exited, status=0/SUCCESS)
   Main PID: 580 (xrdp-sesman)
      Tasks: 1 (limit: 19132)
     Memory: 1.9M
        CPU: 7ms
     CGroup: /system.slice/xrdp-sesman.service
             └─580 /usr/bin/xrdp-sesman

Apr 10 15:42:04 kotomi systemd[1]: Starting xrdp session manager...
Apr 10 15:42:04 kotomi xrdp-sesman[580]: [INFO ] starting xrdp-sesman with pid 580
Apr 10 15:42:04 kotomi systemd[1]: Started xrdp session manager.
Apr 10 15:42:04 kotomi xrdp-sesman[580]: [INFO ] listening to port 3350 on 127.0.0.1

that looks like sesman never sees a connection…

systemctl status xrdp

● xrdp.service - xrdp daemon
     Loaded: loaded (/usr/lib/systemd/system/xrdp.service; enabled; vendor preset: disabled)
     Active: active (running) since Sat 2021-04-10 15:42:05 EDT; 9min ago
       Docs: man:xrdp(8)
             man:xrdp.ini(5)
    Process: 581 ExecStart=/usr/bin/xrdp (code=exited, status=0/SUCCESS)
   Main PID: 597 (xrdp)
      Tasks: 1 (limit: 19132)
     Memory: 3.7M
        CPU: 20ms
     CGroup: /system.slice/xrdp.service
             └─597 /usr/bin/xrdp

Apr 10 15:44:33 kotomi xrdp[1630]: [INFO ] adding channel item name cliprdr chan_id 1006 flags 0xc0a00000
Apr 10 15:44:33 kotomi xrdp[1630]: [INFO ] adding channel item name drdynvc chan_id 1007 flags 0xc0800000
Apr 10 15:44:33 kotomi xrdp[1630]: [ERROR] error unknown xrdp_sec_process_mcs_data tag 0xc006 size 8
Apr 10 15:44:33 kotomi xrdp[1630]: [ERROR] error unknown xrdp_sec_process_mcs_data tag 0xc00a size 8
Apr 10 15:44:33 kotomi xrdp[1630]: [INFO ] xrdp_load_keyboard_layout: keyboard_type [4] keyboard_subtype [0]
Apr 10 15:44:33 kotomi xrdp[1630]: [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
Apr 10 15:44:33 kotomi xrdp[1630]: [INFO ] TLS connection established from 192.168.0.100 port 65082: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
Apr 10 15:44:33 kotomi xrdp[1630]: [WARN ] libxrdp_force_read: header read error
Apr 10 15:44:33 kotomi xrdp[1630]: [ERROR] libxrdp_process_data: libxrdp_force_read failed
Apr 10 15:44:34 kotomi xrdp[1630]: [ERROR]   out xrdp_mcs_disconnect error - 2

I’m assuming the last errors here are relevant, but google hasn’t been of much help finding any similar issues. (The first two “unknown tag” errors also showed up when everything was working, so I’m assuming they’re not related.)

Here’s the full xrdp.log from the related time period:

[20210410-15:44:32] [INFO ] Socket 12: AF_INET connection received from 192.168.0.100 port 65080
[20210410-15:44:32] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20210410-15:44:32] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20210410-15:44:32] [WARN ] libxrdp_force_read: header read error
[20210410-15:44:32] [ERROR]   out xrdp_mcs_disconnect error - 2
[20210410-15:44:33] [INFO ] Socket 12: AF_INET connection received from 192.168.0.100 port 65082
[20210410-15:44:33] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20210410-15:44:33] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20210410-15:44:33] [INFO ] connected client computer name: RYOKO
[20210410-15:44:33] [INFO ]   client supports 40 bit encryption
[20210410-15:44:33] [INFO ]   client supports 128 bit encryption
[20210410-15:44:33] [INFO ]   client supports 56 bit encryption
[20210410-15:44:33] [INFO ]   client supports fips encryption
[20210410-15:44:33] [INFO ] adding channel item name rdpdr chan_id 1004 flags 0x80800000
[20210410-15:44:33] [INFO ] adding channel item name rdpsnd chan_id 1005 flags 0xc0000000
[20210410-15:44:33] [INFO ] adding channel item name cliprdr chan_id 1006 flags 0xc0a00000
[20210410-15:44:33] [INFO ] adding channel item name drdynvc chan_id 1007 flags 0xc0800000
[20210410-15:44:33] [ERROR] error unknown xrdp_sec_process_mcs_data tag 0xc006 size 8
[20210410-15:44:33] [ERROR] error unknown xrdp_sec_process_mcs_data tag 0xc00a size 8
[20210410-15:44:33] [INFO ] xrdp_load_keyboard_layout: keyboard_type [4] keyboard_subtype [0]
[20210410-15:44:33] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20210410-15:44:33] [INFO ] TLS connection established from 192.168.0.100 port 65082: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
[20210410-15:44:33] [WARN ] libxrdp_force_read: header read error
[20210410-15:44:33] [ERROR] libxrdp_process_data: libxrdp_force_read failed
[20210410-15:44:34] [ERROR]   out xrdp_mcs_disconnect error - 2

I’ve tried with UFW enabled and disabled, no difference. Since the above log says the TLS connection was made, I’m assuming it’s not a network issue.

xrdp.ini

[Globals]
; xrdp.ini file version number
ini_version=1

; fork a new process for each incoming connection
fork=true

; ports to listen on, number alone means listen on all interfaces
; 0.0.0.0 or :: if ipv6 is configured
; space between multiple occurrences
; ALL specified interfaces must be UP when xrdp starts, otherwise xrdp will fail to start
;
; Examples:
;   port=3389
;   port=unix://./tmp/xrdp.socket
;   port=tcp://.:3389                           127.0.0.1:3389
;   port=tcp://:3389                            *:3389
;   port=tcp://<any ipv4 format addr>:3389      192.168.1.1:3389
;   port=tcp6://.:3389                          ::1:3389
;   port=tcp6://:3389                           *:3389
;   port=tcp6://{<any ipv6 format addr>}:3389   {FC00:0:0:0:0:0:0:1}:3389
;   port=vsock://<cid>:<port>
port=3389

; 'port' above should be connected to with vsock instead of tcp
; use this only with number alone in port above
; prefer use vsock://<cid>:<port> above
use_vsock=false

; regulate if the listening socket use socket option tcp_nodelay
; no buffering will be performed in the TCP stack
tcp_nodelay=true

; regulate if the listening socket use socket option keepalive
; if the network connection disappear without close messages the connection will be closed
tcp_keepalive=true

; set tcp send/recv buffer (for experts)
#tcp_send_buffer_bytes=32768
#tcp_recv_buffer_bytes=32768

; security layer can be 'tls', 'rdp' or 'negotiate'
; for client compatible layer
security_layer=negotiate

; minimum security level allowed for client for classic RDP encryption
; use tls_ciphers to configure TLS encryption
; can be 'none', 'low', 'medium', 'high', 'fips'
crypt_level=high

; X.509 certificate and private key
; openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 365
certificate=
key_file=

; set SSL protocols
; can be comma separated list of 'SSLv3', 'TLSv1', 'TLSv1.1', 'TLSv1.2', 'TLSv1.3'
ssl_protocols=TLSv1.2, TLSv1.3
; set TLS cipher suites
#tls_ciphers=HIGH

; concats the domain name to the user if set for authentication with the separator
; for example when the server is multi homed with SSSd
#domain_user_separator=@

; Section name to use for automatic login if the client sends username
; and password. If empty, the domain name sent by the client is used.
; If empty and no domain name is given, the first suitable section in
; this file will be used.
autorun=

allow_channels=true
allow_multimon=true
bitmap_cache=true
bitmap_compression=true
bulk_compression=true
#hidelogwindow=true
max_bpp=32
new_cursors=true
; fastpath - can be 'input', 'output', 'both', 'none'
use_fastpath=both
; when true, userid/password *must* be passed on cmd line
#require_credentials=true
; when true, the userid will be used to try to authenticate
#enable_token_login=true
; You can set the PAM error text in a gateway setup (MAX 256 chars)
#pamerrortxt=change your password according to policy at http://url

;
; colors used by windows in RGB format
;
blue=009cb5
grey=dedede
#black=000000
#dark_grey=808080
#blue=08246b
#dark_blue=08246b
#white=ffffff
#red=ff0000
#green=00ff00
#background=626c72

address=0.0.0.0

;
; configure login screen
;

; Login Screen Window Title
#ls_title=My Login Title

; top level window background color in RGB format
ls_top_window_bg_color=009cb5

; width and height of login screen
ls_width=350
ls_height=430

; login screen background color in RGB format
ls_bg_color=dedede

; optional background image filename (bmp format).
#ls_background_image=

; logo
; full path to bmp-file or file in shared folder
ls_logo_filename=
ls_logo_x_pos=55
ls_logo_y_pos=50

; for positioning labels such as username, password etc
ls_label_x_pos=30
ls_label_width=65

; for positioning text and combo boxes next to above labels
ls_input_x_pos=110
ls_input_width=210

; y pos for first label and combo box
ls_input_y_pos=220

; OK button
ls_btn_ok_x_pos=142
ls_btn_ok_y_pos=370
ls_btn_ok_width=85
ls_btn_ok_height=30

; Cancel button
ls_btn_cancel_x_pos=237
ls_btn_cancel_y_pos=370
ls_btn_cancel_width=85
ls_btn_cancel_height=30

[Logging]
; Note: Log levels can be any of: core, error, warning, info, debug, or trace
LogFile=xrdp.log
LogLevel=INFO
EnableSyslog=true
#SyslogLevel=INFO
#EnableConsole=false
#ConsoleLevel=INFO
#EnableProcessId=false

[LoggingPerLogger]
; Note: per logger configuration is only used in XRDP_DEBUG builds of XRDP.
#xrdp.c=INFO
#main()=INFO

[Channels]
; Channel names not listed here will be blocked by XRDP.
; You can block any channel by setting its value to false.
; IMPORTANT! All channels are not supported in all use
; cases even if you set all values to true.
; You can override these settings on each session type
; These settings are only used if allow_channels=true
rdpdr=true
rdpsnd=true
drdynvc=true
cliprdr=true
rail=true
xrdpvr=true
tcutils=true

; for debugging xrdp, in section xrdp1, change port=-1 to this:
#port=/tmp/.xrdp/xrdp_display_10

; for debugging xrdp, add following line to section xrdp1
#chansrvport=/tmp/.xrdp/xrdp_chansrv_socket_7210


;
; Session types
;

; Some session types such as Xorg, X11rdp and Xvnc start a display server.
; Startup command-line parameters for the display server are configured
; in sesman.ini. See and configure also sesman.ini.
[Xorg]
name=Xorg
lib=libxup.so
username=ask
password=ask
ip=127.0.0.1
port=-1
code=20

[Xvnc]
name=Xvnc
lib=libvnc.so
username=ask
password=ask
ip=127.0.0.1
port=-1
#xserverbpp=24
#delay_ms=2000
; Disable requested encodings to support buggy VNC servers
; (1 = ExtendedDesktopSize)
#disabled_encodings_mask=0


[vnc-any]
name=vnc-any
lib=libvnc.so
ip=ask
port=ask5900
username=na
password=ask
#pamusername=asksame
#pampassword=asksame
#pamsessionmng=127.0.0.1
#delay_ms=2000

[neutrinordp-any]
name=neutrinordp-any
lib=libxrdpneutrinordp.so
ip=ask
port=ask
username=ask
password=ask

; You can override the common channel settings for each session type
#channel.rdpdr=true
#channel.rdpsnd=true
#channel.drdynvc=true
#channel.cliprdr=true
#channel.rail=true
#channel.xrdpvr=true

I’m not sure where to go from here. Is it possibly an encryption failure somehow? I’ll be happy to post additional settings or logs if they would be useful.