xrdp: Homedir gets not correctly created at first login

I’m opening this as a new issue, although my problems have already been started to be discussed at #325 which has been marked as “closed” before finding a solution for my specific problem.

Short summary here:

I built xrdp upstream version and have a problem when connecting to the server for the very first time with a user. The server should then create a homedir (via pam_mkhomedir). The server is a member of an active directory, PAM is configured to authenticate via samba/winbind and create home directories for users on their first login. This works perfectly for e.g. ssh or other services but not for xrdp.

I had a look at bugs #182 and #196 which seem similar but have a different background. The problem of these referenced bugs is that VNC needs some time to start up before connecting to it - so a timeout (loop) has been implemented. This is not the case here.

In this bug, VNC starts but there seems to be a problem with the order a full PAM based authentification. xrdp is supposed to first do the PAM authentication (which would initially creating a users’ home directory) and then start VNC. This seems not to work as expected somehow… In general, the home directory gets created somewhen, but obviously too late in the process so that VNC spits out the error lines

[20160229-15:51:49] [ERROR] env_set_user: error creating .vnc dir
[20160229-15:51:49] [WARN ] can't write vnc password hash file - /home/DOMAIN/username/.vnc/sesman_username_passwd

If I kill the Xvnc process (or completely restart xrdp) and try it again (after the homedir got created while the first login try), it works. If I create the user’s home directory manually it also works.

I wasn’t able to let PAM create the homedir before VNC starts, I tried almost every possible order of “session” entries (especially those with pam_mkhomedir.so at the beginning).