MailKit: NTLM authentication not working in some cases

It’s not in the list of SASL mechanisms that are tried by default, but starting with 2.0, you can use it manually:

var ntlm = new SaslMechanismNtlm ("username", "password");
client.Authenticate (ntlm);

Official docs for NTLM can be found here: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nlmp/b38c36ed-2804-4868-a9ff-8dd3182128e4

Another useful resource can be found here: http://davenport.sourceforge.net/ntlm.html

In fact, that second resource was the basis for the current NTLM implementation in MailKit (which was borrowed from Mono’s NTLM code).

@pawanbhabad

I sent you an email earlier detailing the differences that I found between MailKit and System.Net.Mail.

I did a bit of reading of the NTLM docs and have spotted 2 potential problems in MailKit’s code that I have “fixed”, but I am not 100% confident in either of the fixes.

What I need you to do is to test the NuGet package located at https://www.myget.org/feed/mimekit/package/nuget/MailKit/2.10.0.4

Once you’ve switched to using the above NuGet package, what I want you to do is to test the following cases for me:

1. According to the docs, that 24-byte array of null bytes is supposed to be sent if the server’s challenge includes a TargetInfo with a timestamp. In your case, the server is sending a timestamp and as I mentioned in the email, System.Net.Mail is sending this 24-byte array of null bytes.

To enable it, do the following:

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2IncludeZ24
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

1. According to the docs, it sounds like the NTLMv2 password hashing/encryption logic is supposed to use the timestamp included in the server’s TargetInfo (if it is included; otherwise it should use the current time). Currently, MailKit always uses the current time, ignoring the timestamp sent by the server.

To enable this fix, do the following:

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2UseTargetInfoTimestamp
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

1. Here’s where I want you to test both fixes combined.

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2IncludeZ24 | NtlmFixes.NTLMv2UseTargetInfoTimestamp
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

Let me know if any of the above solutions work.

@alex-jitbit

if (client.AuthenticationMechanisms.Contains ("NTLM")) {
    var ntlm = new SaslMechanismNtlm ("username", "password");
    client.Authenticate (ntlm);
} else {
    // use the default supported mechanisms
    client.Authenticate ("username", "password");
}

@jstedfast I’m sorry, that was wrong alarm about not working SmtpClient on Windows from internet. I wrote about my troubles to the administrator of my smtp-server and he informed me that a connection to smtp-server from internet was denied in server settings. He allowed connection to the smtp-server from my client machine via internet, and my SmtpClient established the connection successfully. So, now I have troubles only connecting to the smtp-server from Linux client machine via NTLM-authentication using 3.0.0. version of MailKit (with your fix at version 3.0.0.271). At the same environment NTLM-authentication using 2.17.0 version of MailKit or other kinds of authentication (I tested LOGIN-authentication) using 3.0.0. version of MailKit works successfully.

Hi, @jstedfast ! I’ve tried your recent fix where you authenticate to the smtp-server using domain supplied in client credentials. And I’m happy to report you that it works in one of my cases when smtp-client is located on windows machine in local network where smtp-server is located. Thanks, I think that’s a good progress in my case!

But unfortunately this fix doesn’t work in case when the smtp-client is on a linux machine (I tested it on Ubuntu 20.04). In this case smtp-server still responds “535: 5.7.3 Authentication unsuccessful”. To say the truth, I have some troubles with server-side certificate (smtp-client thinks that certificate is invalid) and I have to connect to the smtp-server without SSL/TLS (hack ServerCertificateValidationCallback = (s,c,h,e) => true doesn’t help me). But regardless that fact, your fix with local credentials domain applied to version 2.13.0 of MailKIt and original version 2.0.7 without fixes works successfully on the same my client-side code (include client credentials), the same linux machine and the same smtp-server.

I have also debugged https://github.com/jstedfast/MailKit/blob/master/MailKit/Security/Ntlm/NtlmAuthenticateMessage.cs#L59 as you requested and found out that in my case code goes by path

if ((challenge.Flags & NtlmFlags.TargetTypeDomain) != 0) {
    // The server is domain-joined, so the TargetName will be the domain.
    Domain = challenge.TargetName;
} 

In my case challenge.Flags has value Negotiate56 | NegotiateUnicode | RequestTarget | NegotiateNtlm | NegotiateAlwaysSign | TargetTypeDomain | NegotiateExtendedSessionSecurity | NegotiateTargetInfo | NegotiateVersion | Negotiate128. But in fact challenge.TargetName and challenge.TargetInfo?.DomainName are equal.

@jstedfast While we have no such local SMTP server with NTLM support to play around with configurations, but still we can test it using the above branch with few SMTP servers we were having issues already. Will let you know how it goes by early next week.

Thanks @pawanbhabad - I’ll see if I can poke at it some more this weekend.

@pawanbhabad Thanks for testing out these fixes… I’ll see if I can think of some other ideas to try.

@pawanbhabad and @sauravcaptech if you guys can find a way to get me the logs of a successful NTLM authentication to a server where MailKit fails (via NTLM), please open a new bug.

I’m going to close this one because the original issue is resolved (and was really a question rather than a bug).

I’m definitely interested in fixing bugs in NTLM if I can, I just need info that unfortunately I don’t have.

Basically, what I want to do, is to compare what MailKit is sending with what a working implementation is sending in your cases, this way I can hopefully figure out what MailKit is doing wrong.

@pawanbhabad

It looks like you connected to 2 different SMTP servers.

In the case of System.Net.Mail, you used a Sendmail 8.14 server while in MailKit’s case, you used a Microsoft Exchange ESMTP server. In the System.Net.Mail case, you also did not authenticate (well, the System.Net.Mail SmtpClient did not authenticate even if you gave it credentials to use).

I don’t suppose @sauravcaptech or @pawanbhabad can get me a log of some other SMTP client/library successfully authenticating via NTLM so that I can compare what MailKit is sending to what a successful authentication is sending?

I’m stepping thru the debugger comparing @sauravcaptech’s successful vs failed NTLM exchanges and I can’t tell why one works and the others don’t. In all cases, the exact same NTLM flags are used by the server (and MailKit). The only differences (other than usernames and passwords) are the domain names provided by the server (and used in the final authentication request to the server). That’s it.

Hello, I have recently switched to Mailkit 2.5.0.0. Earlier I was using “System.net.mail.SmtpClient”. In case of NTLM authentication, only authentication supported by SMTP relay, client application is working with SmtpClient but with Mailkit it’s not working (error: Authentication unsuccessful), even though the credentials provided are also same and same configuration.

I have tried using code mentioned above,

var ntlm = new SaslMechanismNtlm (“username”, “password”); client.Authenticate (ntlm);

but same error.

i.e. username and password are windows login credentials.

Sharing Protocol logs,

Connected to smtp://pawan_dev.vxcountry.cinemax.com:25/?starttls=when-available S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Thu, 5 Nov 2020 12:37:21 +0530 C: EHLO [10.217.79.102] S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.79.102] S: 250-AUTH NTLM S: 250-TURN S: 250-SIZE 2097152 S: 250-ETRN S: 250-PIPELINING S: 250-DSN S: 250-ENHANCEDSTATUSCODES S: 250-8bitmime S: 250-BINARYMIME S: 250-CHUNKING S: 250-VRFY S: 250 OK C: AUTH NTLM TlRMTVNTUAABAAAABxIAAAkACQAgAAAAAAAAACAAAABDT01NVU5JVFk= S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECMR2TRr16f68AAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAq475QQrPWAQAAAAA= C: TlRMTVNTUAADAAAAAAAAAG4AAAD2APYAbgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAABkAQAAAQIAAgoAWikAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZADczddHWVbSM1KTMjMJgPsWAQEAAAAAAADrAHxqcLPWAa0J73JjwXLDAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAq475QQrPWAQAAAAA= S: 535 5.7.3 Authentication unsuccessful

Does Mailkit supports NTLM authentication ?

Please suggest me some work-around for this issue.

Thanks!