PowerShell: Get-ADUser throws "the encryption operation failed" on latest PowerShell 7.4-preview.5 daily build

Prerequisites

Steps to reproduce

This fails on PowerShell 7.4.0-preview.5 daily build:

Get-ADUser -Filter {SamAccountName -like "*test*"}

(This works on PowerShell 7.3.6 and PowerShell 5.1, and the AD sever is NOT down, contrary to the error message)

Expected behavior

Should return requested AD user objects

Actual behavior

Throws an exception "the encryption operation failed"

Error details

See visuals

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.0-preview.5
PSEdition                      Core
GitCommitId                    7.4.0-preview.5-19-gab5213e4b6ea4ac3ba0b84378ed1f31f6e14619c
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0, 5.0, 5.1, 6.0, 7.0}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

image

ad_error_2

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 1
  • Comments: 21 (2 by maintainers)

Most upvoted comments

Would you mind sharing how did you trace the bug? With PerfView or something else? I would love to read a blog post about it 😉

I was able to replicate the error myself and saw the exception message in the inner stacktrace being

The encryption operation failed, see inner exception

I then did a search in the github runtime repository and found it was stored as net_io_encrypt. This string was used in two places but the stacktrace showed it was from NegotiateStream and it brought me to this block of code. I knew it wasn’t affecting NTLM as the -Server $ipAddr was working so I knew it was calling Encrypt which in turn I know calls the Win32 call EncryptMessage. Doing a git blame I saw the PR mentioned which made a few changes so that’s the first place I assumed was the culprit.

As for the actual traces for EncryptMessage. It was through a special tool I’ve written PSDetour which can hook C APIs and have it run PowerShell code instead. I have a few known APIs I’ve hooked for tracing purposes in the past at PSDetour-Hooks and this includes one for EncryptMessage. Using it is as simple as installing the deps and then using the Trace-Process.ps1 script in that repository:

# AllUsers is needed if you are tracing a process running as another user
Install-Module -Name PSDetour, Yayaml -Scope AllUsers

# Press ctrl+c when done to stop the trace
$traceParams = @{
    Metadata = @{
        Secur32 = 'EncryptMessage'
    }
    ProcessId = $targetProcId
    OutputFormat = 'Yaml'
}
./Trace-Process @traceParams  | Tee-Object -FilePath "C:\something.log"

There can be a lot of output depending on what you are tracing which is why I like to tee the output to a file. You can even use this to target non-PowerShell processes (like lsass!) but it is quite invasive so not something I recommend for production purposes and it won’t work for .NET Framework based processes (like Windows PowerShell 5.1).

Thanks for testing it! I’ll submit a PR later today.

Can confirm the changes in your dll now work

screenshot_2023-08-26-170144

Looking at the raw SSPI calls I now see

- Function: Secur32.dll!EncryptMessage
  Time: 2023-08-26T16:59:53.1049824+10:00
  ThreadId: 7988
  Arguments:
    Context: 0x1EDB4038420 - PCtxtHandle
    Qop: 0
    Message:
      Raw: 0xBFCF5FE860 - PSecBufferDesc
      Version: 0
      Count: 3
      BufferPtr: 0xBFCF5FE6B0 - PSecBuffer
      Buffers:
      - Type: 0x00000002 - SECBUFFER_TOKEN
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 76
        Raw: 0x1EDB403A8E8 - Pointer
        Data: "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
      - Type: 0x00000001 - SECBUFFER_DATA
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 1
        Raw: 0x1EDB403A934 - Pointer
        Data: 0C
      - Type: 0x00000009 - SECBUFFER_PADDING
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 1
        Raw: 0x1EDB403A935 - Pointer
        Data: "00"
    MessageSeqNo: 0
- Function: Secur32.dll!EncryptMessage
  Time: 2023-08-26T16:59:53.1337551+10:00
  ThreadId: 7988
  Result: 0
  Info:
    Message:
      Raw: 0xBFCF5FE860 - PSecBufferDesc
      Version: 0
      Count: 3
      BufferPtr: 0xBFCF5FE6B0 - PSecBuffer
      Buffers:
      - Type: 0x00000002 - SECBUFFER_TOKEN
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 60
        Raw: 0x1EDB403A8E8 - Pointer
        Data: 050406FF0000001C00000000680D287489943F5DB7574A80DE16FDBDD535058A21F70CA60CA284A1508723773A512CF961DC1AB2FDCEBE14135D9EA0
      - Type: 0x00000001 - SECBUFFER_DATA
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 1
        Raw: 0x1EDB403A934 - Pointer
        Data: ED
      - Type: 0x00000009 - SECBUFFER_PADDING
        Flags: 0x00000000 - SECBUFFER_NONE
        Size: 0
        Raw: 0x1EDB403A935 - Pointer
        Data: ""

This lines up with what I saw previously in net7 so the changes seem good.

Thanks for the quick turnaround, love your work!

Should this be added to the preview 5 and 6 release notes as a known issue or breaking change? That’s the first place many folks I know look when run in to issues… and would help others who routinely perform Get-ADxx commands to know if they should avoid using these versions.

Just an FYI while preview 6 of pwsh is based on rc1, the fix for this issue is not in rc1. The backport commit which fixed the problem should be in rc2+ https://github.com/dotnet/runtime/commit/217be6c5b81c38bcfdd8d014a3a0b4412b6c2c1c when it’s released.

image

@jborean93 Thanks! This is a great explanation, and another tool in my toolkit.

It’s not working for me either, different error tho but it confirms it is clearly a preview issue.

image

Checked in 2 different Domains, same error.