PowerShell: System.OutOfMemoryException thrown by Out-String in PS 7.4

Prerequisites

Write a descriptive title.
Make sure you are able to repro it on the latest released version
Search the existing issues.
Refer to the FAQ.
Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

I am experiencing this strange issue with Out-String in PS 7.4 giving System.OutOfMemoryException in certain situations. I think it is the size of the object I am inputting for the cmdlet. I will try to provide steps for reproduction but it is hard as I can reproduce this on Azure Pipelines Hosted agents which run on containers. The issue appeared when we upgraded PS on the hosted agents from 7.2 to 7.4. Same code with same input was working fine before that. The code we run is the following:

$whatIfResult = Get-AzDeploymentWhatIfResult @deploymentParameters
($whatIfResult | Out-String).replace("$e[38;5;208m", "$($PSStyle.Foreground.Red)").replace("$e[38;5;77m" , "$($PSStyle.Foreground.Green)").Replace("$e[38;5;141m", "$($PSStyle.Foreground.Magenta)").Replace("$e[38;5;246m", "$($PSStyle.Foreground.BrightBlack)")

I have removed the replace part and still getting error: Error: Exception of type ‘System.OutOfMemoryException’ was thrown. The error is not thrown every time if the results in variable whatIfResult are not big everything works. But as soon as the variable has some big results the error appears. If needed I can share more details but it needs to be in private as the parameters for cmdlet Get-AzDeploymentWhatIfResult are something I cannot share in public. I can also reproduce it with a file that I have downloaded from the pipeline logs that is in size 1.57 GBs. When I reproduce it I run it on my laptop with may more memory and it is on Windows but also PS 7.4. If I do Get-Content <file name> | Out-String I get the same error I am hoping that this was working fine in PS 7.2 can help you find what in the code of PS 7.4 for Out-String is causing it. I am not sure if it somehow depended on the Memory where PS 7.4 runs but as I have mentioned same machines with same capacity did not had the problem with 7.2.

Expected behavior

Out-String to succeed.

Actual behavior

Error: Exception of type 'System.OutOfMemoryException' was thrown.

Error details

CategoryInfo          : NotSpecified: (:) [Out-String], OutOfMemoryException
FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.OutStringCommand
InvocationInfo        :
    MyCommand        : Out-String
    ScriptLineNumber : 1
    OffsetInLine     : 91
    HistoryId        : 13
    Line             : Get-Content "C:\Users\stanislav.zhelyazkov\Downloads\logs_200053\1_Deploy solution.txt" | out-st
ring
    Statement        : out-string
    PositionMessage  : At line:1 char:91
                       + … .zhelyazkov\Downloads\logs_200053\1_Deploy solution.txt" | out-string
                       +                                                              ~~~~~~~~~~
    InvocationName   : out-string
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.0
PSEdition                      Core
GitCommitId                    7.4.0
OS                             Ubuntu 20.04.6 LTS
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

About this issue

Original URL
State: open
Created 7 months ago
Comments: 28

Most upvoted comments

@slavizh This is a community project, and no one (including MSFT members) has an obligation to others. Even if you open a case in official MSFT support, they will ask you to do additional steps (and most likely you will have to pay for their work if it is not a bug).

There was one change in pipeline so that could be the source of the problem. /cc @SeeminglyScience

iSazonov on Dec 13, 2023

@iSazonov really? How about someone from the PS team troubleshoot this instead. If I do not get support here I will log official case anyway.

slavizh on Dec 13, 2023

Someone else might now better but I think

$x.replace( ).replace( ).replace()

Won’t dispose of the first replace until last replace has run. So the code as shown has one copy of the variable in a large object, another copy in ($object | out-string) and multiple copies at each of the replacement steps

I would try

Get-AzDeploymentWhatIfResult @deploymentParameters | Out-String -stream | foreach-object {$_ -replace '\$e\[38;5;208m' $PSStyle.Foreground.Red -replace "\$e\[38;5;77m" , $PSStyle.Foreground.Green  }

I’m assuming the azure command returns multiple objects, this way each one goes into Out-String , is output as multiple small strings (that’s -stream) and each of those is processed . (Not that -replace uses regular expressions so $ and [ need to be escaped with \

Possibly something has changed in .NET rather than what the PowerShell commands do, possibly some other change means you have less memory available or the objects are generating bigger output. But making enough copies of a big enough string will cause you to run out of memory. (and as @rhubarb-geek-nz has said, if you read a 1GB ascii text or UTF 8 encoded file it becomes 2GB of UTF-16 in memory. )

jhoneill on Dec 12, 2023