PowerShell: The last cell of an empty column read by `ConvertFrom-Csv` is inconsistently `$Null`

If the last cell of an unquoted Csv file is empty, the cmdlets ConvertFrom-Csv and Import-Csv inconsistently returns a $Null rather than am empty string.

Steps to reproduce

$Data = ConvertFrom-Csv @'
Id,Name,Note
01,John,
02,Jack,
03,Ryan,Just a note
04,Luke,
05,Noah,
'@

$Data.Note.ForEach{ if ($Null -eq $_) { 'Null' } else { 'String' } }

Expected behavior

String
String
String
String
String

Actual behavior

String
String
String
String
Null

As shown above all other cells in the above Note column are interpreted as string, except for the last cell with appears to be $Null. This would have been correct if the last comma was omitted but that is not the case.

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.2.5
PSEdition                      Core
GitCommitId                    7.2.5
OS                             Microsoft Windows 10.0.22000
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

See also: how to filter empty values in a column in powershell

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 21 (6 by maintainers)

Most upvoted comments

@iRon7 These are obvious code snippets. The right question is whether there are popular applications or services that create such corrupted files that could lead to code like yours. I don’t know of any. And I can’t think of a reason why they would be created. From this I conclude that we should get rid of nulls.

I tend to think that nulls make no sense at all for CSV.

$a = @{q = $null}
$a | convertTo-Csv
"q"

ConvertTo-Csv: Object reference not set to an instance of an object.

It is a bug we must fix. But no null literals are in CVS format. So we have to output empty string. Then we could conclude it makes no sense to designate nulls and empties on read too. This looks more consistent.

Not valid CSV?

$Data = ConvertFrom-Csv @'
Id,Name,Note
01,John
02,Jack
03,Ryan,Just a note
04,Luke
05,Noah
'@

$Data.Note.ForEach{ $_ ? 'String' : 'Null' }

<#
Null
Null
String
Null
Null
#>

Still reproduces in: PS>$PSVersionTable

Name Value


PSVersion 7.3.9 PSEdition Core GitCommitId 7.3.9 OS Microsoft Windows 10.0.22621 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0

@dkaszews Thanks! It is not a scenario we should care.

@iSazonov From original example, it should be String, Null, String, String - default value if there is a comma but value is empty, null if there are not enough commas so some columns are completely missing. We are focusing on the last column, but this should apply to every one:

$Data = ConvertFrom-Csv @'
Id,Note,Name
01,,John
02,
03,Just a note,Ryan
04,,Luke
'@

The way I see it, first you split on the delimiter, which gives you array of potentially empty values, then pad with nulls to correct length. With the examples present, it somehow also depends on the very presence of another row after, so reordering rows or truncating the table can change types.