PowerShell: The last cell of an empty column read by `ConvertFrom-Csv` is inconsistently `$Null`
If the last cell of an unquoted Csv
file is empty, the cmdlets ConvertFrom-Csv
and Import-Csv
inconsistently returns a $Null
rather than am empty string.
Steps to reproduce
$Data = ConvertFrom-Csv @'
Id,Name,Note
01,John,
02,Jack,
03,Ryan,Just a note
04,Luke,
05,Noah,
'@
$Data.Note.ForEach{ if ($Null -eq $_) { 'Null' } else { 'String' } }
Expected behavior
String
String
String
String
String
Actual behavior
String
String
String
String
Null
As shown above all other cells in the above Note
column are interpreted as string, except for the last cell with appears to be $Null
. This would have been correct if the last comma was omitted but that is not the case.
Error details
No response
Environment data
Name Value
---- -----
PSVersion 7.2.5
PSEdition Core
GitCommitId 7.2.5
OS Microsoft Windows 10.0.22000
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
See also: how to filter empty values in a column in powershell
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 21 (6 by maintainers)
@iRon7 These are obvious code snippets. The right question is whether there are popular applications or services that create such corrupted files that could lead to code like yours. I don’t know of any. And I can’t think of a reason why they would be created. From this I conclude that we should get rid of nulls.
I tend to think that nulls make no sense at all for CSV.
It is a bug we must fix. But no null literals are in CVS format. So we have to output empty string. Then we could conclude it makes no sense to designate nulls and empties on read too. This looks more consistent.
Not valid CSV?
Still reproduces in: PS>$PSVersionTable
Name Value
PSVersion 7.3.9 PSEdition Core GitCommitId 7.3.9 OS Microsoft Windows 10.0.22621 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0
@dkaszews Thanks! It is not a scenario we should care.
@iSazonov From original example, it should be
String, Null, String, String
- default value if there is a comma but value is empty, null if there are not enough commas so some columns are completely missing. We are focusing on the last column, but this should apply to every one:The way I see it, first you split on the delimiter, which gives you array of potentially empty values, then pad with nulls to correct length. With the examples present, it somehow also depends on the very presence of another row after, so reordering rows or truncating the table can change types.