machinelearning: F# Records not compatible with ML .NET

Perhaps related to #92. Apparently ML .NET doesn’t work with F# records! Is this perhaps to do with the fact that F# Records use a different naming convention to Classes for backing fields, and the ML .NET Library. If you use records, the library simply can’t find the columns and complains of missing columns. If you port from records to classes with mutable properties, it doesn’t fail with that error any longer.

However, mutable classes are pretty much non-idiomatic in F# - no one uses them for the sorts of data-bound workloads that you’ll see with ML.

Instead of this:

type SentimentData =
    { [<Column(ordinal = "0")>] SentimentText : string
      [<Column(ordinal = "1", name = "Label")>] Sentiment : float }

[<CLIMutable>]
type SentimentPrediction =
    { [<ColumnName "PredictedLabel">] Sentiment : bool }

You’ll have to use something like this monstrosity.

type SentimentData() =
    [<Column(ordinal = "0"); DefaultValue>]
    val mutable SentimentText : string
    [<Column(ordinal = "1", name = "Label"); DefaultValue>]
    val mutable Sentiment : float32

type SentimentPrediction() =
    [<ColumnName "PredictedLabel"; DefaultValue>]
    val mutable Sentiment : bool

You can’t even use the member val shorthand syntax that F# provides for mutable get / set properties since the ColumnName attribute doesn’t work with them. Plus, you lose all the standard features that Records bring such as lightweight syntax, easy creation, copy-and-update, immutability and structural equality.

I strongly recommend adding support for them by ensuring that whatever internal hydration logic that is currently coupled to classes supports records as well.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 11
Comments: 23 (10 by maintainers)

Most upvoted comments

I would hope that the library doesn’t modify data that you supply to it! 😃

isaacabraham on Jun 7, 2018

Assuming F# records are just producing get/set properties it’s not even just a problem for F# — public fields aren’t canonical C# code either. We gave the same feedback in the API review but it didn’t make the cut yet.

Presumably #254 would fix that then.

terrajobst on Jun 6, 2018

It’s been a while since I looked at this but can you now actually access ML net without being coupled to the data access layer? If so, that would ideally remove all this stuff and we could just worry about that ourselves.

isaacabraham on Feb 3, 2019

The “backing fields” bit is an assumption on my part - normally [<CLIMutable>] does the trick for libraries that want to use a default constructor and then set the properties using e.g. reflection, but it doesn’t work here. So I assume that it’s using some field-based reflection.

Having said that, if we want a truly F#-friendly API then even [<CLIMutable>] shouldn’t be required.

isaacabraham on May 17, 2018

@isaacabraham Some notes on immutable F# records:

For immutable records, a schema mapper or serialization framework really has no choice but to work with the “internal” representation of the record, i.e. the hidden, private fields. This is what binary serialization does, for example. This is because there are no setter properties on those immutable records, so there’s really no other choice: the IDataView schema mapper works by setting fields one at a time on target objects and there is no way we could convert it to use the immutable object constructor for example.

However inputs to ML.NET currently uses field/property names in string form, e.g. "SentimentText" here:

    pipeline.Add(
        TextFeaturizer(
            "Features", [| "SentimentText" |],...

So, if the ML.NET IDataView schema mapper is dealing with internal fields (e.g. private field SentimentText@ used by F#), then it would also have to know about the relationship between internal and external names, i.e.

public property "SentimentText" <-> private field "SentimentText@"

It’s possible but F# emits no specific metadata about this relationship, so the rule would have to be F#-specific. It’s a stretch but not impossible. However would require more subtle coding than I’m willing to try for now, so CLIMutable is a reasonable stepping point.

dsyme on Jul 31, 2018

Any chance of getting ctor support as well to avoid the need for CLIMutable?

@isaacabraham I’m not sure. First things first.

dsyme on Jul 30, 2018