google-cloud-go: datastore: embedded anonymous struct regression?

This issue copied from stackoverflow. Original issue here: http://stackoverflow.com/questions/42659697/google-datastore-breaking-change-re-anonymous-struct-fields Original reporter: @ianrose14

Report body:

We have a struct definition like this:

type TimeMs struct {
    time.Time
}

And various of our datastore entity definitions include fields of this type, e.g.:

type Whatever struct {
    Created TimeMs
}

In the past, this always worked fine without any special hackery on our part. If you examine these entities in the cloud console, the field’s name was “Created.” (note the trailing dot) which was a little odd but didn’t seem to cause any problems. However, the latest versions of the cloud datastore package don’t seem to play nicely with this. When we attempt to do a client.Put, we get an error of:

unexpected error: rpc error: code = 3 desc = The flattened property.name is the empty string.

We also saw a recent commit of “if nested struct implements PLS, use it” which seemed potentially useful. So we tried making the TimeMs struct implement PropertyLoadSaver (saving its anonymously-embedded time.Time as a property with Name=“”), but this results in the same error.

It sure seems like this is a regression. Am I missing something? Is there some kind of workaround that we haven’t thought of?

Please note that “just make it implement PropertyLoadSaver and save the Time with some non-empty Name” isn’t really viable since we already have (literally) millions of entities stored using “” so it doesn’t seem viable to migrate all of them.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 18 (7 by maintainers)

Most upvoted comments

closing, as we’re resolving these issues in #552, #551 please reopen if this is incorrect.

s-mang on Mar 17, 2017

ah, i see what you’re saying. will make a separate issue.

s-mang on Mar 10, 2017

Ah! Yes. So we made this change consciously, before the beta release. The dot at the end of field names was a mistake carried over from the old appengine datastore code.

I’m so sorry for the inconvenience.

I’d suggest writing a script, that loops through your entities and fixes the names to eliminate the “.” suffix, something like this (below). Alternatively, you could bake this behavior into your code.

package main

import (
	"context"
	"fmt"
	"strings"
	"time"

	"cloud.google.com/go/datastore"
)

type TimeMs struct {
	time.Time
}

type Whatever struct {
	Created TimeMs
}

func (t *Whatever) Load(props []datastore.Property) error {
	for i, p := range props {
		if strings.HasSuffix(p.Name, ".") {
			props[i].Name = strings.TrimSuffix(p.Name, ".")
		}
	}

	return datastore.LoadStruct(t, props)
}

func (t *Whatever) Save() ([]datastore.Property, error) {
	return datastore.SaveStruct(t)
}

func main() {
	ctx := context.Background()
	dc, err := datastore.NewClient(ctx, "shadams-dev-testing")
	if err != nil {
		panic(err)
	}

	k := datastore.NameKey("Entity", "stringKey", nil)

	e := new(Whatever)
	if err := dc.Get(ctx, k, e); err != nil {
		panic(err)
	}

	if _, err := dc.Put(ctx, k, e); err != nil {
		panic(err)
	}
}

LMK if this doesn’t help, or if there’s anything else I can do.

So sorry again for the inconvenience. Now that the package is in beta, I don’t anticipate that we’ll make any breaking changes like this again. Once it’s GA (soon), we guarantee that we won’t.

s-mang on Mar 8, 2017