google-cloud-go: iot: Unable to list all devices in a registry

Client

IoTCore

Environment

MacOS

Go Environment

$ go version - go1.18.3 darwin/amd64

Code

e.g.

package main

func fetchAllDevices(ctx context.Context, client *gcpiotcore.DeviceManagerClient, parent string) ([]*gcpiotpb.Device) {

         var fields = &fieldmaskpb.FieldMask{
	        Paths: []string{
		        "id",
		        "name",
		        "credentials",
		        "last_heartbeat_time",
		        "last_event_time",
		        "last_state_time",
		        "last_config_ack_time",
		        "last_config_send_time",
		        "blocked",
		        "last_error_time",
		        "last_error_status",
		        "config",
		        "state",
		        "log_level",
		        "metadata",
		        "gateway_config"},
        }

	req := &gcpiotpb.ListDevicesRequest{
		Parent:       parent,
		FieldMask:    fields,
	}

	var devices []*gcpiotpb.Device

	it := client.ListDevices(ctx, req)

	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}

		if err != nil {
			log.Fatalln("Unable to iterate over device records: ", err)
		}

		devices = append(devices, resp)
	}

	defer client.Close()
	fmt.Println(string(colorGreen), "\u2713 Fetched", len(devices), "devices!", string(colorReset))
	return devices
}

Expected behavior

ListDevices should return all the devices in the registry (22766 devices in my case).

Actual behavior

ListDevices returns only 10966 out of 22766 devices. I have another registry where I have more than 200k devices, but the function only returns 19,129 devices.

Additional context

Tried logging response returned by it.Next() to see if it has a NextPageToken value, but at the end of all the iterations, it always returns nil.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (10 by maintainers)

Most upvoted comments

We started seeing this issue a couple of weeks back.

This is how the parent is being constructed - parent := "projects/" + getProjectID(val) + "/locations/" + Args.gcpRegistryRegion + "/registries/" + Args.registryName

getProjectID(val) -> function that extracts project ID from the gcp service account file Args.gcpRegistryRegion & Args.registryName -> cli flags