amplify-js: DataStore does not update data when @Auth directive used with multiple owners

Summary

At the 11th hour in our app development we have just discovered that after following the official AWS documentation use cases of adding @Auth directives on my GraphQL schema in order to allow for read-only access to a selection of users, and also built it using DataStore, which is promoted as best practice for mobile app development, the two are not compatible with each other.

Example

Here’s a simplified view of the schema and the use case:

type WorkerTimesheet
  @model
  @auth(
    rules: [
      { allow: owner, ownerField: "workerCognitoUserId" }
      { allow: owner, ownerField: "approverCognitoUserIds", operations: [read] }
    ]
  ) {
  id: ID!
  workerCognitoUserId: ID!
  approverCognitoUserIds: [ID!]
  ...
}

Details

In a pure GraphQL subscription, taking DataStore out of the equation, I can only get a subscription to fire when a WorkerTimesheet is updated, if I subscribe with a matching workerCognitoUserId field as follows:

subscription onWorkerTimesheetUpdated {
  onUpdateWorkerTimesheet(workerCognitoUserId: "owner-cognito-user-id") {
    id
    workerCognitoUserId
    approverCognitoUserIds
  }
}

That fires when the workerCognitoUserId parameter matches the owner. But how to inform the approvers?

Well this is the GraphQL subscription automatically generated by Amplify:

onUpdateWorkerTimesheet(workerCognitoUserId: String, approverCognitoUserIds: String): WorkerTimesheet @aws_subscribe(mutations: ["updateWorkerTimesheet"]) @aws_iam @aws_cognito_user_pools

So perhaps I can specify approverCognitoUserIds parameter - it’s a String so that’s promising:

subscription onWorkerTimesheetUpdated {
  onUpdateWorkerTimesheet(approverCognitoUserIds: "approver-cognito-user-id") {
    id
    workerCognitoUserId
    approverCognitoUserIds
  }
}

This doesn’t work, apparently due to quite a large limitation in how AppSync subscriptions work with array fields that I’ve since found in my increasingly desperate research has been around for many years. This begs the question how @Auth directives using multiple owners were ever anticpated would work, since subscriptions are pretty fundamental to AppSync and GraphQL, and absolutely critical for DataStore.

What is confusing to me is why the generated subscription has a approverCognitoUserIds: String field in the first place if it’s never possible to satisfy it given the targeted field is an array. Must be a bug, but this functionality has been available in Amplify for years, without exageration.

This is made much worse by the fact that turning on Amplify debug logging shows that DataStore is using that field for its subscription:

[DEBUG] 47:34.112 AWSAppSyncRealTimeProvider - subscription ready for
 {"query":"subscription operation($approverCognitoUserIds: String!) {
onUpdateWorkerTimesheet(approverCognitoUserIds: $approverCognitoUserIds) ...

And so even for WorkerTimesheets that are created with a matching workerCognitoUserId, DataStore never gets any updates for that type.

The only way it shows up is if we force it through a DataStore.stop() followed by a DataStore.start() and then it pulls the data for the matching workerCognitoUserId as well as for any user logged in that appears in the approverCognitoUserIds list. But that’s because of the initial load of the data rather than through any subscription.

If this is all correct it basically makes using the @Auth directive with a list of authorized users incompatible with using DataStore and yet I never read about any such limitation. We are at the 11th hour in our app development and it looks like we have a fundamental design change required. This is not something to be treated lightly since it’s going to affect both frontend and backend teams who have integrated around this design.

I don’t understand how this sort design limitation has been tolerated for so long and viewed as an acceptable user experience by the Amplify + AppSync teams. At the very least there should be clear documentation about these limitations to avoid wasting dozens of hours on implementations that won’t work. But even that wouldn’t really be good enough for something like this. If DataStore is not compatible with @Auth directives like this, it should fail fast, perhaps on DataStore.start() and not just silently never update its DataStore. Honestly, it makes a mockery of the first-class experience Amplify and DataStore is intending to provide.

This report may perhaps sound a bit overblown, but we have kept running into fairly serious impediments as we’ve wholeheartedly attempted to embrace both Amplify and AppSync in our latest application. I have refused to believe that a roll-your-own, undifferentiated heavy-lifting approach is better, and actively promoted using higher-level services such as AppSync and libraries such as DataStore, but after several years of them being available and these sorts of issues largely ignored/overlooked/deprioritized, I am struggling to reconcile my beliefs with my experiences unfortunately.

References to other related issues:

This scenario I’m documenting here is far from an edge case.

References of other issues we encountered in the very recent past:

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 11
  • Comments: 20 (3 by maintainers)

Most upvoted comments

The solution we implemented is a standard replication pattern where for our Timesheet scenario a worker would create a timesheet and own it themselves, and then a backend lambda would subscribe to the DynamoDB stream determine a Timesheet has been created and then identify the appropriate approvers and then add duplicate/replica records for each approver so that each record has one owner: a master record owned by a worker and 1 or more replica records owned by each approver.

Sounds great, but the amount of backend logic required to handle controlled 2 way replication of updates was not trivial. Especially because we want to avoid feedback loops, but still have an update made in one replica record by an approver update all the other replica records (as well as master) for each approver.

And we haven’t even tackled the complexity of optimistic locking to handle scenarios that 2 approvers edit their own replicas at the same time. You see, when the burden is put on the application developer to solve these problems, it quickly sucks so much of the productivity gains from using the tooling initially.


We use an Event Driven Architecture on the backend so that simplified being notified of when a Timesheet was created/updated/deleted (but that’s a whole another level of undifferentiated code we had to write to achieve that - aws-amplify/amplify-category-api#829).

So the irony of all this is in order to use Amplify, and DataStore so that we simplify frontend programming patterns (which it largely has) we’ve had to create a lot more complexity on the backend in order to architect the backend in a scalable, flexible and maintainable fashion in its own right and not just create a big ball of mud there.

I would love our team to be able to avoid implementing so much undifferentiated code on the backend especially, and I think it would be very achievable by the AWS teams if that was a serious goal. There’s lots of talks about EDA and its advantages, but I see little evidence of that from AWS’ offerings (e.g. why does EventBridge talk about 500ms latencies for event delivery, why doesn’t it support ordered event delivery, why isn’t there a first class event store service for supporting event sourced architectures etc)

Anyway massive tangent, but needless to say this issue we ran into on this ticket is but one of many issues we’ve had to solve with undifferentiated solutions that I would seriously love AWS to solve for us as you guys are way more capable of doing so than us.

Hi @iartemiev

Exactly same here, we want to allow CRUD operations authorised based on dynamic and static groups to enable collaboration among invited members to a group and admins Our approach for now is to add the below to the shareable models, then at start-up fetch which data the user should have access to (if online otherwise bummer) and then set syncExpressions on each sharable model

schema.graphql

///

@model
  @auth(
    rules: [
      { allow: private }
    ]
  )

///

After hours trying to solve the privacy component here which is a huge concern, we came to your very same conclusion on the subscription For the aws team, quite frankly in this thread we have been talking of a very common use case and we would really appreciate some assistance. At this point in time, for my team use case, sadly, Datastore is not production ready

Thanks!

Same for me. In my case, I’ve detected that DataStore successfully subscribe to changes if the current user belongs to the “last” auth rule (so I had to swap mine due to priorities. Let me explain. This is my model:

type Appointment
  @model
  @auth(
    rules: [
      { allow: groups, groups: ["Admins"] }
      { allow: owner, ownerField: "practitionerId", operations: [read] }
      { allow: owner, ownerField: "userId", operations: [read] }
    ]
  ) {
  id: ID!
  datetime: AWSDateTime!
  status: AppointmentStatus!
  userId: ID!
  user: User @belongsTo(fields: ["userId"])
  practitionerId: ID!
  practitioner: Practitioner! @belongsTo(fields: ["practitionerId"])
}

In this case, my user “practitioner” received live updates from DataStore subscription, but not my user “user”.

Since in my app, users are the ones making the appointments (and therefore I want to give them live feedback on screen), I changes the last four lines as:

  practitionerId: ID!
  practitioner: Practitioner! @belongsTo(fields: ["practitionerId"])
  userId: ID!
  user: User @belongsTo(fields: ["userId"])

So my user “user” is the one receiving live updates. For now, the user “practitioner” will have to “refresh” the page, but tbh I don’t expect them to be online waiting for appointments so I can live with this. Next time they visit there’ll be a full sync and they will get the data anyway.

Maybe this workaround helps someone.