dynamoose: Type `Array` or `Object` not working even with useDocumentTypes

Hey guys, hope you’ve been doing well.

First of all, thank you for this awesome SDK, is amazing to have something like this to work with Dynamo.

I’m having an issue to create my tables that I do not believe that is related with configuration, so I bring it to you guys.

When I try to create a table using type Array or Object it’s keeping create the field with String and JSON.stringify on it’s content.

This is my Schema:

const schema = new dynamoose.Schema({
  ScheduledID: {
    type: String,
    required: true,
    hashKey: true,
  },
  ScheduledTime: {
    type: String,
    required: true,
    rangeKey: true,
    index: {
      global: true,
      name: 'ScheduledTimeIndex',
      project: true,
      throughput: 2,
    },
  },
  CustomerID: {
    type: String,
    required: true,
  },
  SubmissionID: {
    type: String,
    required: true,
  },
  SubmissionTimeout: {
    type: String,
    required: true,
  },
  CustomFields: {
    type: Object,
  },
  Channels: {
    type: Array,
    required: true,
  },
  InteractionID: {
    type: String,
    required: true,
  },
  RetryRule: {
    type: Array,
  },
  StepSequenceID: {
    type: String,
    required: true,
  },
  StepChannel: {
    type: String,
    required: true,
  },
  TTSVoiceType: {
    type: String,
    required: true,
  },
  TTSService: {
    type: String,
    required: true,
  },
  TTSLanguage: {
    type: String,
    required: true,
  },
  ClientContactData: {
    type: String,
    required: true,
  },
  Priority: {
    type: Boolean,
    default: false,
    required: true,
  },
}, {
  throughput: {
    read: 2,
    write: 2,
  },
  create: true,
  update: true,
  useNativeBooleans: true,
  useDocumentTypes: true,
  saveUnknown: true,
});

But when I create a new item it keeps creating Array and Objects as Strings. Let’s take RetryRule as an example:

  RetryRule: {
    type: Array,
  },

If I save it like this:

RetryRule: [
        {
            retry: 1,
            time: 15
        },
        {
            retry: 2,
            time: 120
        }
    ],

This is how it got saved on Dynamo:

"RetryRule": {
  "S": "[{\"retry\":1,\"time\":15},{\"retry\":2,\"time\":120}]"
},

I believe that there is nothing wrong with my schema. But, if it is, I apologize for the inconvenience and will be glad if you guys guide me through the right way.

Regards.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 4
  • Comments: 20 (7 by maintainers)

Commits related to this issue

Most upvoted comments

@leanderr That’s because you have useDocumentTypes: false. As per the doc, the default type for arrays and objects is ‘S’ unless useDocumentTypes: true.

https://dynamoosejs.com/api#attribute-types

So, I spent a little more time investigating this and this is what I’ve found:

You can actually provide the strings 'list' or 'map' for the attribute type, and Dynamoose will already do what my fork ultimately tries to do and what I believe you all are trying to do. There’s a weird extra (as far as I can tell undocumented) bit that needs to be done whether you’re working with the main repo or my fork, however: you must provide a corresponding list or map KEY on the attribute, which must contain a sort of mini-schema for what the inside of the given list or map should look like. And the weirdest part is that when you define the list key, it must be a single item array containing the type of its elements. Here are a few example attributes that all assume useDocumentTypes is set to true:

Array of Strings saved as a native DynamoDB List:

thing: {
  type: 'list',
  list: [String],
},

Object with three arbitrary String keys saved as a native DynamoDB Map:

thing: {
  type: 'map',
  map: {
    text1: String,
    text2: String,
    text3: String,
  },
},

Array with the above Object as its elements saved as a native DynamoDB List with native DynamoDB Map elements:

thing: {
  type: 'list',
  list: [
    {
      text1: String,
      text2: String,
      text3: String,
    },
  ],
},

My fork currently achieves this using Array and Object directly, but I could see it being a breaking change for anyone who unwittingly left useDocumentTypes on and currently has tables full of stringified Arrays and Objects, so I don’t think it would be a good idea to merge it in (at this time at least). A better solution would be to simply update the documentation to reflect the above details, unless the explanations are already there and we’ve simply missed them.

Just to be more clear, I’ll put also an example that don’t mix types (like on RetryRule that I have an array of objects).

Using CustomFields:

CustomFields: {
  type: Object,
},

Saving with this code:

CustomFields: {
  name: 'test1',
  lastname: 'test2',  
},

And getting:

"CustomFields": {
  "S": "{\"name\":\"test1\",\"lastname\":\"test2\"}"
},

@leanderr You said:

Also Using Combinations of ‘map’, ‘Map’, Map, ‘obj’, ‘Obj’, Object’

I assume that means your table looks something like the following:

[
  {
    address1: {
       S: '{firstname: "Test", lastname: "User"}'
    }
  },
  {
    address1: {
      M: {
        firstname: {S: "Other"},
        lastname: {S: "User"}
      }
    }
  }
]

(hopefully I got my syntax and everything correct)


^ that is not allowed in Dynamoose. You can’t have one document property be a string, and another be a map. In Dynamoose currently, a map is different than an object. That is something I find very confusing and took me a while to wrap my head around. I think we should consider changing that in the future (it would be a breaking release), but that is WAY long term I think, and not something I’m prioritizing now. The priority now is updating the documentation. If someone wants to take on a rewrite of this system, I’d be very open to that.

In the example above the first option is an object. And the second is a map.


Hopefully that makes sense.


Do you mean you cannot have an Object that has attributes of e.g. String and NUmber?

If I understand correctly, you can.

Are there any working examples with Maps or Arrays of Maps at all?

Yes. In my previous comment I linked you to a test that we are using to test this functionality.

@leanderr See https://github.com/dynamoosejs/dynamoose/issues/230#issuecomment-361412970 and the following test:

https://github.com/dynamoosejs/dynamoose/blob/e4e0eeb4e3fdee80f02d41306ed92f952510cf02/test/Schema.js#L82-L168


It’s really hard to read your code due to the lack of good formatting, but from what I can gather you need to change your schema to the following:

var testSchema = new Schema({
  address1: {
    type: 'map',
    map: {
      firstName: String,
      lastName: String
    }
  }
}, {
    useDocumentTypes: true
});

I have worked with this before with no issues at all. If you are still running into problems feel free to create a PR with a failing test case to show how it’s not working and we can try to find time to investigate further.


As I mentioned in https://github.com/dynamoosejs/dynamoose/issues/445#issuecomment-445836105, I think the documentation does need to be updated. But I think that is the only problem that is going on here. If you would like to work on that, feel free to submit a PR and I would be happy to review it. Otherwise it will happen when someone gets around to it.

That’s a good point that fixing useDocumentTypes could be a breaking change at this point. One option could be to make it parse the string like the old behavior did if a string is found. This generally seems the safest since it also guards against people turning useDocumentTypes on and off during development. Another option could be to abandon useDocumentTypes as an option, and create a new one called useDocTypes.

I’m having the same problem as ReeSilva. The “useDocumentTypes: true” configuration option doesn’t seem to make any difference. Object types are always stringified.