amplify-cli: Filtering on weeknumbers greater then 10 doesn't return results

I’m having a table of around 4000 recipes. Every recipe has a weeknumber.

When I run:

this.apiService.ListRecipes({ weeknumber: { eq: moment().week() } }, 4000)

I get zero results sinds it is currently week 20

It only works when entering a static number between 1 and 10. My table Recipes has also records with weeknumbers between 11 and 52

Another unexpected behavior is, that it is performing the limit: 4000 before filtering the weeknumbers.

I was expecting it would first ask the database for all entries with weeknumber X and then limit the results.

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 1
Comments: 16 (3 by maintainers)

Most upvoted comments

If you need to download 5MB of data, you can use a recursive approach. Here is a draft of how it looks like:

let allRecipes = []
let search = {limit: 100000}
const recursiveGetRecipes = (currentResult) => {
    // if first iteration or if there is a next Token
    if (currentResult === null || currentResult.nextToken != null) {
        if (currentResult !== null && currentResult.nextToken != null) {
           // other iterations
            search.nextToken = currentResult.nextToken
        }
        API.graphql(graphqlOperation(listRecipes, search)).then((result) => {
            currentResult = result.data.listRecipes;
            allRecipes = allRecipes.concat(currentResult.items);
            this.recursiveGetRecipes(currentResult)
        }).catch((error) => {
            // error
        })
    } else {
        // done
    }
}
recursiveGetRecipes(null);

vparpoil on May 16, 2019

Ok the dynamoDB scan function indeed first limits the results and apply the filtering on that subset, this is not a very helpful order in my opinion.

In a response, DynamoDB returns all the matching results within the scope of the Limit value. For example, if you issue a Query or a Scan request with a Limit value of 6 and without a filter expression, DynamoDB returns the first six items in the table that match the specified key conditions in the request (or just the first six items in the case of a Scan with no filter). If you also supply a FilterExpression value, DynamoDB will return the items in the first six that also match the filter requirements (the number of results returned will be less than or equal to 6).

You wouldn’t believe the reason behind why I’m only getting back records for week 1 to 10. My total table size is 5,56MB Divide this bij 52 weeks and I’m on 0.1MB per week. Times 10 and I’m on 1MB which is unbelievable the maximum data size for the scan operation!

This took me 11 hours of debugging to find the reason.

So how on earth are other companies getting API’s build with all these limits?

sprucify on May 16, 2019