pytodoist: keyerror on get_completed_tasks

Last week my script started throwing errors when getting tasks in a project, but only for some projects. I created new projects, moved my tasks there, and all was fine for a while. However, now it’s resurfacing with the new projects.

Traceback (most recent call last):
  File "script.py", line 64, in <module>
    copy_due("Tickler", "Solitary-actionable")
  File "script.py", line 36, in copy_due
    due_tasks = user.get_project(inp).get_uncompleted_tasks()
  File "/home/martin/.local/lib/python3.5/site-packages/pytodoist/todoist.py", line 1026, in get_uncompleted_tasks
    completed_tasks = self.get_completed_tasks()
  File "/home/martin/.local/lib/python3.5/site-packages/pytodoist/todoist.py", line 1058, in get_completed_tasks
    project = self.owner.projects[task_json['project_id']]
KeyError: 1490683247

Any ideas?

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 23 (9 by maintainers)

Commits related to this issue

Ignore orphan tasks, comments and reminders Todoist API can sometimes return inconsistent data, containing tasks, comments and reminders of archived projects. While it's not fixed by the API itself, ... — committed to imankulov/pytodoist by imankulov 6 years ago

Most upvoted comments

Hey guys,

Thanks a lot for the report and for the investigation, especially for @pete-adriano-debiase, who went to great lengths to find out the root cause and the workaround. It was an API issue indeed, and we’ve just fixed it and deployed changes to production. It should fix the issue, but please let me know if it still doesn’t.

imankulov on Jul 4, 2018

Hi @Garee,

I spent a bit more time digging into this tonight. Per your suspicion, Todoist is indeed returning invalid project IDs for what appears to be a very particular subset of tasks.

Specifically, at least in my Todoist account, tasks completed prior to 2018-04-09 in projects created prior to 2018-04-09 seem to have nine-digit project IDs starting with a 1 (such as 137_028_573). Tasks completed on or after 2018-04-09 in projects created prior to 2018-04-09 seem to have nine-digit project IDs starting with a 9 (such as 942_334_941). The difference between the 9-prefixed project IDs and the 1-prefixed project IDs is always exactly 805_306_368.

Furthermore, it appears that projects created on or after approximately 2018-04-09 have ten-digit project IDs starting with a 2 (such as 2_168_462_001).

Among tasks with 1-prefixed project IDs, 9-prefixed project IDs, and 2-prefixed project IDs, the get_completed_tasks() function only returns the KeyError exception discussed in this thread for tasks with 9-prefixed project IDs. I am able to successfully retrieve all completed tasks for all of my Todoist projects by changing line 1058 of pytodoist\todoist.py from:

project = self.owner.projects[task_json['project_id']]

try:
    project = self.owner.projects[task_json['project_id']]
except KeyError:
    project = self.owner.projects[task_json['project_id'] - 805_306_368]

This temporary fix works for me, but I am not sure if this will work for other users. In particular, the magic 805_306_368 number may be specific to my account.

Is there a more elegant way to implement handling for the invalid project ID formats? I am not yet familiar enough with the Todoist API to be confident in creating a pull request-worthy solution.

pete-debiase on Jun 4, 2018

Hi @pete-adriano-debiase. Sorry, I tried hard to reproduce the issue, but no luck at all. I used to be able to reproduce the problem on my account before we applied the fix, but now it returns correct results.

Some more details.

If the issue can be reproduced, then it’s only valid for old projects (ones that were created more than about a year ago). Inbox is probably the best candidate for the test. So it makes sense to create and complete a task in the inbox to see if it’s still a problem.

The API URL has to be https://api.todoist.com/API/v7/

>>> api.URL
'https://api.todoist.com/API/v7/'

The output for project ids has to be coherent and has to contain “small integers” (ones which start with 1, not those which start with 9)

Finally, to make it a bit less obscure, 805_306_368 is nothing but 3 << 28. It’s a constant for your account, totally internal, and related to the fact that your account belongs to the database shard number three.

Can you please try to test it one more time to see if it’s still a problem.

imankulov on Jul 12, 2018

Hi @rohitpaulk, thanks so much for looking into this. Here is the requested info for a few completed tasks from my account that might be worth looking at. I picked two tasks on either side of the date of 2018-04-09 on which this issue may have started to occur.

Task ID	Project ID in Response	Correct Project ID	Date Completed
2584361591	936939558	131633190	Wed 11 Apr 2018 08:49:57 +0000
2599950584	936939558	131633190	Tue 10 Apr 2018 11:23:04 +0000
2587633490	131633190	131633190	Mon 09 Apr 2018 03:33:32 +0000
2586839303	131633190	131633190	Mon 09 Apr 2018 03:14:41 +0000

For the first two tasks in this table, the value in the items > project_id field in the response I’m getting from Todoist seems to be larger than the correct project ID by the magic number of 805_306_368. This response was obtained via a call to the get_all_completed_tasks function of Todoist’s Python API.

pete-debiase on Jun 12, 2018