calcurse: calcurse-caldav does not create sync.db

First of all, thank you for calcurse, I really love it, though I do have a small problem with the calcurse-caldav script.

The script does not create sync.db, so I have to run calcurse-caldav --init keep-remote instead of just calcurse-caldav each time. To be clear, ~/.calcurse/caldav/ only contains config and oauth2_cred, not sync.db.

I am using MacOS with python 3.6.5.

I changed the shebang in the script to use my anaconda python interpreter:

#!/Users/username/anaconda3/bin/python

import argparse
...

The only error I get is after importing new objects: error: Missing ETag. Here is the abbreviated output of calcurse-caldav --init keep-remote --debug.

Removing all local calcurse objects...
> REPORT https://apidata.googleusercontent.com/caldav/v2/username@gmail.com/events/
> Headers: {'Content-Type': 'application/xml; charset=utf-8', 'Depth': '1'}
> <?xml version="1.0" encoding="utf-8" ?><C:calendar-query xmlns:D="DAV:"                   xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><D:getetag /></D:prop><C:filter><C:comp-filter name="VCALENDAR" /></C:filter></C:calendar-query>

< Headers: {'vary': 'Origin, X-Origin', 'content-type': 'text/xml; charset=UTF-8', 'date': 'Wed, 08 Aug 2018 02:58:39 GMT', 'expires': 'Wed, 08 Aug 2018 02:58:39 GMT', 'cache-control': 'private, max-age=0', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '1; mode=block', 'server': 'GSE', 'alt-svc': 'quic=":443"; ma=2592000; v="44,43,39,35"', 'transfer-encoding': 'chunked', 'status': '207', 'content-length': '9858', '-content-encoding': 'gzip'}
< <?xml version="1.0" encoding="UTF-8"?>
< <D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
<   <D:href>/caldav/v2/username%40gmail.com/events/040000008200E00074C5B7101A82E008000000001032E219E727D401000000000000000010000000520C9754E7195348B8D8330B8095443D.ics</D:href>
<   <D:propstat>
<    <D:status>HTTP/1.1 200 OK</D:status>
<    <D:prop>
<     <D:getetag>"63669375554"</D:getetag>
<    </D:prop>
<   </D:propstat>
<  </D:response>
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
...
New sync database entry: /caldav/v2/username%40gmail.com/events/040000008200E00074C5B7101A82E0080000000080A684B65D01D4010000000000000000100000004AD2AA5C136245499E905152F2B0D28F.ics 63669375575 9ceb385c95332e061e2c0e44fbff8d50f9cb92bd
Importing new object 63669373923.
New sync database entry: /caldav/v2/username%40gmail.com/events/0bmpjovrcdbgnn8vsu4gkre20h%40google.com.ics 63669373923 6b98e5881efcb36271c62ff4c9f5bd7ebf3c7f56
error: Missing ETag.
error: The error occurred while processing the following XML node:
error: <ns0:response xmlns:ns0="DAV:">
error:   <ns0:href>/caldav/v2/username%40gmail.com/events/poo9eff8smpd4q82t9nd7rd
error: 120_R20180213T171500%40google.com.ics</ns0:href>
error:   <ns0:status>HTTP/1.1 404 Not Found</ns0:status>
error:  </ns0:response>

I just realized that this issue is similar to (or the same as) #90 by @scndthe2nd.

I’m happy to send the full debug output. I saved it as a txt file.

About this issue

Original URL
State: open
Created 6 years ago
Comments: 22 (9 by maintainers)

Commits related to this issue

calcurse-caldav: handle objects with status code 404 Display a warning if any of the events reported by the server cannot be found instead of bailing out. Addresses GitHub issue #137 and #245. Sign... — committed to lfos/calcurse by zzandland 5 years ago
calcurse-caldav: handle objects with status code 404 Display a warning if any of the events reported by the server cannot be found instead of bailing out. Addresses GitHub issue #137 and #245. Sign... — committed to lfos/calcurse by zzandland 5 years ago

Most upvoted comments

Strange. Well, maybe we should just display a warning if any of the reported events returns a 404 and continue syncing.

lfos on Feb 26, 2019

I’m seeing the same thing as @rslindee (Missint ETag, 404 in debug output, trying to view in a browser says "Login Required).

sjl-10x on Feb 26, 2019

Has there been any progress on this? I’m experiencing a similar issue (“Missing ETag” followed by a “404 Not Found” in the debug output when attempting to create a sync.db. If I attempt to view said .ics, I get a “Login Required” message).

Is there perhaps a way to override or ignore said problematic items?

edit: One interesting data point is that if I export my “main” calendar to an ics file, then import said file into a “new” calendar and set caldav to the “new” calendar, everything appears to work fine.

rslindee on Feb 11, 2019

I hacked in a quick check for this to see if it would work:

diff --git a/contrib/caldav/calcurse-caldav.py b/contrib/caldav/calcurse-caldav.py
index 7b9b56c..e8505eb 100755
--- a/contrib/caldav/calcurse-caldav.py
+++ b/contrib/caldav/calcurse-caldav.py
@@ -442,6 +442,16 @@ def pull_objects(hrefs_missing, hrefs_modified, conn, syncdb, etagdict):
             die_atnode('Missing href.', node)
         href = hrefnode.text
 
+        statusnode = node.find("./D:status", namespaces=nsmap)
+        if statusnode is not None:
+            status = re.match(r'HTTP.*(\d\d\d)', statusnode.text)
+            if status is None:
+                die_atnode('Could not parse status.', node)
+            statuscode = status.group(1)
+            if statuscode == '404':
+                print('Skipping missing item: {}'.format(href))
+                continue
+
         etagnode = node.find("./D:propstat/D:prop/D:getetag", namespaces=nsmap)
         if etagnode is None:
             die_atnode('Missing ETag.', node)

This did allow it to get past the problematic items and sync them down. Unfortunately now I can see there there are other problems that mean calcurse isn’t going to work for me:

Many of the items in my calendar show up as two separate events, probably because I have Google Calendar set to show both Eastern and Pacific times because I work remotely.
When I run the sync again after changing nothing, it started sending lots of “missing objects” to the server. I’m guessing it’s seeing the duplicates from the last item and sending them, but I ctrl-c’ed the thing before it could do too much damage so I’m not 100% sure.
When I view an item in calcurse I just see the title, no description. I need the descriptions for things like Zoom meeting links.

So, I don’t think I’m going to spend any more time on this. If someone else wants to use the diff above as a starting point, feel free. Thanks anyway.

sjl-10x on May 8, 2019

Well, maybe we should just display a warning if any of the reported events returns a 404 and continue syncing.

This seems like a reasonable workaround to me. There’s probably just one wonky event somewhere in my calendar that’s breaking things.

sjl-10x on Mar 25, 2019