paper2remarkable: Exported annotations on cropped PDFs don't align

That’s great to hear, thanks so much for all your efforts on this @sternj!

According to ReMarkable support, they’re targeting 3.6 for this fix

An update-- this is something that I’ve actually brought to ReMarkable support and the developers apparently have a few theories as to the root cause that they’re in the process of testing. Basically, I don’t think the issue is actually with this repo at all, it’s either in Xochitl or in both Xochitl and in rmapi. It’s worth keeping this issue open for tracking purposes, but the root problem comes from and will be resolved elsewhere.

Well, I tried messing around with this in a few different ways only to conclude that there’s something else going on other than just mismatches in the coordinate system. I reached out to Remarkable Support who informed me that the tablet expects something that’s shaped like A4, which means there’s some additional scaling happening (which is also reflected in the rmapi code). They suggested using “print to pdf” from Chrome, which doesn’t solve the issue at hand, but the equivalent from Firefox does work, so… progress I guess? I spent a few hours trying to figure out what those web browsers are actually doing when they “print to PDF” but so far what I’ve found is a rather inscrutable event loop.

What’s weird to me (and sufficiently weird to remarkable support that they’ve escalated it to the devs) is

• Annotations show up properly on the desktop app
• Annotations are irregular for irregularly positioned PDFs exported from Chrome, but not Firefox.

I’ve made some good progress reverse engineering the PDF renderer from Xochitl itself, so I’m also digging into that and seeing what transformations it does to actually display the PDF.

I also reached out to the person who originally wrote the annotation code in rmapi, but he seems to be quite busy in his real life and his understanding largely relies on the tacit assumption that Xochitl makes that a PDF is shaped like an A4.

Once I’m out of this current push at work I’m going to try to figure out in a more systematic way what makes Firefox-rendered PDFs different from Chrome-rendered PDFs or p2r-rendered PDFs.

Long story short, this… is really weird, even by PDF standards. More to come as I read more \MediaBox entries and specify the proper arity of various QT functions.

So, to summarize:

• paths are defined by their coordinates in “User Space”
• User Space is defined as… something. It seems to be an invariant, but is mapped onto the device by an object called a transformation matrix
• User space doesn’t seem to be affected by changes to things like the CropBox or MediaBox
• The visible page’s origin is not (0,0) in user space

Idea: perhaps the ReMarkable itself is defining coordinates based on its own screen, in which case what would need to happen is that rmapi would just need a minor edit to make the MediaBox (or whatever box) start at its (0,0)

Thank you for letting your email inboxes be filled by my comments! I think I know where to look now. More info to come.

Just like a normal person, I have found and started reading the pdf standard. What rmapi does is it uses a path, which is defined in section 8.5.2. The path is created relative to the coordinates in “user space”. What worries me is that the objects in the PDF itself might be defined in terms of userspace, which isn’t something p2r manipulates from what I can tell. That said, if that’s the case, I can’t tell for the life of me how any of this works-- if coordinates are absolute or quasi-absolute, why does cropping work at all?

Confirming that the displacement of the annotations is directly related to how the box was created, I removed the upper and lower cropping and the displacement was only horizontal. What on earth are things drawn on top of?

Reassessing my previous assessment, the salient code happening in rmapi is the end of this function. I’m concerned by the reference to the transformation matrix, since it means that the math here might be more complex than I was thinking. However, it seems like the annotations are being translated down and to the right, rather than being uneven. I’m trying to figure out how annotations are drawn on the PDF, because it seems like they’re being translated opposite the cropping

For reference, this is how the PDF library thinks about annotations. My largely uninformed theory is that unipdf is deciding the page height based off of some metadata field that isn’t changed in here. I’m going to do some more hunting, it seems like the dimensions of each PDF page is decided individually, specifically (it seems) by the mediabox. I think it’s done here, since rmapi appears to put in the original PDF pages as “background” pages. I’m not entirely sure how this all connects yet, but my suspicion is that the recomputation being done might be causing the issues.

Thanks for sharing your approach @gwtaylor! This seems like a good intermediate solution to have around while #95 is unfinished, and I’m sure it’ll help people who are dealing with the same issue.

FWIW, I did indeed mean a soft-crop + export, so good to know that that does work. Thanks!

Done!