openseadragon: Tile alignment bug (possible rounding error?)
As described in a comment on another PR, I have noticed a problem that causes tiles to get out of alignment on certain images at certain zoom images. The linked comment contains screen shots and details on how to reproduce.
I have done a git bisect and have found that the problem seems to have been introduced by this commit, which was first introduced in v2.4.0: https://github.com/openseadragon/openseadragon/commit/075a37ce8c7f82651f823d7f0c0bebce4214f058
I can see that tinkering with the numbers in getTileBounds in tilesource.js has some impact on the behavior – I suspect there’s some kind of rounding error putting tiles out of alignment in certain situations… but it’s a bit over my head to figure out what the solution might be!
Any ideas, @iangilman?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 38 (20 by maintainers)
Commits related to this issue
- Change to getTileUrl to only use "width," request syntax only with a level 0 compliant IIIF source. Otherwise, use "width,height". This avoids rounding errors when using Image API versions 1 and 2. Re... — committed to ruven/openseadragon by ruven 2 years ago
- Refactored code related to the use of the 'sizes' array. Reworking of pull request https://github.com/openseadragon/openseadragon/pull/2337 to perform stricter checking on the contents of the 'sizes'... — committed to ruven/openseadragon by ruven 2 years ago
My pleasure – I appreciate the great community effort!
@demiankatz Great! Thank you for confirming, and for working through this issue 😊
@ruven @iangilman I’ve tested the new code (https://github.com/openseadragon/openseadragon/commit/ae500afbcb143a8af49f0f8bf7c61d8f3e1ae56f) and it seems to work for us. Our IIIF server always generates “scaleFactors: [1, 2, 4, 8, 16]”, so “this.levelSizes” will be set only when the sizes array has 5 elements (which depends on the original resolution). Even in that case, I’ve seen no problems: no misalignment, no negative values in the request URL. Can be tested with this: info.json I hope this will fix it for everyone. Thank you, ruven.
On the question of the use of the
sizesarray:The use of the
sizesarray was to fix problems related to differences in rounding between OSD and different image codecs / image servers when requesting the sub-resolutions of an image (original discussion: https://github.com/openseadragon/openseadragon/issues/2321). In addition, for image formats such as multi-resolution pyramid TIFF or JPEG2000, these sub-resolutions exist “physically” in the image format, so aligning to these exact dimensions makes processing faster and more efficient. So, it’s definitely worth using an exact list of resolution sizes if this is available.In many cases, for example when using IIIF servers such as IIPImage or Canteloupe, this list will indeed consist of the sizes of the various resolution levels. However, as we can see in some of the problem examples, the
sizesarray does not necessarily contain a list of sub-resolutions and the API spec doesn’t, in any case, explicitly state that this array should be used in this way. The API spec only states that thesizesarray should contain the “preferred height and width combinations for representations of the full image”. The use ofsizesis, therefore, in reality a bit of hack to fix an ambiguity in the API itself (https://github.com/IIIF/api/issues/2232).So, the solution here is not to discard
sizes, but to make the checks much stricter. My original pull request https://github.com/openseadragon/openseadragon/pull/2337 assumes that thesizeslist is well-behaved, but clearly this is not a valid assumption. So I’ve re-factored the code (https://github.com/openseadragon/openseadragon/pull/2710/commits/ae500afbcb143a8af49f0f8bf7c61d8f3e1ae56f) to make it much stricter and to make it check each size against the givenscalingFactors. With this patch, the use of sizes is skipped in all the cited problem examples. This should resolve issue https://github.com/openseadragon/openseadragon/issues/2618 and fix the problem with the 2 images referred to in https://github.com/openseadragon/openseadragon/issues/2701#issuecomment-2792268149. Please take a look and let me know what you all think. This change may possibly also fix the issues in https://github.com/openseadragon/openseadragon/issues/2691 and https://github.com/openseadragon/openseadragon/issues/2452.@iangilman @demiankatz Problem started here: https://github.com/openseadragon/openseadragon/commit/0c358c140dae8bcd5cbc568ef6236a840af84efe and was published with v4.1.0. The comment of the commit is “Use resolution level dimensions provided in the info.json “sizes” field to determine tile sizes as well as the number of tiles that exist at a particular resolution.” I don’t think you can use the sizes array to calculate tile sizes since it is only for scaled full(!) images.
https://iiif.io/api/image/3.0/compliance/#5-level-0-compliance
I think the two has been combined by mistake and it is kind of random whether it works or not: depending on the number of scaleFactors and elements in the sizes array for example.
EDIT: I think it all started here: https://github.com/openseadragon/openseadragon/issues/2321#issuecomment-1507624044 Maybe this is why the sizes array was used… But it should have been much easier… in the tiles array, when the height is missing, it simply means to use the same as the width, so it’s a square: https://iiif.io/api/image/3.0/#54-tiles
Level 1+ services should support w,h requests if they’re compliant:
https://iiif.io/api/image/2.0/compliance/#size
Even if Cantaloupe fix their
sizesbug, this particular request will never work correctly as there is no way to get a height of 1024 if you only specify the width and if it is set to an integer value. If you set the width to 181 you get height = 1021 and if you set width to 182 you get height = 1027. Their use of an extra-large tile size of 1024 means that every rounding error gets exaggerated.You could do something like
But this risks to break a number of existing v2 installations which don’t expect
width,heighttile requests.I did not realize that Wellcome was using Cantaloupe, but all signs do now point to this being some kind of Cantaloupe bug. I used the Wellcome example in my ticket because it’s from a manifest that’s commonly used as a Universal Viewer example, but I first noticed the problem when working with Villanova manifests. Villanova currently uses Cantaloupe 5.0.6, so the issue is definitely still present there. I notice that Cantaloupe 5.0.7 was released very recently, and it includes a number of bug fixes. Next week, I will see if I can replicate the problem in my test environment, then upgrade to the latest Cantaloupe to see if that makes a difference.
I do think there may also be value in investigating ways to make the OSD requests more specific when feasible, to reduce the chances of ambiguity and inconsistent behavior – but I understand that the “when feasible” part is important, and perhaps there isn’t a reliable way to make things better for this case without potentially creating new problems.
I’ll report back when I have more information – thanks for the discussion and investigation, everyone!
Or change the OSD code to include height as well? Perhaps something is in the info.json that would indicate a level 0 with width only, so that case would be handled separately?
Canteloupe supports both 2 and 3 iirc, but some sites are probably stuck on 2 (for instance, not having the bandwidth to upgrade their manifests to v3).
So, the only way to fix this is to upgrade the server to IIIF v3 because it’s impossible to get a tile height of exactly 1024 for this particular image when specifying only the width.
That’s strange - Canteloupe usually correctly populates the sizes field in the info.json.
The response headers say:
x-powered-by: Cantaloupe/5.0.5So perhaps it’s a bug there?
Just to note the reason for only specifying the width for v2 is due to Canonical URI Syntax from: https://iiif.io/api/image/2.1/#canonical-uri-syntax. This is particularly important for a level 0 IIIF image which would only thave the width only image.
If we just consider the image request in isolation, the server is in fact responding the way it should.
The image has a native size of 2411x3372 according to the info.json file. The request asks for the region
2048,0,363,2048from this full resolution image, which means a region from x=2048 of size 363 in width (so up to the right-hand edge of the image) and 2048 in height starting from the top (y=0). So, an image of size 363x2048.The IIIF request then says it wants this 363x2048 image to be scaled to 182 pixels horizontally. The height calculation is (182/363) * 2048 = 1026.82 which rounds to 1027 and this is what is correctly returned by the server. By the way, if you ask for a width of 181 instead, the height = 1021.18 (so still not 1024) and the server does indeed correctly return a height of 1021:
https://iiif.wellcomecollection.org/image/b18035723_0003.JP2/2048,0,363,2048/181,/0/default.jpg
So, is the problem on the OSD-side? Not really. The problem does indeed come from a rounding issue, but the problem is server-side and related to the info.json file (I don’t know what IIIF server they are using - it’s not IIPImage in any case). If you take a look at this info.json, you’ll notice it doesn’t provide sufficient information on the available sizes of each resolution. In the
Tilessection, it says there are 6 resolution levels available withscaleFactorsfrom 1 to 32. However, thesizesfield only contains 4 sets of dimensions and these don’t correspond to the 6 resolution levels. OSD, therefore, cannot use these values and, therefore, has to calculate the sizes of these levels itself. If these resolution sizes were in the info.json, OSD would have used these to determine the exact tile sizes. The final rounding error is then compounded by the fact they are using a very large tile size of 1024x1024.In fact the OSD function getTileURL() for IIIF includes both the width and height if it’s version 3, but not if it’s 1 or 2 (the server is only v2):
If I remember correctly, this is coded this way because the use of a single dimension for tile requests is stipulated in the spec somewhere.
So, the best solution is to get the Welcome Collection to upgrade to IIIF v3 and/or fix the info.json files. Alternatively, an OSD solution when using IIIF version 2 or lower, would be to switch between specifying only width or height depending on which is larger.
For example, if you request a specific height rather than width for this same image you get the correct size of 182x1024.: https://iiif.wellcomecollection.org/image/b18035723_0003.JP2/2048,0,363,2048/,1024/0/default.jpg