Lychee: Some tag albums do not load size_variants for images

Detailed description of the problem [REQUIRED]

I have a tag album that seems to be getting null for all size_variants for all child images. I confirmed the size_variants are in db, and images load in their normal albums. This leads to an empty album being displayed. Not all tag albums are effected, and I am not able to reproduce this behavior in a new test tag album.

Sample of json response:

{
   "id":"MQTPtB8HTwEjaZqGyAsQNuzs",
   "show_tags":[
      "Alex"
   ],
   "max_taken_at":null,
   "min_taken_at":null,
   "thumb":{
      "id":"kdQAigqTbenfget2WUJkk5xW",
      "type":"image\/jpeg",
      "thumb":"uploads\/thumb\/a7ccb66be0baead7b7b02e6d264ecede.jpeg",
      "thumb2x":"uploads\/thumb\/a7ccb66be0baead7b7b02e6d264ecede@2x.jpeg"
   },
   "photos":[
      {
         "id":"GPjbt0B3KBKQm_iCStmEWZXb",
         "created_at":"2022-05-13T18:13:13.113101-07:00",
         "updated_at":"2022-05-13T18:13:59.488932-07:00",
         "album_id":"NpvFgSmSs1fqocFavGATgq_X",
         "title":"2022-05-13-0034",
         "description":"Red Cliffs, NV",
         "tags":[
            "Alex"
         ],
         "license":"CC-BY-NC-ND-4.0",
         "is_public":0,
         "is_starred":false,
         "iso":null,
         "make":"Nikon",
         "model":"LS-4000",
         "lens":null,
         "aperture":null,
         "shutter":null,
         "focal":null,
         "latitude":null,
         "longitude":null,
         "altitude":null,
         "img_direction":null,
         "location":null,
         "taken_at":"2022-05-01T12:00:00.000000-07:00",
         "taken_at_orig_tz":"America\/Los_Angeles",
         "type":"image\/jpeg",
         "filesize":0,
         "checksum":"a3b4f128860cb566927cd699e7ef19b0d6c15537",
         "original_checksum":"a3b4f128860cb566927cd699e7ef19b0d6c15537",
         "live_photo_content_id":null,
         "live_photo_checksum":null,
         "live_photo_url":null,
         "is_downloadable":true,
         "is_share_button_visible":true,
         "size_variants":{
            "original":null,
            "medium2x":null,
            "medium":null,
            "small2x":null,
            "small":null,
            "thumb2x":null,
            "thumb":null
         },
         "next_photo_id":"656pjb4jIMx_gstz3jS32wkt",
         "previous_photo_id":"Ql4Ef8AQPE-G83yz-lRivhNq"
      },
      {
         "id":"656pjb4jIMx_gstz3jS32wkt",
         "created_at":"2022-05-13T18:13:07.534035-07:00",
         "updated_at":"2022-05-13T18:13:59.452552-07:00",
         "album_id":"NpvFgSmSs1fqocFavGATgq_X",
         "title":"2022-05-13-0031",
         "description":"Red Cliffs, NV",
         "tags":[
            "Alex"
         ],
         "license":"CC-BY-NC-ND-4.0",
         "is_public":0,
         "is_starred":false,
         "iso":null,
         "make":"Nikon",
         "model":"LS-4000",
         "lens":null,
         "aperture":null,
         "shutter":null,
         "focal":null,
         "latitude":null,
         "longitude":null,
         "altitude":null,
         "img_direction":null,
         "location":null,
         "taken_at":"2022-05-01T12:00:00.000000-07:00",
         "taken_at_orig_tz":"America\/Los_Angeles",
         "type":"image\/jpeg",
         "filesize":0,
         "checksum":"4bf5ec7a2678ca6fb386705e932900457174a389",
         "original_checksum":"4bf5ec7a2678ca6fb386705e932900457174a389",
         "live_photo_content_id":null,
         "live_photo_checksum":null,
         "live_photo_url":null,
         "is_downloadable":true,
         "is_share_button_visible":true,
         "size_variants":{
            "original":null,
            "medium2x":null,
            "medium":null,
            "small2x":null,
            "small":null,
            "thumb2x":null,
            "thumb":null
         },
         "previous_photo_id":"GPjbt0B3KBKQm_iCStmEWZXb",
         "next_photo_id":"CcjCgUHwqcO7K8affrREgdCC"
      },

Console error msg: Uncaught TypeError: _photo.size_variants.original is null

In DB: MariaDB [lychee]> select * from size_variants where photo_id=‘GPjbt0B3KBKQm_iCStmEWZXb’; ±------±-------------------------±-----±------------------------------------------------±------±-------±---------+ | id | photo_id | type | short_path | width | height | filesize | ±------±-------------------------±-----±------------------------------------------------±------±-------±---------+ | 63697 | GPjbt0B3KBKQm_iCStmEWZXb | 0 | big/a3b4f128860cb566927cd699e7ef19b0.jpeg | 5632 | 3688 | 7582218 | | 63705 | GPjbt0B3KBKQm_iCStmEWZXb | 1 | medium/a3b4f128860cb566927cd699e7ef19b0@2x.jpeg | 3299 | 2160 | 2253539 | | 63704 | GPjbt0B3KBKQm_iCStmEWZXb | 2 | medium/a3b4f128860cb566927cd699e7ef19b0.jpeg | 1649 | 1080 | 651860 | | 63703 | GPjbt0B3KBKQm_iCStmEWZXb | 3 | small/a3b4f128860cb566927cd699e7ef19b0@2x.jpeg | 1100 | 720 | 316462 | | 63701 | GPjbt0B3KBKQm_iCStmEWZXb | 4 | small/a3b4f128860cb566927cd699e7ef19b0.jpeg | 550 | 360 | 90236 | | 63699 | GPjbt0B3KBKQm_iCStmEWZXb | 5 | thumb/a3b4f128860cb566927cd699e7ef19b0@2x.jpeg | 400 | 400 | 65902 | | 63698 | GPjbt0B3KBKQm_iCStmEWZXb | 6 | thumb/a3b4f128860cb566927cd699e7ef19b0.jpeg | 200 | 200 | 20629 | ±------±-------------------------±-----±------------------------------------------------±------±-------±---------+

Steps to reproduce the issue

Steps to reproduce the behavior: I am not sure what triggered this condition or what it is choking on.

Screenshots If applicable, add screenshots to help explain your problem.

Output of the diagnostics [REQUIRED]

Diagnostics
-----------
Warning: Dropbox import not working. dropbox_key is empty.
Warning: zend.assertions is disabled although Lychee is in debug mode. For easier debugging code generation for assertions should be enabled.

System Information
------------------
Lychee Version (git):            master (2fdd6bc) --  - Up to date (13 seconds ago).
DB Version:                      4.5.1

composer install:                dev
APP_ENV:                         production
APP_DEBUG:                       true

System:                          Linux
PHP Version:                     8.1
PHP User agent:                  Lychee/4 (https://lycheeorg.github.io/)
Timezone:                        America/Los_Angeles
Max uploaded file size:          500M
Max post size:                   500M
Max execution time:              200
MySQL Version:                   10.6.7-MariaDB-2ubuntu1

Imagick:                         1
Imagick Active:                  1
Imagick Version:                 1691
GD Version:                      2.3.0

Config Information
------------------
version:                         040501
check_for_updates:               1
sorting_photos_col:              created_at
sorting_photos_order:            DESC
sorting_albums_col:              max_taken_at
sorting_albums_order:            DESC
imagick:                         1
skip_duplicates:                 1
small_max_width:                 0
small_max_height:                360
medium_max_width:                1920
medium_max_height:               1080
lang:                            en
layout:                          1
image_overlay_type:              none
default_license:                 CC-BY-NC-ND-4.0
compression_quality:             90
full_photo:                      1
delete_imported:                 1
Mod_Frame:                       1
Mod_Frame_refresh:               30
thumb_2x:                        1
small_2x:                        1
medium_2x:                       1
landing_page_enable:             1
landing_owner:                   Vincent Juliano
landing_title:                   Life on 35mm
landing_subtitle:                Click top right to continue.  Next page top left to login
landing_facebook:                
landing_flickr:                  
landing_twitter:                 
landing_instagram:               
landing_youtube:                 
landing_background:              /uploads/medium/3d4bf4e35e5ea06853d7983c88efb6a2.jpg
site_title:                      Vince Juliano's Photo Gallery
site_copyright_enable:           1
site_copyright_begin:            2021
site_copyright_end:              2022
additional_footer_text:          "Alex's Photos" copyright Alex Panicacci 2022
display_social_in_gallery:       0
public_search:                   0
SL_enable:                       0
SL_for_admin:                    0
public_recent:                   0
recent_age:                      60
public_starred:                  0
downloadable:                    1
photos_wraparound:               1
map_display:                     1
zip64:                           1
map_display_public:              1
map_provider:                    Wikimedia
force_32bit_ids:                 0
map_include_subalbums:           1
update_check_every_days:         3
has_exiftool:                    1
share_button_visible:            1
import_via_symlink:              0
has_ffmpeg:                      1
location_decoding:               1
location_decoding_timeout:       30
location_show:                   1
location_show_public:            1
rss_enable:                      0
rss_recent_days:                 7
rss_max_items:                   100
prefer_available_xmp_metadata:   0
editor_enabled:                  1
lossless_optimization:           1
swipe_tolerance_x:               150
swipe_tolerance_y:               250
local_takestamp_video_formats:   .avi|.mov
log_max_num_line:                10000
unlock_password_photos_with_url_param: 0
nsfw_visible:                    0
nsfw_blur:                       0
nsfw_warning:                    0
nsfw_warning_admin:              0
map_display_direction:           1
album_subtitle_type:             oldstyle
upload_processing_limit:         4
public_photos_hidden:            1
new_photos_notification:         1
legacy_id_redirection:           1

Browser and system

Firefox 100.0 Mac Os 12.3.1

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 110 (81 by maintainers)

Most upvoted comments

Yeah, we could, but at the moment I am still busy with PHPStan for Nested Set and I believe that in 6 month from now nobody will encounter this bug anymore as the patch has been applied to all actively supported versions of MariaDB. Hence, even someone running an LTS version of Debian or Ubuntu should get the patch in a timely manner. This means the usefulness of the diagnostic will only have a very limited time span. Personally, I don’t feel like implementing it.

nagmat84 on Jun 25, 2022

@jln646v It seems to be even easier. From your diagnostic output, I see that you are running MariaDB 10.6.7 which is affected by this bug: https://jira.mariadb.org/browse/MDEV-27937

Can you update to 10.6.8 and report back? The affected and unaffected versions are listed in the referenced upstream bug.

As @cmb69 correctly pointed out there is no reason to implement a work around but just ask our user to update or downgrade to an unaffected version.

nagmat84 on Jun 25, 2022

Confirming that upgrading to mariadb 10.8 solved the issue. Thanks for all your help with this.

jln646v on Jun 27, 2022

A workaround is needed for users on hosted Environments with no chance to change anything 😃

You can ask your provider. I don’t think we should be spending time and effort for a problem that has already been fixed in a dependency, and only affects a very small number of users.

My Hoster is running v10.5.15-MariaDB-1:10.5.15+maria~focal-log btw.

According to their bug tracker, 10.5.16 contains the fix (along with 24 security issues) and was released last month.

d7415 on Jun 25, 2022

@cmb69 raised the idea that the bug might be related to ATTR_EMULATE_PREPARES. I added a 3rd test to MySQL1000Bug. Now, the failing test is executed twice: one time with ATTR_EMULATE_PREPARES set to true and one time with set to false. @qwerty287 could you re-run the test script once again and report the results?

nagmat84 on Jun 15, 2022

I LOVE my Hoster and can confirm the problem is gone 😃 v10.5.16-MariaDB

Thanks for all the effort.

SerenaButler on Jun 27, 2022

It seems that we are slowly getting somewhere. @craigfrancis could reproduce the problem with ATTR_EMULATE_PREPARES set to false and a low value for in_predicate_conversion_threshold. So enabling ATTR_EMULATE_PREPARES or increasing in_predicate_conversion_threshold should prevent the bug from being triggered. This does not solve the bug, but at least I see chances for a work-around. However, I cannot estimate the drawbacks of either work-around.

@jln646v I will try to update the test branch today. I will ping you when I will have it finished.

nagmat84 on Jun 25, 2022

I disabled opcache for php running under apache, and I still have the issue. The test script seems to yield the same results as before.

jln646v on Jun 22, 2022

No, if my suspicion is right and it is a regression bug between mysqlnd 8.0.13 and 8.1.5 then it is not a Heisenbug, but C.

nagmat84 on Jun 1, 2022

OK, that’s crazy. Test 1 and 2 do not work, but test 3 does. This means that the problem is not on the interface between MySQL and PHP PDO, but that Laravel fails to properly “implode” the array with the photo IDs. WTF?

For query 3 I construct the query manually like this:

DB::select(
  'SELECT * from size_variants WHERE photo_id IN (' .
  implode(',', array_map(fn (string $id) => '"' . $id . '"', self::PHOTO_IDS)) .
')');

Note, that I use PHP native methods array_map to enquote every ID in the array and implode to construct a comma-separated list. The failing query 2 uses the fancy Laravel Arr helpers internally. oh oh

But at least we now have a better idea where to look. That’s a job for tomorrow.

nagmat84 on May 31, 2022

Interesting. Again a “double” error. So it is not a memory resource problem on the ORM layer. The results do not even leave the DBA layer. I will write a new test diagnostic tomorrow. Today I call it a day.

nagmat84 on May 28, 2022