osxphotos: When exiftool fails on export | Pic still gets exported with missing data | Subsequent exports will not re-export the pic!

Latest version

  • Version: 0.55.2

Describe the bug When exiftool fails exporting an edited Pic (rotated + added GPS location in iPhoto/Photos):

  • File is exported without GPS data (of course it depends on the error from exiftool. Possible other errors to from exiftool
  • Noticed that touch does not seem to also run on the file!
  • Although the edited file is still exported
  • …and subsequent runs consider the file up-to-date, when, in fact it does not have GPS data.

IMPORTANT:

  • Just want to make you aware of this sequence/logic for this situation in case it may be helpful for other cases. I know I have to fix the pics myself. Will do that with “Script Editor” later on…
  • The problem happened with a certain Camera (OLYMPUS D800, which I no longer use) and rotated pics in iPhoto.
  • Somehow the edited pic in Photos (that comes from ~8 years ago through several iPhotos versions) generates “Maker Notes” and “Duplicated Orientation” tag on exiftool
  • I know I’ll have to fix them all.
1st run: exiftool error!
  • But the file is exported but does not get all the EXIF info. For instance no GPS position data.
2022-12-14 22:24:11.961484 -- Exporting P4021768.JPG (P4021768.JPG) (6/6)
2022-12-14 22:24:11.962024 -- Skipping original version of P4021768.JPG
2022-12-14 22:24:11.963077 -- Exporting edited version of P4021768.JPG (P4021768.JPG)
2022-12-14 22:24:11.967205 -- Writing metadata with exiftool for P4021768_edited.jpeg
2022-12-14 22:24:12.367336 -- Exported P4021768.JPG to /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg
2022-12-14 22:24:12.376409 -- ⚠️  exiftool warning for file /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg: Warning: Duplicate Orientation tag in IFD0 - /var/folders/28/x8vbdy4j4vn680b5pt33hs0w0000gn/T/osxphotos_export_8vg04xwd/CDF617F6-9839-4A65-BE82-B28C45247FC6_1_201_a_exiftool.jpegWarning:  Possibly incorrect maker notes offsets (fix by -18?) - /var/folders/28/x8vbdy4j4vn680b5pt33hs0w0000gn/T/osxphotos_export_8vg04xwd/CDF617F6-9839-4A65-BE82-B28C45247FC6_1_201_a_exiftool.jpeg
2022-12-14 22:24:12.380512 -- Retrying export for photo (CDF617F6-9839-4A65-BE82-B28C45247FC6: P4021768.JPG)
2022-12-14 22:24:12.389445 -- Skipped up to date file /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg
2nd run: consider the file up-to-date! But GPS data is missing.
2022-12-14 22:26:15.367527 -- Exporting P4021768.JPG (P4021768.JPG) (6/6)
2022-12-14 22:26:15.368193 -- Skipping original version of P4021768.JPG
2022-12-14 22:26:15.369319 -- Exporting edited version of P4021768.JPG (P4021768.JPG)
2022-12-14 22:26:15.496074 -- Skipped up to date file /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg
On Photos “Revert to Original”(Cancels rotation but keep GPS data) and Re-edit to rotate: 3rd run
3rd run: perfect run. No exiftool error. GPS data exported. Touched executed.
2022-12-14 22:28:00.247428 -- Skipping original version of P4021768.JPG
2022-12-14 22:28:00.248482 -- Exporting edited version of P4021768.JPG (P4021768.JPG)
2022-12-14 22:28:00.377742 -- Writing metadata with exiftool for P4021768_edited.jpeg
2022-12-14 22:28:00.605418 -- Exported P4021768.JPG to /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg
2022-12-14 22:28:00.613356 -- Exported updated file /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg
2022-12-14 22:28:00.613782 -- Touched date on file /Users/Shared/Pictures/Export/bad_pics_export_osxphotos/local/(...)/P4021768_edited.jpeg

To Reproduce Steps to reproduce the behavior:

  • Command line on the crontab
osxphotos export "${DSTFOLDER}" --exportdb ${FOLDER}/.osxphotos_export.db --load-config ${WORKFOLDER}/colibri.toml >> ${LOGFILE} 2>&1
  • TOML file being used
[export]
cleanup = true
db = "/Users/Shared/Pictures/iPhoto Shared Library.photoslibrary"
description_template = "Album:{album,}{newline}Description:{descr,}"
directory = "{album|filter(startswith 1|2)[/,.|:,.]}"
exiftool = true
exiftool_merge_keywords = true
exiftool_merge_persons = true
keyword_template = [ "{keyword}", "{label}", "{searchinfo.activity?activity:{searchinfo.activity},}", "{searchinfo.venue_type?venue_type:{searchinfo.venue_type},}", ]
no_progress = true
not_hidden = true
ramdb = true
retry = 2 
skip_bursts = true
skip_live = true
skip_original_if_edited = true
strip = true
timestamp = true
touch_file = true
update = true

Expected behavior Possibilities thinking out loud:

  1. Either continue to try to re-export the file – may be dumb as it will continue to give the same error!
  2. One additional message to the user that he must fix it and possibly no re-exports will occur. Partially addressed as I already get the error from exiftool.
  3. Delete the file in case of exif_tool error – maybe too drastic.
  4. Leave as it is.

Screenshots N/A Desktop (please complete the following information):

  • OS: Monterey 12.6.32
  • osxphotos version: 0.55.2

Additional context

  • DB with referenced pics.
  • Edited pics done inside iPhoto or Photos

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 16 (15 by maintainers)

Commits related to this issue

Most upvoted comments

@teh-hippo can you send me one of the files in question? Ideally I’d like the original from Photos and the version processed by exiftool.

osxphotos.py@gmail.com

I think I’ve fixed this in the branch I’m currently working on for #999. Will try to get a new release out this weekend.

I understand last-errors would report last run errors tagged with the same start date,

--last-errors does not look at the date/time, it looks at the most recent export results stored in the database (the source for --report) and extracts any errors found there. --errors searches for any errors stored in the error column of the database. Neither filters on date/time group.

I looked at the code and I can’t see any obvious causes nor can I replicate this. I’ll take a deeper look when I get a chance but it may be a while as I’ve got other projects going. I view osxphotos exportd as a sort of a “best effort” project that I don’t spend a lot of time on. It’s a collection of useful utilities but not intended to be a “primary” feature.

Note: the date/time stamp shown in --errors is the date/time that photo’s record was last updated (when --update was run), not the date/time the error occurred. I realize this isn’t intuitive but it’s a limitation of the export database schema and when I added the errors column I didn’t want to make significant changes to the database (which always introduces risk of more severe bugs).