deffcode: [Bug]: iPhone Portrait Oriented Videos are Squashed to Landscape

Description

Trying to decode a portrait iPhone video results in the video being squashed into landscape. The shape is reversed (width and height values switched), and the video content is stretched horizontally.

This behavior occurs with the native MOV output and persists with conversion to mp4.

If it helps, the behavior matches this bug from MoviePy: https://github.com/Zulko/moviepy/issues/1911 I found this library searching for a MoviePy alternative to circumvent this exact issue, so I would appreciate help in either squashing the bug or figuring out a good workaround.

Issue Checklist

• I have searched open or closed issues for my problem and found nothing related or helpful.
• I have read the Documentation and found nothing related to my problem.
• I’ve read the Issue Guidelines and wholeheartedly agree.

Expected behaviour

The video is loaded in portrait mode, matching the shape in the metadata. Playing in quicktime, my example looks like this:

Actual behaviour

The video is squashed into landscape mode, and the shape is reversed from the metadata. Visually it looks like this:

And in the terminal output, the difference in shape is apparent here:

Steps to reproduce

1. Record a portrait video on an iPhone
2. Use FFdecoder class to decode the video and observe the shape of the video

Terminal log output

18:34:24 ::   Utilities   ::   INFO   :: Running DeFFcode Version: 0.2.5
18:34:24 ::   FFhelper    ::  DEBUG   :: Final FFmpeg Path: ffmpeg
18:34:24 ::   FFhelper    ::  DEBUG   :: FFmpeg validity Test Passed!
18:34:24 ::   FFhelper    ::  DEBUG   :: Found valid FFmpeg Version: `b'5.1.2'` installed on this system
18:34:24 ::    Sourcer    ::  DEBUG   :: Found valid FFmpeg executable: `ffmpeg`.
18:34:24 ::    Sourcer    ::  DEBUG   :: Extracting Metadata...
18:34:24 ::    Sourcer    ::  DEBUG   :: Metadata Extraction completed successfully!
18:34:24 ::   FFdecoder   ::   INFO   :: Using default `rgb24` pixel-format for this pipeline.
18:34:24 ::   FFdecoder   ::   INFO   :: Default source resolution will be used for this pipeline for defining output resolution.
18:34:24 ::   FFdecoder   ::   INFO   :: Default source framerate will be used for this pipeline for defining output framerate.
18:34:24 ::   FFdecoder   :: CRITICAL :: Activating Video-Only Mode of Operation.
18:34:24 ::   FFdecoder   ::  DEBUG   :: Executing FFmpeg command: `ffmpeg -vcodec hevc -i /Users/username/Documents/iPhoneTesting/RawVideos/Cam0.mp4 -pix_fmt rgb24 -s 1920x1080 -framerate 29.97 -f rawvideo -`
ffmpeg version 5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with Apple clang version 14.0.0 (clang-1400.0.29.102)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/5.1.2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/username/Documents/iPhoneTesting/RawVideos/Cam0.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-02-07T22:32:21.000000Z
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 7
    com.apple.quicktime.software: 14.4.2
    com.apple.quicktime.creationdate: 2023-02-07T15:32:21-0700
  Duration: 00:00:31.83, start: 0.000000, bitrate: 7822 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080, 7680 kb/s, 29.97 fps, 29.97 tbr, 600 tbn (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : HEVC
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 89 kb/s (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 2 kb/s (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 34 kb/s (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Metadata
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'pipe:':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    com.apple.quicktime.creationdate: 2023-02-07T15:32:21-0700
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 7
    com.apple.quicktime.software: 14.4.2
    encoder         : Lavf59.27.100
  Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24(pc, gbr/bt709/bt709, progressive), 1920x1080, q=2-31, 1491500 kb/s, 29.97 fps, 29.97 tbn (default)
    Metadata:
      creation_time   : 2023-02-07T22:32:21.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.37.100 rawvideo
    Side data:
      displaymatrix: rotation of -0.00 degrees

Python Code(Optional)

from deffcode import FFdecoder, Sourcer
import cv2

def read_video_deffcode(file_pathstring: str) -> None:
    decoder = FFdecoder(str(file_pathstring)).formulate()

    frame_count = 0

    for frame in decoder.generateFrame():
        print(f"reading frame {frame_count}")
        if frame is None:
            print(f"could not read frame {frame_count}")
            break

        print(frame.shape)

        frame_bgr = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        cv2.imshow("Output", frame_bgr)

        # check for 'q' key if pressed
        key = cv2.waitKey(1) & 0xFF
        if key == ord("q"):
            break

        frame_count += 1

    cv2.destroyAllWindows()

    decoder.terminate()

def get_metadata_from_video_deffcode(file_pathstring: str) -> dict: 
    sourcer = Sourcer(file_pathstring).probe_stream()

    metadata_dict = sourcer.retrieve_metadata()

    print(metadata_dict)
    return metadata_dict

if __name__ == "__main__":
    input_video_pathstring = "VIDEO/PATH"

    read_video_deffcode(file_pathstring=input_video_pathstring)
    get_metadata_from_video_deffcode(file_pathstring=input_video_pathstring)

DeFFcode Version

0.2.5

Python version

Python 3.9.16

Operating System version

MacOS Monterey 12.6.1

Any other Relevant Information?

No response