other_video_transcoding: Error transcoding to HEVC on a 2019 MacBook Pro

Okay, I’m sure everything is up to date with other-transcode 0.9.0 now. I also double-checked that when I made my Big Sur boot volume I had installed version 0.9.0 there too. All good.

other-transcode --crop auto --mp4 --hevc testvideo.mkv This works as before and uses the software encoder on Monterey but the hardware encoder on Big Sur. On Monterey, besides -c:v hevc_videotoolbox, it also adds -pix_fmt:v yuv420p10le, -maxrate:v 12000k, and -bufsize:v 12000k. So we still have differences per operating system for some reason.

other-transcode --crop auto --mp4 --hevc --vt testvideo.mkv Attempting to force hardware encoding with --vt resulted in the same error as before. The ffmpeg command line generated is exactly the same on macOS Monterey and Big Sur but for some reason it works on Big Sur and fails on Monterey.

other-transcode --crop auto --mp4 --hevc --vt --10-bit testvideo.mkv Trying the --10-bit parameter still makes it work. The ffmpeg command line generated is exactly the same on macOS Monterey and Big Sur.

@douglsmith Unrelated to the rest, you might want to remove that system report. There’s a lot of information in there that you might not want to have hanging out in a public space.

@douglsmith Thanks so much for all of your testing on this! I transcoded a bunch of files last night, and adding the --10-bit flag did indeed trigger the hardware transcoder on my machine in Monterey. Super strange why the outputs would be different depending on OS and without that flag, but it’s nice to know that there’s a workaround. Thanks again!

@donmelton You’re welcome to close this if you’d like - I’m not sure the problem is exactly resolved, but now that we have a workaround things are solved for me.

Tagging the original poster, @RusselProuse34. See the workaround above.

@donmelton Let me know if there’s anything else I can test here. I’m happy to help.

BTW, I’ll probably run the same tests on an M1 MacBook Pro when I get a chance just because I’m curious.

I installed macOS Big Sur on an external boot drive and did a bunch of testing comparing it with macOS Monterey on my 2019 MacBook Pro. The results were quite interesting.

TL;DR Some commands work in Big Sur that no longer work in Monterey, but there is a possible workaround.

I started with a test video that is a 01:44 portion of a movie ripped from a Blu-ray. It is 8-bit according to mediainfo. For each test I recorded the encoding time, the file size, and the bit depth since those are good indicators of the kind of encoding that actually happened.

These parameters: other-transcode --crop auto --mp4 --hevc testvideo.mkv were successful under Monterey. It executed this command, which did software encoding: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v libx265 -pix_fmt:v yuv420p10le -b:v 4000k -maxrate:v 12000k -bufsize:v 12000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:02:13, bytes: 54,893,775, 10 bits

The same parameters were successful under Big Sur but the command it executed was different and used hardware encoding: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:18, bytes: 57,759,781, 8 bits

So under Monterey it used the software encoder at 10 bits but under Big Sur it used the hardware encoder at 8 bits.

Then I tried forcing the hardware encoder: other-transcode --crop auto --mp4 --hevc --vt testvideo.mkv Under Monterey it crashed and burned: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4

Transcoding...
[hevc_videotoolbox @ 0x7facd086b000] Error encoding frame: -12905
[hevc_videotoolbox @ 0x7facd086b000] popping: -542398533
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
/usr/local/bin/other-transcode: transcoding failed: testvideo.mp4

But it ran successfully under Big Sur: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:18, bytes: 57,759,781, 10 bits (mediainfo)

The ffmpeg command line was slightly different for each OS and caused ffmpeg to error on Monterey. It says it’s 10 bits under Big Sur. The files size is the exact same number of bytes as the previous test in Big Sur that says it’s 8 bits. Can that be true or is the metadata wrong?

Next I tried forcing both the hardware encoder and 10-bit: other-transcode --crop auto --mp4 --hevc --vt --10-bit testvideo.mkv It actually completed without error on Monterey by adding --10-bit: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -pix_fmt:v yuv420p10le -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:36, bytes: 60,224,739, 10 bits (mediainfo)

The short encode time tells me that it was actually hardware encoding as the ffmpeg command shows. This may be a viable workaround. I’ve tested it many times now with other files and it is working consistently.

The same parameters under Big Sur also worked and produced this: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -pix_fmt:v p010le -b:v 4000k -profile:v main10 -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:33, bytes: 57,737,187, 10 bits (mediainfo)

Is it Other Transcode that is generating the different ffmpeg command line under each operating system when starting with the same parameters?

I hope that is useful in some way.

@khaosx Here is the mediainfo output: Hugo-mediainfo.txt

@ttyS0 Thanks for that.

@douglsmith Can you give us the text output from mediainfo for your source file?

@donmelton ffmpeg version 4.4.1. I ran ffmpeg -version and it gave me a bunch of other info, so I can post that too if it would help.

I’ve attached a log file as well. _ffmpeg_27442_64736.mkv.log

@donmelton I’ve had a look and confirmed that my Mac isn’t running in Low Power Mode on either battery or while plugged in.

I’m going to uninstall Homebrew and the associated packages I set up to use with other_video_transcoding and reinstall them to see if that helps. I’ll report back if anything changes.

Thanks for the quick response @donmelton. Yes, if I leave out —hevc, the system uses VideoToolbox automatically whether I add the —vt argument. So it seems to just be a problem with transcoding to HEVC.