Color

[Lyberta]

Deleted.

[fluffrabbit]

Spielberg is an old man who relies on technicians to take care of his visuals. The technicians behind a big-budget film must use HDR now. It's gospel. I have a filmmaking background myself and I have never worked in HDR, and I suspect that open source tools aren't fully on that bandwagon yet because independents don't have the same distribution channels. A theater screen makes a digital image brighter and shows a lot more artifacts compared to an LCD monitor, but most people working on these tools only have LCD monitors.

I would like to add another HDR format to the list: Radiance HDR. This format is supported by stb_image and is very similar to OpenEXR. AFAIK ffmpeg does not support this format any better.

I presume OpenGL is dead then I guess.

You can render into an HDR framebuffer and then copy that memory to the client for display in a platform-specific way. Though not as performant as s/RGB, it's better than nothing if you really want to output HDR with OpenGL.

IMHO ffmpeg/libav sounds like a difficult way to get HDR information because the software is optimized for performance first and foremost, so it probably throws out extra color information as early in the pipeline as possible to save on memory bandwidth. If you can get the HDR color information somehow, I believe you can use stb_image to save it as OpenEXR or Radiance HDR.

[Lyberta]

Deleted.

[fluffrabbit]

tl;dr I'd give ffmpeg AV1 encoding followed by mp4box AV1->AVIF muxing a shot

That's to save space. stb_image and other tools will convert to and from HDR RGB floats.

There is also AVIF if quality is a concern. I would expect ffmpeg support to be good since it's based on a video codec, but I dunno about ffmpeg's HDR support considering a few years ago ffmpeg strictly dealt with 8-bit color channels.

EDIT: NOPE. Support for AVIF is garbage.

However, I can vouch for the efficacy of mp4box. The tried and true solution for many multimedia challenges on Linux is to chain multiple tools together.

AV1 support in ffmpeg is only available from 2019 distros onwards as the standard has only recently been stabilized (the development timescale was way too optimistic). So, assuming you're on Ubuntu 19.xx or something similar or using a PPA etc.:

{l Code}: {l Select All Code}

sudo apt install ffmpeg gpac

Then, encode the video as AV1 in HDR mode from -ss <wherever> of -t <less than 1 second>. Encoding guide here.

Then use mp4box from the gpac package to demux the first keyframe into an AVIF image file.

[Lyberta]

Deleted.

[fluffrabbit]

Yes, going from anything to AV1 or JPEG2000 would be lossy. They're lossy formats, like h.265. OpenEXR is technically a lossless format, but there is colorspace conversion. It's kinda like how FLAC is a lossless format that doesn't 1:1 correlate with the sample format on an audio CD; it's lossless but there is still a numerical translation that inherently has a small loss. Though the loss would be greater going to AV1 due to lossy compression, the quality is still good and it would probably look better than JPEG2000.

If patents don't bother you, the only truly lossless conversion I can imagine would be demuxing h.265 keyframes into BPG or HEIF via mp4box.

[fluffrabbit]

MP4Box in Ubuntu (and probably other distros) is way outdated (2012). GPAC doesn't like to deal with the community because they're French and you know how the French are. They've got a huge backlog of bug reports, more than ffmpeg, so I guess the tool is more popular for whichever reason.

I have adapted the build instructions from here. Cloning the Git repo may take a while; it's 91 MB. The source files themselves take up 45 MB. I don't know why.

{l Code}: {l Select All Code}

git clone https://github.com/gpac/gpac.git

The build instructions are outdated, so even though I did my best to replace the referenced libraries with newer versions, it may not work 100%:

{l Code}: {l Select All Code}

sudo apt update
sudo apt install build-essential zlib1g-dev libfreetype6-dev libjpeg62-dev libpng-dev libopenjp2-7-dev libmad0-dev libfaad-dev libogg-dev libvorbis-dev libtheora-dev liba52-0.7.4-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libavresample-dev libxv-dev x11proto-video-dev libgl1-mesa-dev x11proto-gl-dev linux-sound-base libxvidcore-dev libssl-dev libjack-dev libasound2-dev libpulse-dev libsdl1.2-dev dvb-apps libavcodec-extra libavdevice-dev libmozjs-52-dev

Then I tried building and installing with the *shudders* configure script. There is no CMake or scons.

{l Code}: {l Select All Code}

cd gpac
./configure
make
sudo make install

Half of the checks failed (big surprise there) but compilation worked perfectly. However, attempting to compose an image from a hevc results in failure.

{l Code}: {l Select All Code}

MP4Box -add-image "video.mkv":time=4841:primary -ab heix -new image.heic

I'm getting a lot of "does not start with sync marker". It's stressful using a tool that is so poorly documented. Supposedly this procedure is supposed to work.

[Lyberta]

Deleted.

[fluffrabbit]

FLAC is lossless and can store 16 bit CDDA samples. A lot of torrents are published as .cue+.flac so you can burn a perfect copy of original CD.

I must have confused "signed" with "floating-point". Yeah, you're right.

HEVC -> OpenEXR losslessness only depends if OpenEXR supports Rec. 2020 directly or have better colorspace that is a strict superset of Rec. 2020.

True, but this is a hypothetical. I don't know about OpenEXR. Going straight from Rec. 2020 HEVC to Rec. 2020 EXR probably requires using libx265 and an EXR encoder directly. ffmpeg is a middle man that kludges everything into whatever pixel format it feels like.

video programs

There's your problem. ffmpeg is optimized for spraying images onto an RGB screen with the "gotta go fast" philosophy.

And why would I bother with only demuxing keyframes?

Because that may be the only way to do it with the standard open source command line AV toolset at this point in time. Sure, it would be great to take a Rec. 2020 image as rendered by a video decoder and put those Rec. 2020 pixels into an EXR. You might have to write software for that.