How to color grade HLG videos with ffmpeg - using LUTs

HLG (Hybrid Log Gamma) is a gamma curve (scheme) for encoding videos in hdr format developed by BBC and Japan's NHK for broadcasting. It provides wider dynamic range compared to the standard picture profile in sony camera. Moreover HLG format is backwards compatible with non-hdr display sets that support only sdr.

If you have a sony camera like ZV-E10 you will most likely see HLG gamma being used in Picture Profile 10. With HLG available color modes are BT.709 and BT.2020 with the latter being a larger color set. It should be noted that HLG is one of the hdr formats besides many others like Dolby Vision and HDR10/HDR10+.

In very simple terms you can think of hdr somewhat like:

HDR = High Dynamic Range (via PQ or HLG) + Wide Color Gamut (BT.2020 or BT.2100) + High Bit Depth (10 bit or 12 bit) + Higher brightness levels (nits)

Check the metadata information

The ffprobe command will scan the video file and give us useful information, like the codec, bitrate, chroma sampling, colorspace, aspect ratio, resolution, fps etc.

> ffprobe .\C0186.MP4 -hide_banner
[mov,mp4,m4a,3gp,3g2,mj2 @ 000002bafd9e0200] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1001
[mov,mp4,m4a,3gp,3g2,mj2 @ 000002bafd9e0200] st: 0 edit list 1 Cannot find an index entry before timestamp: 1001.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\C0186.MP4':
  Metadata:
    major_brand     : XAVC
    minor_version   : 16785407
    compatible_brands: XAVCmp42iso2
    creation_time   : 2023-08-29T04:50:23.000000Z
  Duration: 00:00:19.02, start: 0.000000, bitrate: 100561 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt2020nc/bt2020/arib-std-b67, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 96451 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2023-08-29T04:50:23.000000Z
      handler_name    : Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2023-08-29T04:50:23.000000Z
      handler_name    : Sound Media Handler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (rtmd / 0x646D7472), 1227 kb/s (default)
    Metadata:
      creation_time   : 2023-08-29T04:50:23.000000Z
      handler_name    : Timed Metadata Media Handler
      timecode        : 02:51:37:29
Unsupported codec with id 0 for input stream 2
>

The above is an HLG format video shot using a sony zv-e10 in picture profile 10 with gamma HLG and color mode BT.2020. Sony uses XAVC codec which is an advanced version of the h.264 and keeps the bitrate high at around 100 Mbps. The resolution is 3840x2160 which is 4K and the fps is 29.97 (30/1001) which is the same as 30 fps setting.

Note the term arib-std-b67 in stream 0:0 (video) details. It is commonly seen in HLG based hdr videos. For PQ (Perceptual quantizer) or HDR10/HDR10+ based hdr videos you would see something like smpte2084 instead. These are different EOTFs (electro-optical transfer function) used to increase the dynamic range of the luminance values of individual pixel to make them look more like how human eyes would see it.

Note: HLG1, HLG2, HLG3 are sony's variants of HLG.

The details of the gamma function is not present in the standard metadata as read by ffmpeg. Though it is present in the file and can be read by the vendor specific software.

For sony cameras there is a utility program called Catalyst Browser which is free to download. It can metadata of videos made using sony cameras.

Playing with ffplay

The first thing to do is to play the video using ffplay. If your display is not 4k resolution you will need to scale down the video rendering.

Note that i am specifying only the height in the scale parameter and the width shall be calculated automatically by ffmpeg.

ffplay .\C0186.MP4 -vf "scale=-2:960" -hide_banner

Here is another syntax to use the command where you specify the width only, and the height is calculated automatically

ffplay .\C0186.MP4 -vf "scale=width=1920:height=-2" -hide_banner

As you play an hlg video directly on a sdr display, like the usual computer monitors the video will appear washed out or very "dull" and dark. This is because the video file.

Most of the time we intend to export the final video in BT.709 colorspace because that is what most displays are capable of, ranging from smartphones, computers to televisions.

In order to work on an hlg hdr videos you have 2 options:

1. Use an HDR capable display, more importantly HLG capable display that displays things as it is meant to be.
2. Convert the video to BT.709 colorspace on the fly while editing. This requires a video editor capable of doing this by itself or else use a lut file that does the on-the-fly transformation allowing us to see the colors as intended.

For instance my LG 24GN650 is an HDR10 capable monitor, so it can display hdr videos from youtube quite well. However it cannot display HLG videos correctly, simply because the HDR10 uses SMPTE ST 2084 (PQ) EOTF, whereas HLG is uses HLG EOTF.

But there are monitors targeted towards color grading professionals that support HLG, Dolby vision and multiple HDR standards at the same time. These include "ASUS ProArt Display PA32UCX-PK" and "Apple Pro Display XDR". As you might have already guessed, these kind of monitors are expensive.

Using HLG/BT.2020 to BT.709/Gamma2.4 LUT

The simplest way to work with an hlg bt2020 hdr video on an sdr display is by applying a lut transformation which will adjust the colors to make them more suitable for sdr displays thereby allowing easier editing.

Note that a LUT file does not change the colors of the video right away, but just makes it look different to make it compatible with the lower gamut display so that we can edit it. This allows editing the video without losing color information.

This is something that most professional video editors would allow. Davinci Resolve has integrated support for hlg videos and automatically displays them in the correct manner.

LUTs can not only transform a video from one colorspace to another, but also from one gamma type to another. So basically its a gamma+gamut transform. If you want to learn indepth about how luts do this, i would highly recommend reading this article by Cullen Kelly.

If you have doubts about how LUTs

You can get a free hlg to bt709 lut that works quite well from this link:
https://nopixels.net/hlg-to-rec709/

This lut worked the best for me. Big thanks to the author (Pawel Olas). The lut file can be used with ffplay/ffmpeg and it would apply the color transformations on the fly.

Here is an example

ffplay .\C0186.MP4 -vf "scale=-2:1080,lut3d=nopixels_net_hlg2020_to_rec709.cube" -hide_banner -autoexit

The author also shows steps to properly work with hlg footage in davinci resolve and convert to bt.709.

To my surprise the free lut worked quite well and rendered very nice and decent looking visuals in the video. I tested the above lut with hlg, hlg1, hlg2 and hlg3 gamma settings. It seemed to deliver very realistic colors with hlg gamma, whereas with hlg1, hlg2 and hlg3 the colors were a bit over-saturated so i had to desaturate. But this would also depend on the ambient lighting and shooting setup.

Its quite disappointing to see that sony does not provide a free lut for hlg3 to bt709 conversion. Most display devices as of now including computer monitors, laptop screens and smartphones are still using bt.709 color space and any video creator will have to convert from bt.2020 to bt.709 for delivery. Even youtube serves bt.709 by default.

More HLG to BT.709 LUTs

A youtuber Mark Bennett also provides a free lut for hlg3 to bt.709 at this video and can be downloaded here. This particular lut transformed the colors, but kept the video dark overall, so i had to bump up the contrast a bit.

More free luts for hlg are available here:
https://gfxplugin.com/en/product/328/download-hlg-cinematic-lut-pack.html
https://godownloads.net/hlg-lut-pack-10-luts-free-download/

There are probably more paid luts than free ones. If you want something that is more reliable and well designed then check out the HLG luts from Leeming LUT.

Adjusting Brightness/Contrast/Saturation

Basic color grading starts with adjusting the brightness, contrast and saturation. Often times the video that comes right out of the camera is upto our expectations and we want to make it improve it.

The video could look under or over-exposed or look more bright and less sharp. With the eq filter we can adjust the contrast, brightness, saturation and gamma of the video.

Here is a quick example. I tried lowering the saturation a bit and increase the contrast and gamma. For me it made the video look a lot better.

ffplay .\C0186.MP4 -vf "scale=-2:1080,lut3d=nopixels_net_hlg2020_to_rec709.cube,eq=saturation=0.8:contrast=1.2:brightness=0.1:gamma=1.2" -hide_banner

Adjusting Color Temperature

Ffmpeg also has filter to adjust the color temperature to more warm or cooler. Here i try to make the colors look cooler (less yellowish and more blue). In certain setups warmer colors tend to give a better overall look to the video, whereas in other times cooler colors look better and clearer.

ffplay .\C0186.MP4 -vf "scale=-2:960,lut3d=nopixels_net_hlg2020_to_rec709.cube,eq=saturation=0.8:contrast=1.15:brightness=0:gamma=1.5,colortemperature=temperature=8000" -hide_banner

Taking screenshots

If you want to take screenshots at specific point in time with different settings to compare the output, here is a quick way to do it. The following command will apply the lut file and print a screenshot file.

ffmpeg -ss 00:00:10 -i .\C0186.MP4 -frames:v 1 -q:v 2 -vf "lut3d=nopixels_net_hlg2020_to_rec709.cube" output.jpg

With all the other settings the command would look something like this:

ffmpeg -y -ss 00:00:10 -i .\C0186.MP4 -frames:v 1 -q:v 2 -vf "lut3d=nopixels_net_hlg2020_to_rec709.cube,eq=contrast=0.8:brightness=0:gamma=1.2:saturation=0.8" screenshot.jpg

Final Export

Once you are done playing with various parameters like contrast, brightness, gamma etc, and found the best setting for the best looking video you can re-encode the video with the new settings and export to a new video file. Just replace ffplay with ffmpeg and provide and output file name.

Conclusion

The above examples are just for experiment and fun sake. Real color grading and correction is always done in video editor like davinci resolve or premiere pro. Command line tools help you gain a better understanding of how stuff works under the hood.