Uniform Evaluation Metrics

1 minute read

Published:

The following (work in progress…)

The following metrics are full-reference metics like the conventional PSNR and SSIM.

Black Box Approaches

1. NVidia’s FLIP

FLIP is an excellent tool for visualizing and communicating errors in the rendered images, both for low dynamic range (LDR) and high dynamic range (HDR). Big thanks to the NVlabs for making the tool publicly available with the source code. The FLIP is a command line interface (CLI) tool. I am on Windows platform:

  1. First, git clone --recursive https://github.com/NVlabs/flip.git, then cd flip
  2. BUILD:
    mkdir build
    cd build
    cmake ..
    cmake --build .
    

    After this, the flip.exe should be under build/Debug. Similar way this can be extended to Debug and Release version. As per discussion, Release mode performs better

// for Release
mkdir Release
cd Release
cmake -DCMAKE_BUILD_TYPE=Release ..

// for Debug
mkdir Debug
cd Debug
cmake -DCMAKE_BUILD_TYPE=Debug ..
  1. RUNNING
    // go to Release/Debug folder and run
    .\flip.exe -r .\reference.png -t .\test.png
    // similarly
    .\flip.exe -r .\reference.exr -t .\test.exr
    

    Visually, color represents the error map. Black means no perceived error, and yellow represents high error. For more details, see the Technical Blog and the Flip Publication.

There is an UI version of FLIP implementation, named as FLOP. Find in git with the blog explaining every step clearly. However, I guess the repository is not well-updated (2025) and there are several bugs that need to be resolved.