Perceptual Evaluation Metrics

1 minute read

Published:

White Box Approach

1. FovVideoVDP

Here is the Project Page and the git repo.

  • git clone --recursive https://github.com/gfxdisp/FovVideoVDP.git
  • Open Visual Studio Code and go to the terminal
    pip install torch
    pip install pyfvvdp
    pip install pyexr
    

2. ColorVideoVDP

Here is the Project Page and git repo.

Black Box Approach

1. NVidia’s FLIP

FLIP is an excellent tool for visualizing and communicating errors in the rendered images, both for low dynamic range (LDR) and high dynamic range (HDR). Big thanks to the NVlabs for making the tool publicly available with the source code. The FLIP is a command line interface (CLI) tool. I am on Windows platform:

  1. First, git clone --recursive https://github.com/NVlabs/flip.git, then cd flip
  2. BUILD:
    mkdir build
    cd build
    cmake ..
    cmake --build .
    

    After this, the flip.exe should be under build/Debug. Similar way this can be extended to Debug and Release version. As per discussion, Release mode performs better

// for Release
mkdir Release
cd Release
cmake -DCMAKE_BUILD_TYPE=Release ..

// for Debug
mkdir Debug
cd Debug
cmake -DCMAKE_BUILD_TYPE=Debug ..
  1. RUNNING
    // go to Release/Debug folder and run
    .\flip.exe -r .\reference.png -t .\test.png
    // similarly
    .\flip.exe -r .\reference.exr -t .\test.exr
    

    Visually, color represents the error map. Black means no perceived error, and yellow represents high error. For more details, see the Technical Blog and the Flip Publication.

There is an UI version of FLIP implementation, named as FLOP. Find in git with the blog explaining every step clearly. However, I guess the repository is not well-updated (2025) and there are several bugs that need to be resolved.