Uniform Evaluation Metrics
Published:
The following (work in progress…)
The following metrics are full-reference metics like the conventional PSNR and SSIM.
- FLIP
- LPIPS: git
- pu-PSNR
- ColorVideoVDP: Here is the Project Page and git repo.
- Intel’s CGVQM
Black Box Approaches
1. NVidia’s FLIP
FLIP is an excellent tool for visualizing and communicating errors in the rendered images, both for low dynamic range (LDR) and high dynamic range (HDR). Big thanks to the NVlabs for making the tool publicly available with the source code. The FLIP is a command line interface (CLI) tool. I am on Windows platform:
- First,
git clone --recursive https://github.com/NVlabs/flip.git, thencd flip - BUILD:
mkdir build cd build cmake .. cmake --build .After this, the
flip.exeshould be under build/Debug. Similar way this can be extended toDebugandReleaseversion. As per discussion,Releasemode performs better
// for Release
mkdir Release
cd Release
cmake -DCMAKE_BUILD_TYPE=Release ..
// for Debug
mkdir Debug
cd Debug
cmake -DCMAKE_BUILD_TYPE=Debug ..
- RUNNING
// go to Release/Debug folder and run .\flip.exe -r .\reference.png -t .\test.png // similarly .\flip.exe -r .\reference.exr -t .\test.exrVisually, color represents the error map.
Blackmeans no perceived error, andyellowrepresents high error. For more details, see the Technical Blog and the Flip Publication.
There is an UI version of FLIP implementation, named as FLOP. Find in git with the blog explaining every step clearly. However, I guess the repository is not well-updated (2025) and there are several bugs that need to be resolved.
