Downsample-then-Upsample
Table of Contents
Experiments⌗
I tried out a modern approach to image resampling: an end-to-end learned downsampler-then-upsampler. The application would be to downsample the image, compress it using a traditional image/video compression method, and then upsample the image back to its original size. This might be useful as part of a video pipeline.
This project ( code = https://github.com/sunwj/CAR paper = Learned Image Downscaling for Upscaling using Content Adaptive Resampler ) was found by looking at paperswithcode.com, where it presently ranks as the best method by far.
Here are some other example results you can zoom into:
When there are single-pixel features, it does struggle as you’d expect. But it does a good job preserving features that are ~2 pixels or larger.
Windows Setup⌗
To get this going on a Windows PC with an Nvidia graphics card, I installed:
- Rapid Environment Editor - Used to easily edit the PATH variable.
- Git Bash
- CUDA
- Latest Python 3
- Ninja
Make sure CUDA is in the Windows Path. For me I added: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin and C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\libnvvp.
Make sure Python3 is in the Windows Path. For me I added: C:\Users\leon\AppData\Local\Programs\Python\Python39 and C:\Users\leon\AppData\Local\Programs\Python\Python39\Scripts.
To install Ninja, I dropped the latest ninja.exe release file into the Python39 folder.
Then open up Git Bash and clone the repo and set up dependencies:
git clone [email protected]:sunwj/CAR.git
cd CAR/adaptive_gridsampler
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f
pip install tqdm
pip install scipy
pip install numpy
pip install matplotlib
pip install scikit-image
python setup.py build_ext --inplace
cd ..
I then downloaded the pre-trained models from the link at https://github.com/sunwj/CAR and placed the files under a subfolder.
Now it’s ready to test! The scripts by default support only .PNG losslessly compressed images.
python run.py --scale 4 --img_dir catid_images --model_dir models --output_dir output