My Coding > Artificial Intelligence > Image improvement with AI > Image upscaling with ESRGAN/PyTorch

Image upscaling with ESRGAN/PyTorch

What is Upscaling

The very common problem with images - they do have not enough resolution for required purposes. This can be anything. Old pictures from poor cameras or films. pictures are taken from a very big distance or scientific images with principal impossibility to make a better resolution. And in all cases, it will be beneficial to improve the resolution of these images.

ESRGAN

It is possible to upscale images by using Neural Network algorithms. One of the very well-developed systems nowadays is an ESRGAN, based on the Python library PyThirch. ESRGAN not only mechanism of image upscaling but also have already trained NN with weights adopted for general-purpose images. For your own tasks with special images, it is better to train your own weights in ESRGAN.

Furthermore, there is a big community, that develops its own, models for ESRGAN for different purposes, which gives better results for some selected types of images, for example, "anime images", "nature images", "textures", etc. See Upscale wiki for more details.

In this tutorial, I will show you how to use a basic model with ESRGAN to upscale general images and will do it for the Linux/Ubuntu platform.

Installing ESRGAN

First of all, it is necessary to make sure that you have < href="http://mycoding.uk/a/python.html">Python 3 installed.

Then it is necessary to install the following libraries: PyTorch >= 1.0 (CUDA version >= 7.5 if installing with CUDA), numpy and opencv-python.

In the command prompt - go to the directory where you would like to create your project directory and clone GitHub for ESRGAN:


git clone https://github.com/xinntao/ESRGAN
cd ESRGAN

when the GitHub data will copy - you need to download the trained model and place it into a folder "./ESRGAN/models/". The trained official models are available at the Google drive.

All images for processing will be taken from the "./ESRGAN/LR/" folder and all results will be stored in a "./ESRGAN/results/" folder.

Then it is necessary to check the python file test.py to make sure that the model has the correct name, and all paths are correct. Also make sure that it is prepared to work with GPU, rather than with CPU. CPU will be too slow for any reasonable applications. so, in fact, you need to check only these four lines:


model_path = 'models/RRDB_ESRGAN_x4.pth' # path to model
device = torch.device('cuda') # 'cuda' or 'cpu'
test_img_folder = 'LR/*' # will read all images
    cv2.imwrite('results/{:s}_rlt.png'.format(base), output) # writing

And then run it with the command


python test.py

As simple as that. You can also monitor the load of your GPU with the command


watch nvidia-smi

You can watch a video, on how to use ESRGAN code to see this work in real-time.

Video Upscaling

Video upscaling is the same problem as image upscaling. To upscale video, it is necessary to split it into images, where every image will correspond to one video frame. Then upscale every image individually and then link new upscaled images into one video. How to make a video from images with Python, I already explain in this tutorial.

But it is necessary to understand, that video can have a lot of frames and upscaling can take a very long time, therefore the computer equipment should be very powerful and the presence of a GPU is absolutely crucial.


Published: 2023-06-09 22:53:28
Updated: 2023-06-09 23:08:16

Last 10 artitles


9 popular artitles
© 2020 MyCoding.uk -My blog about coding and further learning. This blog was writen with pure Perl and front-end output was performed with TemplateToolkit.