Investigating neural network upscaling for STK textures

Investigating neural network upscaling for STK textures

Postby Calinou » 05 Jan 2020, 23:43

Hi,

I've recently toyed with ESRGAN and SFTGAN to upscale textures in old games. This is how I ended up releasing a neural upscaled texture pack for Quake 2 a few months ago :)

I've gotten pretty decent results above by also using a denoiser on the output – I used GIMP's noise reduction filter for this. It tends to look better for cartoony textures overall.

Some textures in SuperTuxKart (but not all) tend to be a bit blurry, especially on higher resolution displays. Would the developers be interested in me providing upscaled variants for those? I've noticed a few in specific tracks already, but having a more complete list would be even better. Of course, we need to balance this with the distribution size, but we can probably get substantial improvements in graphics quality with a moderate increase in download size.
User avatar
Calinou
 
Posts: 164
Joined: 22 Jan 2010, 21:43
Location: France

Re: Investigating neural network upscaling for STK textures

Postby tempAnon093 » 06 Jan 2020, 01:44

[I'm not a dev]
Can't hurt to try. If they look better, then I see no reason we wouldn't add them!

It's worth mentioning the Game Upscale Discord room (if you aren't already a member; I didn't check), there are some ERSGAN models which may work well for cartoony stuff like DigitalFrames.
tempAnon093
 
Posts: 165
Joined: 02 Feb 2019, 12:09

Re: Investigating neural network upscaling for STK textures

Postby drummyfish » 06 Jan 2020, 20:24

This is a great topic.

I did some pixel art upscaling in a similar way, just didn't use neural nets but classical pixel art upsaling algorithms found in emulators etc. That created a good initial version which I then manually adjusted in GIMP. (BTW if someone wanted to use machine learning to create algorithms for a better automatic pixel art upscaling, I'd be very interested.)

Image

I think fore the best quality you always have to manually edit the result of automatic upsace, even that generated by a good neural net. The amount of manual work will of course be much smaller than complete upscale by hand.

With machine learning such as the programs you used I see a few potential issues and dangers with copyright -- I don't know if that's the case, but I suppose that if a non-public domain dataset is used to train the net, then the images it will generate may end up being a derivative work of a copyrighted work, i.e. the images in the dataset, so they may be either non-free, or may be burdened by the dataset license. This probably depends on each specific case, but I think the danger is there. Does anyone know anything about this?

Also compiling and using all the these machine learning tools is not very easy, I remember having had hard time setting up a style transfer program and in the end was only able to use the SW (non-GPU) implementation, because I didn't want to use any proprietary drivers and technologies, so the program was slow and unusable. There are web tools but they insert watermarks and have various ugly conditions on what you can do with the results etc. Does any simple, good, offline, fast and free as in freedom tool exist for this? With such tools more artist would definitely do this.
socialist anarcho-pacifist
Abolish all IP laws. Use CC0.
http://www.tastyfish.cz
User avatar
drummyfish
 
Posts: 252
Joined: 29 Jul 2018, 20:30
Location: Moravia

Who is online

Users browsing this forum: No registered users and 1 guest