Page 1 of 1

[Important] Introduction to PBR and New visual style

PostPosted: 22 Jul 2017, 17:06
by samuncle
Hi everyone :D

A lot of people already more or less know this on IRC but now I think it's time to write a proper topic on the Forum:

It has been seven years since I started to contribute to SuperTuxKart.
A lot of things happened during those year, SuperTuxKart had an impressive development and compared to what the game looked seven years ago it's day and night.

I also tried to give to SuperTuxKart it's own style with colorful hand painted textures. While I love the hand painted cartoon style there was a massive drawback.
It requires literally to hand paint EVERY SINGLE textures that are in the game. If you count roughly ~5 hours per textures times the number of textures that where made (around 370) it's an impressive 270 days (8h30 per day) spent on just painting and creating textures.

And you have to be aware those are the number without even taking into account
* Creating 3D model
* Creating new track design
* Integration into SuperTuxKart and tweaking.

A lot of things also happened in my private life, new responsibility, new job, I'm not a student anymore. I don't have as much free time as I had in the past.
I'm also afraid I'm one of the few if not the only artist working on a regular basis and with such limited man power I had to rethink how I want to handle this.
If we continue on this way it's not sustainable (or we might wait ~ years to get a new map).

While the previous visual style was fun to play with it had several issues
*We can't reuse objects made by other artist easily from open source repository like open game art
* We are forced to spend hours creating those textures
* The game reuse a lot of them since it takes so much time to create a new one and become a bit boring (the wood is the same everywhere, etc).

That's why for version post 0.9.3 I will change the Style of SuperTuxKart. We are going to use what I call colorful realism.
We are also going to switch to a PBR model in order to have better and easier to use / understand materials.
It also means that we can more easily share models between SuperTuxKart and other artist and they can create more easily content for the game.
Creating realistic texture is way easier and less time consuming. Now I will give in details what does it change for the workflow and explain how PBR works

Stay tuned

Re: [Important] Introduction to PBR and New visual style

PostPosted: 22 Jul 2017, 18:05
by samuncle
Why PBR ?

Our current pipeline from stk 0.9 to stk 0.9.3 is already pbr friendly. We use what we could call a mix between PBR with a traditional workflow.

In supertuxkart materials are defined by the following textures.
1) The diffuse map contain the main color and shades and pattern of the object in the diffuse component. (this is a RGB image)
2) The normal map contain the normal to add more details (this is also RGB image)
3) Then you can add: A specular map which define the color of the specular reflexion (currently it's a black and white image in stk, but should be RGB)
4) A gloss map to define the glossiness of the material (also black and white)
5) An emit map to define the part that emit light (currently it's a black and white image in stk, but should be RGB)

One of the limitation of the system is the inability to have colored specularity and colored emission (especially useful for metal surfaces). To have this we must have colored specular map/emit map, and it would increase the VRAM consumption.

With the new pipeline
PBR is easy to understand and apply if you know a few things.
1) Diffuse map becomes Albedo map. This contain the pure color of the material without any lighting, any effect. Effect and surface property are defined by the physical property in additional textures
2) Once we have the color we can use simple black and white image to define the physical property of the material.
-> Metal map (black and white) Will define the part that are a metal and the part that are insulators.
-> Rough map (black and white) Will define how smooth the material is
-> Emit map (black and white) Will define if the material emit light
3) The normal map doesn't change and will remain the same

With this we have a full description of all material with only three RGB textures (instead of minimum 5 textures in order to achieve the same result with the traditional workflow).
We keep the VRAM usage down and we got better material (amazing isn't it! :). We can replicate advanced things like metal surfaces that are very important in a racing game (for vehicles).
We also would get nice reflexion for free since our pipeline already support this.

Other engine who use also this workflow like the new Eevee viewer in blender, substance, cycle (with custom shader) etc, would produce a very similar result compared to what we would get in SuperTuxKart.

Optimization possible
This new shading technique comes with a few very interesting optimization. I will divide into two category of optimization. Non destructive optimization which won't alter the quality of the rendering and destructive which will influence the end result.

Non destructive optimization
The general case for a random generic object is:
RGB 8 Bit texture for Albedo map
RGB 8 Bit texture for Normal map
RGB 8 Bit texture for Rough/Emit/Metal map

Now if the object has NO metal part, we can discard the metal map and simply force the value to 0 and everything will be preserved
Also true if the object is only composed of metal, we can force this value to 1 and not use a metal map
If the object don't emit light, we can also safely not use an emit map (force it to 0)
Of course for all case in between, we need the full set of texture.
But this can reduce a lot of VRAM usage, especially for emission which is used only in very specific things like lanterns, etc

Destructive optimization
Since all map will be either 1k or 2k we can resize them on the fly if needed by the engine for low end GPU. (This is already done in SuperTuxKart)
We can also not load normal map to reduce even further the usage of VRAM. To be more aggressive it's also possible to not load the Rough/Emit/Metal map and save more.
We would loose high quality materials and detail but still run on low end GPU.

Re: [Important] Introduction to PBR and New visual style

PostPosted: 22 Jul 2017, 18:22
by samuncle
Technical details (for programmers and artists)

Albedo: the albedo texture specifies for each texel the color of the surface, or the base reflectivity if that texel is metallic. This is largely similar to what we've been using before as a diffuse texture, but all lighting information is extracted from the texture. Diffuse textures often have slight shadows or darkened crevices inside the image which is something you don't want in an albedo texture; it should only contain the color (or refracted absorption coefficients) of the surface.

Normal: the normal map texture is exactly as we've been using before in the normal mapping tutorial. The normal map allows us to specify per fragment a unique normal to give the illusion that a surface is bumpier than its flat counterpart.

Metallic: the metallic map specifies per texel whether a texel is either metallic or it isn't. Based on how the PBR engine is set up, artists can author metalness as either grayscale values or as binary black or white. (in the case of supertuxkart it will be mostly binary (black or white)

Roughness: the roughness map specifies how rough a surface is on a per texel basis. The sampled roughness value of the roughness influences the statistical microfacet orientations of the surface. A rougher surface gets wider and blurrier reflections, while a smooth surface gets focused and clear reflections. Some PBR engines expect a smoothness map instead of a roughness map which some artists find more intuitive, but these values get translated (1.0 - smoothness) to roughness the moment they're sampled.

There is also an ambient occlusion map, but STK already compute in real time ambient occlusion. We can ignore it (and improve instead our SSAO algorithm)
A more detailed example with the equation used is available here

I'm also creating a series of simple test to calibrate our shaders
I will upload them a bit later :)

Re: [Important] Introduction to PBR and New visual style

PostPosted: 22 Jul 2017, 22:47
by samuncle
Here is an object that can be used to calibrate the engine. It covers most of the cases:
stk_pbr_cal.jpg


Things to look for:
1) The black part is a simple rubber material. The color of the albedo map won't affect the color of the reflection. This color is only based on the light (sun, sky, etc) around the object
2) The red part is composed of metal. The color of the albedo map will affect the color of the reflection. This is only the case in metal part (hence the necessity of the metalness map)
3) The blue part is made of plastic. The color of the albedo won't affect the color of the reflection. This color is based on the light (just like the rubber part)
4) The green part is emissive (it emit light). Of course this is local only and won't affect the rest of the scene. In this case the color of the albedo map will be used to "emit" light

Let's have a look at the textures
The albedo is only composed of the color (without any effect/light/etc). As you can see it's super simple
suzanne_calibration_01_DefaultMaterial_BaseColor.jpg

The metal map is only a black and white texture that define the part where you have metal. In this case it's only the red part
suzanne_calibration_01_DefaultMaterial_Metallic.jpg

The rough map define how smooth the surface is
suzanne_calibration_01_DefaultMaterial_Roughness.jpg

The emit map define parts that emit light. As you can see there is no color since all the color is defined in the albedo map.
This is the only case where the material emit more energy than it receives. It also means that the color of the albedo will affect the color of the "bloom, lens flare, etc")
suzanne_calibration_01_DefaultMaterial_Emissive.jpg

The normal map doesn't change and it's usage remain the same.
suzanne_calibration_01_DefaultMaterial_Normal.jpg


Now in SuperTuxKart we can combine three black and white textures into one to simplify the pipeline (and take advantage of the existing structure)

Metal map = red value
Rough map = green value
Emit map = blue value
Here is all texture combined into one
suzanne_calibration_01_DefaultMaterial_STKPBRCombined.png

Re: [Important] Introduction to PBR and New visual style

PostPosted: 22 Jul 2017, 22:57
by samuncle
I provide a Zip file with the Albedo, the combined texture and the normal map texture.
You can experiment, try yourself, it's mainly useful for coders to implement the proper PBR equations and artist to check how it looks

https://samuncle.net/cloud/index.php/s/wqEZaJVnQBHzEKj

Re: [Important] Introduction to PBR and New visual style

PostPosted: 23 Jul 2017, 03:06
by leilei
samuncle {l Wrote}:Creating realistic texture is way easier and less time consuming.

I have to chime in and strongly disagree with this. You're scope-creeping yourself into a labor trap IMHO.

Re: [Important] Introduction to PBR and New visual style

PostPosted: 23 Jul 2017, 04:23
by MTres19
leilei {l Wrote}:
samuncle {l Wrote}:Creating realistic texture is way easier and less time consuming.

I have to chime in and strongly disagree with this. You're scope-creeping yourself into a labor trap IMHO.


Sam didn't include this in the post, but he's been trying a new program called Substance Painter which apparently makes making (realistic) textures much faster.

I really like the current at policy of No Photographs as Textures. (Except for certain exceptions). It's what has given STK a unique style that makes it stand out against the trend toward trying to make renderings as realistic as possible. The first thing I thought of when I saw the retextured lamp in this thread was that it belonged in some sort of first-person-shooter game, not our lovely, cartoony STK.

As I'm not an artist, I don't have much insight on the amount of work it takes to make a texture, but I'm certain that if Substance Painter can partially automate the creation of photorealistic textures, surely there has got to be some software, somewhere, that can do the same with our current art style. I'm not against moving to a PBR pipeline though. It seems possible that we could automate the conversion of the current gloss maps to PBR maps, and thus not have to maintain two separate pipelines. By file naming conventions, it's even easy to identify gloss maps because they're (nearly?) all suffixed with "_gloss".

Re: [Important] Introduction to PBR and New visual style

PostPosted: 23 Jul 2017, 06:55
by samuncle
Hi

1) It will take less time with realistic textures because there are Already tons of textures out there with the realistic art style. As I explained it's easier to take existing resources rather than painting everything yourself.

2) pbr has nothing to do with photorealism. Pbr is only one of the many ways to compute light based on a set of textures. Our current pipeline is almost already pbr.
We can keep the old shader around for legacy so we don't have to redo everything. It's not another pipeline just another shader material.

I will also investigate if it's possible to convert old asset into the new system without too much pain.

3) what I describe is the official supertuxkart art style. Nothing prevents you to do hand painted textures yourself for the map of your dreams.
You can do pbr handpainted textures. I would say it's even easier because you can paint frist the color and then the additional maps.

Let me create a map to see how it looks in our beloved game :)
Also don't forget I said I wanted colorful realism. Not photo realism. I still want the game to be colorful and cartoonish.

Re: [Important] Introduction to PBR and New visual style

PostPosted: 28 Jul 2017, 02:56
by XGhost
This looks like an interesting topic, and I haven't written in a while now. Still lurking around though ;)

The following are just some thoughts and comments of mine about the visual future of STK. You've not heard much from me the last few years and that's because I'm currently not able or willing to spend my time on STK. However, I'm working educationally and also professionally in the same territory. Even though I'm primarily engaged as a programmer, my only contributions to STK are of the artistic kind. I mostly base my opinions on what I've learned the last few years from way more experienced Game Designers as I am and all the amazing and talented people I have the privilege to work with every day.

samuncle {l Wrote}:A lot of things happened during those year, SuperTuxKart had an impressive development and compared to what the game looked seven years ago it's day and night.

Indeed!

samuncle {l Wrote}:I also tried to give to SuperTuxKart it's own style with colorful hand painted textures.

Which is definitely a reason for STKs massive artistic improvement. However, the kind of "hand painted" in STK is not equal to "hand painted". Our style is much more realistic oriented as drawn or painted textures I know from other developments. Take a "photorealistic" texture or go out and shoot your own ones, tile it and tweak it in any image editor, some kind of sponge or cell filtering, take some colors - paint slightly over it, tweak it in engine, bingo. Therefore, your following time calculation is beyond me. Texturing can be a large part of your artistic workflow, yes. But i think, it's not the first thing you want to spare time on. As you obviously know, kart and foremost track design, modelling and testing can soon get a lot more time consuming. But ok Sam, if you really want to help speed up the artistic workflow I won't have anything against it :) . Making it easier for artists texturing their assets, why not? There are plenty of ways to do that.

samuncle {l Wrote}:We are going to use what I call colorful realism.

...Please no...

samuncle {l Wrote}:(...) a PBR model (...) creating realistic textures (...)

Dammit, here we go! Let's join in to the endless list of unrecognized games that we'll never play because they look exactly like the unattractive run-of-the-mill wannabe crap we got offered from the games industry since the last 30 years. I'm being a little sarcastic of course, but still... There are many games which artistic character really glances, also through their realistic look, even if it's applied in an ironical way. But I can not see STK, or any other kart racing party game in this category. Oh gosh, when every asset in STK looks like the lamp post you posted some days ago. :eew: I really don't know what motivated you to follow a such generic design style.

I really appreciated your design decision to move to a more consistent, a little abstract but overall more forgiving art style, as you introduced the community about the new engine a few years ago. I still valued your courage to give this project a unique and identifiable style that perfectly matches the character of the game. Based loosely on the form follows function principle.
And I'm talking now only about the art style specific the textures. I have nothing against a full PBR workflow per se. If it is really a pay off in case of light, more flexibility for materials and better dealing with general hardware limitations, go for it! I just don't understand the necessity of it... I mean, nobody cares about the beautiful normals and reflections of the screws on your lamp post when driving with high speed trying to reach first place.

"Colorful realism" (a contradiction in terms) is not the solution to speed up the texturing workflow. An amazing cell shader would be. Imagine a cartoony look for example by giving the normals (including bump / normal maps) control over the behaviour of the cell shading to easily creating material structures. I wrote something in CG some months ago experimenting with this style but I don't have any image references on my current machine to show. The workflow was so simple because you didn't have to texture anything. Just using plain colors and wrapping some normal maps around it looked satisfactorily and unique.
Overall: artistic contribution is always a damn big amount of work. Track design and asset creation... These things, they take time. Switching art style won't make it any better.


Some clarifications:

samuncle {l Wrote}:It will take less time with realistic textures because there are Already tons of textures out there with the realistic art style. As I explained it's easier to take existing resources rather than painting everything yourself.

If a Lead Artist in the industry while working together with several professionals on a game would say something like texture mashing is fine, this very person would be fired instantly ;) . You will not find textures easily that matches your desired style... and realism is brutally unforgiving. So let's think about it, realism is the thing we know and live in since we were born. The look and character feels different from person to person, location to location. We are so used to it that we subconsciously recognizing when something is wrong, when something feels different or out of place. That's why you almost never just use premade textures. You carefully compose colors and materials. And this can fastly be way more time consuming than you think. And imagine if STK would get some more actively working artists. Every track would follow your "colorful realism" but still look different, something like photo collages?
Therefore, don't just dismiss critical comments - Seriously, leilei is perfectly right.

samuncle {l Wrote}:what I describe is the official supertuxkart art style. Nothing prevents you to do hand painted textures yourself for the map of your dreams.
You can do pbr handpainted textures. I would say it's even easier because you can paint first the color and then the additional maps.

But to get into the official STK package you have to follow this guideline, right?


I personally am not very happy with this announcement. But as always, I'm just adding my two cents here, the design decision is not upon me.

Let me create a map to see how it looks in our beloved game :)

I really encourage you to do that! I am always open and let myself proved wrong. Go ahead, I know you're up to amazing work! :cool:

Cheers
Julian