Out of curiosity I used RenderDoc to dig a bit into Serious Sam 4. The first frame we are going to dissect is the following:
The game starts by rendering the shadow maps. Only the sun has shadows, 4 cascades are rendered to separate textures, the size goes from 2048 to 5120 depending on the settings.
Whether the terrain is rendered or not also depends on the settings.
The first pass on the scene has two render targets. After every opaque element of the scene has been drawn, they look like this:
On lower settings one of the buffers looks like this:
So one of them is clearly the depth buffer. I’m not sure about the other one. We will see how the various elements are drawn later.
For each light in the scene a drawcall is made. The drawcall draws a sphere (made of 960 vertices) around the light, in the pixel the actual light calculation is performed and stored in a RGB16 HDR buffer. The only input to those two buffers are the two buffers generated in the pre pass. Normals are probably reconstructed from depth, materials attributes and normal maps have therefore no way of influencing this pass.
Sphere for green light
Main scene pass
During this pass all of the elements from the first pass are redrawn plus transparent objects and particles. Not all of them are that interesting, so let’s just talk about the most interesting ones.
Instanced rendering is used to draw the various sections
These slides go into some detail on how the terrain system works. In any case, the terrain is drawn in chunks:
A vertex buffer contains the following mesh
which the vertex shader transforms in the fhe final terrain mesh for the current chunk.
Chunk size can vary based on distance and required resolution.
Finally, let’s dig right in to the legion system. Closer enemies and NPCs are rendered normaly in several drawcalls:
Interestingly, the gun for this NPC has 58,341 triangles, despite only covering a couple tens of pixels. Next, for far away enemies an impostor mesh is generated, containing a quad for each enemy.
Their textures are stored in an atlas that seems to contain animations frames as well as various orientations.
In just one drawcall tens of thousand of distant enemies are drawn. The process is repeated for different types of enemies.
The vertex buffer is probably uploaded from the CPU at every frame.
For the sky a cube is rendered in 6 separate drawcalls (one for each side):
The sky itself is stored in a 2D texture.
First a textured quad with alpha blending is rendered:
Then a bumpy circle mesh is rendered multiple times with different textures:
In 3 passes the portal effect is rendered:
This one does distorsion:
Particles are rendered in a single drawcall. I suppose the vertex buffers and particle simulations are done in the CPU.
At this point, in order to create a bloom effect, the frame buffer is downcaled and upscaled multiple times. During the first upscale the dark parts of the buffer are rendered as black.
Various upscaling passes are omitted. At the end a 408×230 image is obtained.
Mixing and tone mapping
So far the image has been renderd into an HDR render target. This pass does tone mapping and mixes the result with the bloom buffer:
At this point the UI is rendered on top of the LDR buffer, and we get the final frame:
During the second pass the pixel shader calculates all of the lighting. Let’s see how it works by dissecting a different frame:
To be precise we are going to look at the shader inputs for one of the terrain patches:
Some of the shader inputs are buffers or black images, but some have a much clearer purpose. This one looks like albedo:
Some kind of normal map:
One of the faces of a cubemap (used for specular reflections):
A texture that seems to contain a light map:
This is the lightmap for a different model:
Sun shadow map:
Let’s also see the shader inputs for the gun:
Not sure about this one, probably contains several material attributes in various channels:
Sun shadow map:
The smoke is rendered as particles. They work like the ones we’ve seen before but those are lit. The shader has (among other things) those two images as input.
The game doesn’t make use of many compute shaders, instead many passes just render a quad to fill the entire screen. Some compute shaders are used at the very beginning for a purpose that I haven’t managed to figure out.
Overall the rendering is mainly done like in previous games of the series and has nothing special, but despite that
the overall look is pleasing to the eye.
Despite the simplicity of the rendering pipeline the game requires a pretty good PC to run properly.
(l’articolo in inglese pone il problema di come rispondere…. Tra l’altro non il top per indicizzazione su Google)
Lavoro interessante. Una cosa importante: quando fai queste robe, abituati a loggare e specificare quanto piu’ possibile del tuo setup, quindi: HW, OS e versione, driver e versione, e il top sarebbe anche salvare e condividere la trace di Renderdoc… La CG ha gia’ un grande problema di reproducibility ed e’ in atto uno sforzo collettivo per provare a risolvere la cosa anche (soprattutto) sul versante della ricerca, abituati ad andare sempre in questo senso.
Per il resto, non ho realmente letto tutto, pero’ mi ha fatto strano l’uso di FXAA, che immagino che tu abbia letto essere usato da qualche parte, o direttamente nelle opzioni del gioco, visto che con Renderdoc su binario attached non si puo’ vedere lo shader code, o mi sbaglio? (e’ parecchio che non uso RD 😃). Perche’ guardando il resto della pipeline, non vedeo nulla che realmente “obblighi” a fare AA in post-processing, tipo pipeline molto deferred, o decals etc. Sarebbe interessante vedere se la pipeline cambia selezionando MSAA (Google dice che si puo’ in quel gioco)…
Va benissimo se rispondi in italiano, ho scritto in inglese così da poter pubblicare l’articolo altrove e poter farlo leggere a chi non conosce l’inglese.
Il mio seup è un po complicato, ho usato una VM con passthrough della GPU. L’OS nella VM è windows 10, la versione del driver 20.9.1
L’host ha il seguente HW:
- CPU: I7 3770
- RAM: 16GB ddr3 1600Mhz
- GPU: RX580 8GB
Ho fatto svariati capture, alcuni in dx11, altri in dx12. Provando a fare capture di Vulkan RenderDoc crashava.
Usando dx11 lo shader che fa FXAA ha questo nome:
RenderDoc permette di mostrare l’assembly ed il DXIL dello shader.
Nei preset del gioco FXAA è sempre abilitato ma è possibile controllare MSAA ed FXAA indipendentemente nelle opzioni avanzate.
Nel post non ho parlato di SSAO, che è “nascosta” nel canale alpha di una tetxture RGBA con nulla negli altri canali.
Potrei fare un altro post on cui faccio un confronto con titoli precedenti (che si comportano in modo molto simile).
I did some more digging and found a few more things:
Explaining the colorfull buffer
I’m talking about his one to be precise:
After having a look at the shader’s assembly code it turns out it encodes depth. I actually suspected that but I wasn’t too sure. Still no clue on why it’s done this way.
SSAO is stored in the alpha channel of an RGBA8 texture. The RGB part is white everywhere except for the sky which is black.
It is done by a pixel shader invocated on a screen filling quad. inputs to it are depth and a noise texture.
Somewhere it gets filtered and then mixed in in the second scene pass.
Weird point light
Point lights in this game are usually fairly limited but this one was doing a bit more than usual:
First it gets rendered into the light buffers just like any other lamp. The intensity of the light is varied according to a cube map, that’s why it seems like the barrel casts a shadow, in reality it does not. Earlier titles of the series had that too.
Now, during the second pass on the scene, this model get’s rendered:
Yes, that’s a piece of geometry from the level…
A depth cube map is given as an input, one of the faces looks like this:
And this is what gets rendered in the hdr buffer:
Now that’s weird, why aren’t shadows calculated in the first light pass? Is the same shadow map applied to all lights?
Is the light buffer ignored for this model? does it substract the light contribution on shadowed parts? I don’t know but perhaps I might look into the shader and update this post.
You might have noticed that the floor hasn’t been rendered yet. Well not to worry because it’s goin to be rendered not once but three times now.
I can’t tell the difference between the first two passes, they both look like this:
(notice that contribution from other point lights is taken into account)
But on the third pass specular light from that light source is added
At this point this geometry get’s drawn:
With this texture as an input to the pixel shader:
The result isn’t too bad:
Thanks to a user who created a custom map for me I was able to better understand how the different types of point lights are handled.
There are mainly two types:
- Dynamic omni lights
- Fast dynamic omni lights
The custom map is divided in two rooms with each containing two lights per type
Fast omni lights
First they are rendered using the usual sphere
Normal maps aren’t taken into account
After that a fragment shader ran on a quad filters the ssao buffer and stores the result in the alpha channel of a texture that otherwise looks blue.
That alpha channel gets copied (using a fragment shader on a quad) to the alpha channel of the light buffer.
Now I’ve chosen a part of the scene geometry that is lit by all lights.
It is rendered two times, first the contribution of the fast lights plus one of the dynamic lights is written to the hdr buffer.
Notice the shadow in the back of the short hallway.
The alpha channel of the lights buffer is multiplied with the other channels.
Essentially this applies SSAO to the lighting.
It is then rendered once again in order to add the second dynamic light’s contribution
Notice there are two shadows now.
Normals are taken into account for dynamic lights
But since fast lights are calculated before the normal map is ever read from the material, it isn’t taken into account for them