The vertex shader is relatively simple once the required conditions to the good work are met. Like any technological progress
in the world of 3D graphics controllers, vertex displacement mapping has some little limitations which, if they are not taken into account,
will transform your Ultra-Boosted 7800 GTX into an old S3 trio (do you remember...).
To achieve vertex displacement mapping, there are two main constraints:
- the displacement map must be in floating point format. In OpenGL terms, that means an internal pixel format that is set to GL_RGBA_FLOAT32_ATI
(valid on ATI as well as on nVidia).
- the displacement map must not be filtered: no linear or trilinear filtering. Nearest mode is the only one accepted.
Thus once the displacement map is loaded in floating point format with nearest filtering, displacement mapping
becomes really simple as shown in the following vertex / pixel shader:
uniform sampler2D displacementMap;
gl_TexCoord.xy = gl_MultiTexCoord0.xy;
dv = texture2D( displacementMap, gl_MultiTexCoord0.xy );
df = 0.30*dv.x + 0.59*dv.y + 0.11*dv.z;
newVertexPos = vec4(gl_Normal * df * 100.0, 0.0) + gl_Vertex;
gl_Position = gl_ModelViewProjectionMatrix * newVertexPos;
uniform sampler2D colorMap;
gl_FragColor = texture2D(colorMap, gl_TexCoord.xy);
We fetch the displacement map the same way as we do for texture fetching in the pixel shader: using
the texture2D() GLSL function.
The Demoniak3D demo shows the deformation of a 80000-polygons mesh with a simple BMP image:
Fig. 4 - DEMO_displacement_mapping.xml
The nearest filtering mode is the only mode available in the vertex shader of today's hardware. But nothing can prevent
us from implementing our own version of the bilinear filtering mode. Here is the function (adapted from nVidia's Cg code) that does
#define textureSize 256.0
#define texelSize 1.0 / 256.0
vec4 texture2D_bilinear( uniform sampler2D tex, vec2 uv )
vec2 f = fract( uv.xy * textureSize );
vec4 t00 = texture2D( tex, uv );
vec4 t10 = texture2D( tex, uv + vec2( texelSize, 0.0 ));
vec4 tA = mix( t00, t10, f.x );
vec4 t01 = texture2D( tex, uv + vec2( 0.0, texelSize ) );
vec4 t11 = texture2D( tex, uv + vec2( texelSize, texelSize ) );
vec4 tB = mix( t01, t11, f.x );
return mix( tA, tB, f.y );
The use of this function is really simple. Just remplace texture2D() by texture2D_bilinear():
dv = texture2D_bilinear( displacementMap, gl_MultiTexCoord0.xy );
This code works fine in both vertex and pixel shaders.