Tuesday, 5 January 2016

Basic lighting (part 14)

We're slowly getting through the boring stuff, next up is basic lighting. I'm keeping this simple with a single light source for now, we'll look at more complex lighting at some later time.

First lets go through a tiny bit of theory. Our basic lighting model will implement three types of lighting: ambient, diffuse and specular lighting.
This type of shading is known as Phong Shading

The ambient part of our lighting model is the easy part. Imagine if you will you are in a dark room and you turn on a single light that illuminate the entire room. Now hold up an object like a book and look at the side that is not facing the light. Even though no light hits the object directly from our light source you would expect it to be completely in shadow and black. Yet you'll still see its colors. This is because the light in the room bounces off all the walls and other surfaces and the object becomes lighted indirectly.
To accurately calculate this is a complex task but in computer graphics we take a rather simple shortcut by stating the object is always illuminated by a fixed amount.

The diffuse part of our lighting model requires a bit more calculation. When light hits a surface it scatters. Depending on the type of surface it can be reflected in all directions relatively evenly (and give a nice solid appearance) or be reflected in roughly a single direction (making it a mirror). More complex surfaces might even react differently for different wave lengths.
For our lighting we assume the first situation, the light gets reflected in all directions evenly. The intensity of the light being reflected is a factor of the angle at which the light hits the surface from reflecting no light if the light travels parallel to the surface to full intensity if the light hits the surface at a perfect 90 degree angle.
In order to calculate this we use the normal vector of the surface. The normal vector is a vector that is perpendicular to the surface and we simply calculate the angle between the normal vector and the vector pointing from the surface to the light source. By taking the cosine of this angle we have a great variable that is 1.0 at full illumination, 0.0 when the light travels parallel to our surface and is negative if the light lies behind the surface and we're thus in shadow.
Luckily for us calculating the cosine of a surface is incredibly simple as this happens to be the dot product of two unit vectors.  So we normalize our normal vector and light direction vector and call this handy function called dot().

The specular part of our lighting is the most complex one presented here. Here we are looking at light reflecting off the surface in a specific or narrow direction. If the light reflects towards our viewpoint we see its reflection. Our normal vector again is used but now to determine the vector of the light reflected off of our surface, we have a nice function called reflect() for this.
We again calculate a cosine but now using our reflected light vector and a vector from our eye to the surface.

Because our specular calculation needs to calculating a vector from our eye/camera/viewport it makes sense to do all lighting calculations after we apply our model and view matrices but not our projection matrix. As long as we also adjust the position of our light using our view matrix we're all set.

Normal vectors


As we discussed above we need the normal vector of our surface to perform our lighting calculations. For our cube this is incredibly simple, as each face of our cube is a flat surface the normal vector applies to the entire face and can be easily calculated using the cross product of two edge vectors of our face.

But for more complex shapes this becomes a lot more complex. Also when we look at curved surfaces, even though we're rendering them with flat triangles, we can interpolate the normals between each vertex to create the illusion of a curved surface (we'll see an example of this later). For this reason OpenGL assumes we store a normal for each vertex and just like with texture mapping we have to duplicate vertexes when used for different faces with different normals.

As all this can be relatively complex it makes a lot of sense to calculate all the normals one time and store them for each vertex. To be exact, most 3D modeling software does this for us and handily store all the normals along with the model.

I've adjusted our vertex structure and vertex array to include our normals for our cube:
// we define a structure for our vertices
typedef struct vertex {
  vec3    V;          // position of our vertice (XYZ)
  vec3    N;          // normal of our vertice (XYZ)
  vec2    T;          // texture coordinates (XY)
} vertex;

vertex vertices[] = {
  // front
  -0.5,  0.5,  0.5,  0.0,  0.0,  1.0, 1.0 / 3.0,       0.0,          // vertex 0
   0.5,  0.5,  0.5,  0.0,  0.0,  1.0, 2.0 / 3.0,       0.0,          // vertex 1
   0.5, -0.5,  0.5,  0.0,  0.0,  1.0, 2.0 / 3.0, 1.0 / 4.0,          // vertex 2
  -0.5, -0.5,  0.5,  0.0,  0.0,  1.0, 1.0 / 3.0, 1.0 / 4.0,          // vertex 3

  // back
   0.5,  0.5, -0.5,  0.0,  0.0, -1.0, 1.0 / 3.0, 1.0 / 2.0,          // vertex 4
  -0.5,  0.5, -0.5,  0.0,  0.0, -1.0, 2.0 / 3.0, 1.0 / 2.0,          // vertex 5
  -0.5, -0.5, -0.5,  0.0,  0.0, -1.0, 2.0 / 3.0, 3.0 / 4.0,          // vertex 6
   0.5, -0.5, -0.5,  0.0,  0.0, -1.0, 1.0 / 3.0, 3.0 / 4.0,          // vertex 7
   
   // left
  -0.5,  0.5, -0.5, -1.0,  0.0,  0.0, 1.0 / 3.0, 1.0 / 4.0,          // vertex 8  (5)
  -0.5,  0.5,  0.5, -1.0,  0.0,  0.0, 2.0 / 3.0, 1.0 / 4.0,          // vertex 9  (0)
  -0.5, -0.5,  0.5, -1.0,  0.0,  0.0, 2.0 / 3.0, 2.0 / 4.0,          // vertex 10 (3)
  -0.5, -0.5, -0.5, -1.0,  0.0,  0.0, 1.0 / 3.0, 2.0 / 4.0,          // vertex 11 (6)

  // right
   0.5,  0.5,  0.5,  1.0,  0.0,  0.0, 1.0 / 3.0, 1.0 / 4.0,          // vertex 12 (1)
   0.5,  0.5, -0.5,  1.0,  0.0,  0.0, 2.0 / 3.0, 1.0 / 4.0,          // vertex 13 (4)
   0.5, -0.5, -0.5,  1.0,  0.0,  0.0, 2.0 / 3.0, 2.0 / 4.0,          // vertex 14 (7)
   0.5, -0.5,  0.5,  1.0,  0.0,  0.0, 1.0 / 3.0, 2.0 / 4.0,          // vertex 15 (2)

  // top
  -0.5,  0.5, -0.5,  0.0,  1.0,  0.0,       0.0,       0.0,          // vertex 16 (5)
   0.5,  0.5, -0.5,  0.0,  1.0,  0.0, 1.0 / 3.0,       0.0,          // vertex 17 (4)
   0.5,  0.5,  0.5,  0.0,  1.0,  0.0, 1.0 / 3.0, 1.0 / 4.0,          // vertex 18 (1)
  -0.5,  0.5,  0.5,  0.0,  1.0,  0.0,       0.0, 1.0 / 4.0,          // vertex 19 (0)

  // bottom
  -0.5, -0.5,  0.5,  0.0, -1.0,  0.0, 2.0 / 3.0,       0.0,          // vertex 20 (3)
   0.5, -0.5,  0.5,  0.0, -1.0,  0.0, 3.0 / 3.0,       0.0,          // vertex 21 (2)
   0.5, -0.5, -0.5,  0.0, -1.0,  0.0, 3.0 / 3.0, 1.0 / 4.0,          // vertex 22 (7)
  -0.5, -0.5, -0.5,  0.0, -1.0,  0.0, 2.0 / 3.0, 1.0 / 4.0,          // vertex 23 (6)
};
Note that we change our attributes so that attribute 0 remains our position, 1 becomes our normal and 2 is now our texture coordinate.

Matrices


First lets revisit our matrices for a minute. So far we've calculated our model-view-projection matrix as that is all we needed but for our light calculations we need a few more. We need our model-view matrix and we need what is called our normal matrix.

The normal matrix is a matrix that only applies our rotation and can be used to update the direction of our normal vectors.

Because we adjust everything to our view matrix we also apply our view matrix to our normal matrix. The easiest way to get your normal matrix is to take your model-view matrix and take the inner 3x3 matrix from it. This is what I normally do but it does create some issues when your model applies a non-uniform scale. Now I read somewhere that the solution is to take the inverse of your matrix and then transpose it. Since we may need the inverse of our matrix later on I've gone down this route for now but I may revert to just using the model-view matrix directly as calculating the inverse of a matrix is costly as I rarely use non-uniform scaling anyway.

Anyway, because we'll be doing these calculations a lot I've added a structure to shader.h that can store IDs for the most applicable matrices:
// typedef to obtain standard information, note that not all ids need to be present
typedef struct shaderStdInfo {
  GLuint  program;
  GLint   projectionMatrixId;       // our projection matrix
  GLint   viewMatrixId;             // our view matrix
  GLint   modelMatrixId;            // our model matrix
  GLint   modelViewMatrixId;        // our model view matrix
  GLint   modelViewInverseId;       // inverse of our model view matrix
  GLint   normalMatrixId;           // matrix to apply to our normals
  GLint   mvpId;                    // our model view projection matrix
} shaderStdInfo;
There is also a function called shaderGetStdInfo() that populates this structure.
Finally I've added a function called shaderSelectProgram() that binds the shader program referenced by our structure and then calculates and applies all our matrices from a model, view and projection matrix that is passed to it.

What is very important to know is that GLSL will remove any uniform that isn't actually used in the source code so there is no use defining say modelViewInverse if you're not using it.
While the code logs that it doesn't exist shaderSelectProgram() simply skips those.

There is a way to use a VBO to load all matrices in one go which I may look into at a later date.

Our load_shaders function in engine.c now calls our shaderGetStdInfo function. We still have two uniforms that fall outside of our structure: our light position and our texture sampler (and there are several other uniforms in the shader.

Our engineRender function similarly now calls shaderSelectProgram. 

Our new shaders


The real magic however happens inside of our shaders. The changes to our vertex shader are very straight forward. First we now have an attribute for our normals.
We also have 2 new outputs, one for our position (V) and one for our normal (N). For both the correct matrix is applied.

Our fragment shader has grown substantially, we'll look at each part individually:
#version 330

// info about our light
uniform vec3      lightPos;                         // position of our light after view matrix was applied
uniform float     ambient = 0.3;                    // ambient factor
uniform vec3      lightcol = vec3(1.0, 1.0, 1.0);   // color of the light of our sun

// info about our material
uniform sampler2D boxtexture;                       // our texture map
uniform float     shininess = 50.0;                 // shininess

in vec4           V;                                // position of fragment after modelView matrix was applied
in vec3           N;                                // normal vector for our fragment
in vec2           T;                                // coordinates for this fragment within our texture map
out vec4          fragcolor;                        // our output color
We have a couple of new uniforms. Note that we've added default values for a couple of them so you can set them from code but don't have to:
  • lightPos - the position of our lightsource with view matrix applied to it
  • ambient - our ambient factor, our default is 30%
  • lightcol - the color of our light source, white for now
  • boxtexture - we had this one already, our texture sampler
  • shininess - the shininess for our specular lighting
Our input variables also match the output variables of our vertex shader.
void main() {
  // start by getting our color from our texture
  fragcolor = texture(boxtexture, T);  
  if (fragcolor.a < 0.5) {
    discard;
  };
  
  // Get the normalized directional vector between our surface position and our light position
  vec3 L = normalize(lightPos - V.xyz);
We start by getting our color from our texture map as before. Then we calculate vector L as a normalized directional vector that points from our surface to our lightsource. This vector we'll need in both our diffuse and specular calculations.
  // We calculate our ambient color
  vec3  ambientColor = fragcolor.rgb * lightcol * ambient;
This is our ambient color calculation, we simply multiply our surface color with our light color and our ambient factor.
  // We calculate our diffuse color, we calculate our dot product between our normal and light
  // direction, note that both were adjusted by our view matrix so they should nicely line up
  float NdotL = max(0.0, dot(N, L));
  
  // and calculate our color after lighting is applied
  vec3 diffuseColor = fragcolor.rgb * lightcol * (1.0 - ambient) * NdotL; 
For our diffuse color we first calculate our dot product for our normal and light vector.
We then multiply our surface color with our light color and our dot product taking out the ambient factor we've already used.
  // now for our specular lighting
  vec3 specColor = vec3(0.0);
  if ((NdotL != 0.0) && (shininess != 0.0)) {
    vec3 R = reflect(-L, N);
    float VdotR = max(0.0, dot(normalize(-V.xyz), R));
    float specPower = pow(VdotR, shininess);
  
    specColor = lightcol * specPower;
  }; 
For our specular highlight we first calculate our reflection vector then calculate our dot product using our position.
Finally we apply our shininess using the power function and multiply the outcome with our light color. The higher the shininess value the smaller our reflection.
We only do this if we have a shininess value and if we're not in shadow.
Note that we do not apply our texture color here because we are reflecting our light. An additional specular color related to our surface material may be applied here or even a separate specular texture map but I've left that out in our example.
  // and add them all together
  fragcolor = vec4(clamp(ambientColor+diffuseColor+specColor, 0.0, 1.0), 1.0);
The last step is to add all these colors together. The clamp function we call here makes sure our color does not overflow.

And the end result:
A box really is a terrible shape to show off the lighting, as the normals for each face are all parallel we're basically 'flat shading' this cube. Once we start loading more complex shapes it should look a lot nicer. Also with the specular highlighting implemented the way it is the box has a mirror finish, not really something suitable for cardboard.

Download the source here

So where from here?


You could easily extent the shader to allow for more then one lightsource by simply repeating the ambient, diffuse and specular lighting calculations for those other light sources and just adding the results together. There are a number of additional parameters that you could add to improve on this One I already mentioned is the surface materials specular color. Another that is relatively simple to add are restraints on the angle to the lightsource and a limit to the distance to the lightsource to create a spotlight type light source.

There are many other shader techniques that are worth learning:
Bump mapping or normal mapping which is a technique to simulate groves and imperfections on the surface by adjusting the surface normal. This can be achieved by using a texture encoding normals that you look up instead of using our vertex normals.
Environment mapping is a cool way to do reflections or to solve our ambient issue discussed above. Ever looked at a CGI movies documentary and wondered why they hold up a mirror ball and take a picture? Use that picture as a texture and use your normal vectors X and Y (ignoring it's Z) as a lookup (adjusted so 0.0 is at the center of that image) and voila.
Shadow maps, so objects cast shadows onto other object, but that's a topic for later.

What's next


Now that we've got our basic rendering of 3D objects all sorted the next quick detour will be looking at moving the camera around.
We also need to write something that will let us load 3D objects from disk instead of hardcoding them in source code.
So that will be our next two subjects, hopefully after that we'll jump back into our platform game.

Saturday, 2 January 2016

Texture mapping (part 13)

Now that we have the basic logic for displaying a 3D object it is time to slowly make it look nicer.

We'll start by adding a texture map to our object. We'll be replacing the funky color logic here. It would be easy to combine the two but I see little point in doing so. The color of an object is usually uniform (lighting not withstanding) and the texture map replaces that function. But there are nice things you can do with mixing color in especially to create certain lighting effects. Such colors more often then not aren't set per vertex but instead set as uniforms.

That reminds me about a quick sidestep I'm not sure I made clear enough in previous examples. Our shaders have 4 prefixes for "global" (global to the shader) variables:
  • uniform, this means the value of the variable does not change and is generally set from outside of the shader, i.e. we set in in code
  • layout, which points to one of our attributes in our vertex buffer object(s)
  • out, which is an output variable which allows us to set variables and give them to the next shader in the pipeline. As mentioned before, these variables are often interpolated 
  • in, which is an input variable which should match the output of the previous shader in our pipeline. So the output of our vertex shader becomes the input of our fragment shader.
There aren't many things we need to change from our previous example.

Our texture map

First we need to load our texture map, I've added a box texture I grabbed somewhere off of the internet where we have different textures for each side of our box and placed this into our resources folder and then re-introduced a few things from our earlier tutorials.

First in our engine.h I've defined an enumeration that I use in conjunction with an array of textures. This is just convenience. Seeing we only have one texture it is also overkill but I like to be prepared:)
enum texture_types {
  TEXT_BOXTEXTURE,
  TEXT_COUNT
};
Then in engine.c I've defined the texture array into which we'll load our texture ids:
GLuint textures[TEXT_COUNT] = { 0 };
Now we need to load our texture, I've reintroduced my setTexture helper function and added the required logic at the end of our load_objects function:
void setTexture(GLuint pTexture, GLint pFilter, GLint pWrap) {
  glBindTexture(GL_TEXTURE_2D, pTexture);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, pFilter);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, pFilter);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, pWrap);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, pWrap);  
};

void load_objects() {
  int x, y, comp;
  unsigned char * data;

  ...

  // Now lets load our textures, note that this does not relate to our VAO state
  glGenTextures(TEXT_COUNT, textures);
  
  // and we load our box texture into textures[TEXT_BOXTEXTURE]
  data = stbi_load("boxtexture.jpg", &x, &y, &comp, 4);
  if (data == 0) {
    engineErrCallback(-1, "Couldn't load boxtexture.jpg");
  } else {
    setTexture(textures[TEXT_BOXTEXTURE], GL_LINEAR, GL_CLAMP_TO_EDGE);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, x, y, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
 
    stbi_image_free(data);
  };
};

void unload_objects() {
  glDeleteTextures(TEXT_COUNT, textures);
  glDeleteBuffers(2, VBOs);
  glDeleteVertexArrays(1, &VAO);
};
Note also that we delete our textures on our unload.

Changes to our model

Next we need to change our model data to contain coordinates within our texture for each vertex. Note that our coordinates range from (0.0, 0.0) - (1.0, 1.0) and are scaled up to the resolution of the texture image. (0.0, 0.0) is the top left of the image and (1.0, 1.0) is the bottom right. When loading the texture we told it to clamp to the edge but you can also set it to wrap the texture so it can be used as a pattern.

Now we do have a problem in the way we apply our texture because we only have 8 vertexs, we'll come back to that later. We'll change our vertex structure to hold a texture coordinate T instead of our color C and then adjust our vertex array:
typedef struct vertex {
  vec3    V;          // position of our vertice (XYZ)
  vec2    T;          // texture coordinates (XY)
} vertex;

vertex vertices[] = {
  -0.5,  0.5,  0.5, 1.0 / 3.0,       0.0,          // vertex 0
   0.5,  0.5,  0.5, 2.0 / 3.0,       0.0,          // vertex 1
   0.5, -0.5,  0.5, 2.0 / 3.0, 1.0 / 4.0,          // vertex 2
  -0.5, -0.5,  0.5, 1.0 / 3.0, 1.0 / 4.0,          // vertex 3
   0.5,  0.5, -0.5, 1.0 / 3.0, 1.0 / 2.0,          // vertex 4
  -0.5,  0.5, -0.5, 2.0 / 3.0, 1.0 / 2.0,          // vertex 5
  -0.5, -0.5, -0.5, 2.0 / 3.0, 3.0 / 4.0,          // vertex 6
   0.5, -0.5, -0.5, 1.0 / 3.0, 3.0 / 4.0,          // vertex 7
};
Our index array stays the way it is for now but we do need to change our vertex attribute pointers in our load_objects function:
  // now we need to configure our attributes, we use one for our position and one for our color attribute 
  glEnableVertexAttribArray(0);
  glEnableVertexAttribArray(1);
  glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid *) 0);
  glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid *) sizeof(vec3));
Note, the only change here is that our 2nd attribute is now 2 floats instead of 3 (the 2nd parameter in the last call).

Changes to our shaders

Now we need to update our shader code, again only small changes are needed. Our vertex shader now looks like this:
#version 330

layout (location=0) in vec3 positions;
layout (location=1) in vec2 texcoords;

uniform mat4 mvp;              // our model-view-projection matrix

out vec2 coords;

void main(void) {
  // load up our values
  vec4 V = vec4(positions, 1.0);
  coords = texcoords;
  
  // projection of V
  gl_Position = mvp * V;
}
The only change here is that our 2nd attribute has been renamed and is now a vec2 and our output variable is changed in unison to output a texture coordinate instead of a color. Just like with our color variable the values will be interpolated between the vertices before being sent to our fragment shader.

Our fragment shader has a few more changes:
#version 330

uniform sampler2D boxtexture;

in vec2 coords;
out vec4 fragcolor;

void main() {
  fragcolor = texture(boxtexture, coords);  
  if (fragcolor.a < 0.5) {
    discard;
  } 
}
Here we see the definition of a sampler2D uniform which is what we'll bind our texture to.
It is then used in our shader function together with our coordinate input variable "coords" to lookup our color.
The alpha check isn't needed for what we're doing now but a nice one to have, this allows us to do render shapes like leafs on a tree without having an elaborate model (see my tree generation blog post from a few months back).

Finally when loading our shader we need to grab the ID of our boxtexture sampler uniform in our load_shaders function:
  ...
  boxTextureId = glGetUniformLocation(program, "boxtexture");
  if (boxTextureId < 0) {
    engineErrCallback(boxTextureId, "Unknown uniform boxtexture");
  };
  ...

Rendering our cube

Finally to render our cube with a texture we need to bind our texture and inform our shader to use it. As I mentioned before, the texture is not part of our VAO state and thus not loaded automatically when we bind our VAO. This is actually a handy thing because we could reuse our cube model to render multiple cubes with different textures.

All we need to add is a little bit of code to our engine_render function when we select our shader:
  ...
  // select our shader
  glUseProgram(program);
  glUniformMatrix4fv(mvpId, 1, false, (const GLfloat *) mvp.m);
  glActiveTexture(GL_TEXTURE0);
  glBindTexture(GL_TEXTURE_2D, textures[TEXT_BOXTEXTURE]);
  glUniform1i(boxTextureId, 0);
Those last 3 lines:
  • make texture0 active
  • bind our texture to texture0
  • assign texture0 to our boxtexture uniform
And voila, we have a rotating textured box:

Correctly texturing our cube

However we're not there yet, only the front and back of the box is properly textured. The sides, top and bottom are all wrong. Here we have a problem that we often run into because we need different texture coordinates for the same position when rendering the other sides of the cube.

What we see in many 3D formats is that our index array actually contains multiple indexes for each vertex of the model pointing to separate position and texture coordinate arrays. This is not something OpenGL is designed to do.

Our only option is to duplicate vertices. This may result in some more data but its a small price to pay. I'm creating duplicates each time a vertex is used for a different side of our cube, you could remove some that are exactly the same, but in our next session we'll see there is a good reason for this.

So here are our updated arrays:
vertex vertices[] = {
  // front
  -0.5,  0.5,  0.5, 1.0 / 3.0,       0.0,          // vertex 0
   0.5,  0.5,  0.5, 2.0 / 3.0,       0.0,          // vertex 1
   0.5, -0.5,  0.5, 2.0 / 3.0, 1.0 / 4.0,          // vertex 2
  -0.5, -0.5,  0.5, 1.0 / 3.0, 1.0 / 4.0,          // vertex 3

  // back
   0.5,  0.5, -0.5, 1.0 / 3.0, 1.0 / 2.0,          // vertex 4
  -0.5,  0.5, -0.5, 2.0 / 3.0, 1.0 / 2.0,          // vertex 5
  -0.5, -0.5, -0.5, 2.0 / 3.0, 3.0 / 4.0,          // vertex 6
   0.5, -0.5, -0.5, 1.0 / 3.0, 3.0 / 4.0,          // vertex 7
   
  // left
  -0.5,  0.5, -0.5, 1.0 / 3.0, 1.0 / 4.0,          // vertex 8  (5)
  -0.5,  0.5,  0.5, 2.0 / 3.0, 1.0 / 4.0,          // vertex 9  (0)
  -0.5, -0.5,  0.5, 2.0 / 3.0, 2.0 / 4.0,          // vertex 10 (3)
  -0.5, -0.5, -0.5, 1.0 / 3.0, 2.0 / 4.0,          // vertex 11 (6)

  // right
   0.5,  0.5,  0.5, 1.0 / 3.0, 1.0 / 4.0,          // vertex 12 (1)
   0.5,  0.5, -0.5, 2.0 / 3.0, 1.0 / 4.0,          // vertex 13 (4)
   0.5, -0.5, -0.5, 2.0 / 3.0, 2.0 / 4.0,          // vertex 14 (7)
   0.5, -0.5,  0.5, 1.0 / 3.0, 2.0 / 4.0,          // vertex 15 (2)

  // top
  -0.5,  0.5, -0.5,       0.0,       0.0,          // vertex 16 (5)
   0.5,  0.5, -0.5, 1.0 / 3.0,       0.0,          // vertex 17 (4)
   0.5,  0.5,  0.5, 1.0 / 3.0, 1.0 / 4.0,          // vertex 18 (1)
  -0.5,  0.5,  0.5,       0.0, 1.0 / 4.0,          // vertex 19 (0)

  // bottom
  -0.5, -0.5,  0.5, 2.0 / 3.0,       0.0,          // vertex 20 (3)
   0.5, -0.5,  0.5, 3.0 / 3.0,       0.0,          // vertex 21 (2)
   0.5, -0.5, -0.5, 3.0 / 3.0, 1.0 / 4.0,          // vertex 22 (7)
  -0.5, -0.5, -0.5, 2.0 / 3.0, 1.0 / 4.0,          // vertex 23 (6)
};

// and now define our indices that make up our triangles
GLint indices[] = {
  // front
   0,  1,  2,
   0,  2,  3,

  // back
   4,  5,  6, 
   4,  6,  7,  

  // left
   8,  9, 10,
   8, 10, 11,
  
  // right
  12, 13, 14,
  12, 14, 15,
  
  // top
  16, 17, 18,
  16, 18, 19,
  
  // bottom
  20, 21, 22,
  20, 22, 23,
};
And our end result:

Download the source code here

What's next

In our next tutorial we'll start looking at basic lighting techniques.

Friday, 1 January 2016

The world, in 3 dimentions (part 12)

I hope that the previous sections gave a bit of an overview behind the thinking of creating a 2D game. The stable of 2D games are sprites which are then placed on screen usually within a fixed grid and while we'll leave most of that behind when jumping into 3D it is still usable for things like the UI.

But it isn't just the rendering side that is interesting, when we look at how we made Conrad move throughout the 2D environment having him move from one tile to the next, and thus giving properties to the tiles in the form of our interaction map, we very much simplified the logic needed to enable this.

When we look at 3D games we often think about games with a lot of freedom and indeed many of the out of the box 3D engines focus on enabling this using relatively expensive collision detection and similar techniques to let a character interact with the 3D world.

But there are legions of games out there that take a simpler approach very much keeping the same logic for interacting with a 2D world but rendering the environment in 3D. Take many RTS games for instance which still play on a 2D map (often with a few levels giving more of a 3D feel to the maps), but simply render everything in 3D giving the player full control of the camera.

When we look at platform games, Flashback tells an interesting story as its successor, Fade to Black, was an early attempt to bring this kind of game into the 3D world with mixed success. Fast forward to today and we see many successful games in this genre that allow full 3D interactivity, Tomb Raider and Uncharted spring to mind.
But there also was a remake of Flashback that essentially kept the game dynamics the same as the original but rendered the environment in 3D.
Another title that comes to mind that made a similar jump is Oddworld. I had the pleasure of playing an altered version of the 3D version of Oddworld using stereoscopic rendering and it simply blew me away, after that I firmly believe the industry gave up on 3D stereoscopic gaming way to quickly.

The reason for this rant? Simple, for an indie game developer on a budget I believe that looking at developing games that internally are essentially 2D games but rendered to 3D is a worthwhile endeavor. This will have my main focus for the continuation of this series, to turn our little 2D platformer into a 3D platformer keeping much of the game dynamics the same.

But before we get there, we've got some boring ground to cover. For now I've gutted our little example and put the bare minimum in there to render a box in 3D. On purpose I'm leaving a few things out as to not put to much information in one write-up. This one will be too long as it stands.

Oh, I've also not removed some of the files that are now no longer in use so please ignore those.

3D projection

In our 2D tutorial we introduced the concept of orthographic projections which basically told OpenGL how to transform a vertex to the correct coordinates on the screen. This type of projection ignores the Z component of our vectors and simply scales and translates the X and Y for display on screen. The Z is still used for layering but does not influence where things are drawn.

In 3D this changes. The further away an object is, the smaller it should be drawn on screen. I won't go to much in-depth to the internals of this but in essence we will be using a matrix that divides the X and Y by our Z to create the illusion of depth (in the matrix itself the division actually happens through our W component which is derived from our Z, this so we retain our Z value which comes in handy in our rendering process). This matrix creates what is called a viewing frustrum which defines a 3D volume that confines what we are seeing on screen:
ViewFrustum
(images courtesy of wikipedia)

We can create this matrix using the function mat4Frustrum in our math3d.h library however there is a second helper function that is a little bit nicer called mat4Projection. This function takes a FOV (Field Of View) value and the aspect ratio of your display to calculate the right values for our mat4Frustrum.

We also need to define our near and far plane. When we look at the human eye near is very close and far is infinite but for a computer these values are important because they determine how our Z is scaled so we can use our depth buffer. The bigger the gap between near and far, the less precise our Z buffer becomes, the more chance things won't look right on screen as something that is behind another object could be drawn on top of it.
The flipside of this argument is that the human eye is focused on a particular distance and only objects near that distance are sharp and everything else gets blurry while the computer renders everything in focus (mimicking focus we might look into much further down as there are some neat tricks to make this work though I am not a big fan of it).

In our engineRender function in engine.c we can see this function used where we previously had our mat4Ortho call:
  // set our model view projection matrix
  ratio = (float) pWidth / (float) pHeight;

  // init our projection matrix, we use a 3D projection matrix now
  mat4Identity(&projection);
  mat4Projection(&projection, 45.0, ratio, 1.0, 10000.0);
This sets a projection matrix with a 45 degree field of view.

As before in our 2D tutorial next to our projection matrix we need a view matrix which defines our "camera". At this point in time we'll leave this as an identity matrix, we'll discuss positioning a "camera" in a future part of this tutorial.

With this combination we now have our X axis going from left to right (positive X is to the right), our Y axis going from bottom to top (positive Y is pointing up) and our Z axis going from far to near (so we're looking into negative Z) with 0,0,0 being at the center of the screen.

So point 10, 20, -30 is position 10 units to the right, 20 units up and 30 units in front of our "camera".

Note that we do later on use our  orthographic projection so we can draw our FPS counter and you would normally do this for other UI elements as well.

3D Models

Now that we have our projection all set up we need something to render to screen. For our first step we're going to keep this pretty boring and render a cube. This will however allow us to slowly introduce individual concepts and deal with individual problems as we enhance our example.

We construct our cube using two primitives, vertices which are shown as the green dots in the diagram above, and polygons which are solid shapes formed by connecting 3 or more vertices. While many 3D modeling packages allow any number of vertices that form a polygon OpenGL prefers the use of triangles and in fact since OpenGL 3 you can only use triangles for rendering.

In the fixed pipeline of OpenGL 1 the definition of our vertices was fixed but with the advent of programmable shaders we've now got a lot of freedom in how much information we record for each vertex. We call these attributes of our vertex and they can be things like:
- the position of the vertex in 3D space
- the normal for that vertex (we'll get back to this)
- one or more texture coordinates for that vertex (we'll also come back to this)
- the color for that vertex
The only attribute we're likely to always find is the position of the vertex. You have a lot of control over how this data is organized but for our example we're going to define a structure for our vertex and define an array. Our initial example will record the position and a color for that vertex.
// we define a structure for our vertices, for now we define the location and color of each of our vertices on our cube
typedef struct vertex {
  vec3    V;          // position of our vertex (XYZ)
  vec3    C;          // color of our vertex (RGB)
} vertex;

vertex vertices[] = {
  -0.5,  0.5,  0.5, 1.0, 0.0, 0.0,          // vertex 0
   0.5,  0.5,  0.5, 0.0, 1.0, 0.0,          // vertex 1
   0.5, -0.5,  0.5, 0.0, 0.0, 1.0,          // vertex 2
  -0.5, -0.5,  0.5, 1.0, 0.0, 1.0,          // vertex 3
   0.5,  0.5, -0.5, 1.0, 1.0, 0.0,          // vertex 4
  -0.5,  0.5, -0.5, 0.0, 1.0, 1.0,          // vertex 5
  -0.5, -0.5, -0.5, 1.0, 1.0, 1.0,          // vertex 6
   0.5, -0.5, -0.5, 0.0, 0.0, 0.0,          // vertex 7
};
The position (V) is a simple X,Y,Z coordinate, for the color (C) we also use a 3D vector which now contains an R,G,B value. In this way we store 8 vertices for our cube.

For our triangles we use a simple index list in which we store 3 entries for each triangle we render:
// and now define our indices that make up our triangles
GLint indices[] = {
  0, 1, 2,
  0, 2, 3,

...
  
  4, 5, 6, 
  4, 6, 7,  
};
Now the order of our 3 vertices that make up each triangle is very important. For any 3D model that forms a solid any 'back facing' polygon will always be obscured by a 'forward facing' polygon. On our cube it is never possible to have more then 3 sides visible at any given time. Now luckily there is a very easy check for this by looking at the "normal" vector of the polygon and seeing if this points towards or away from the camera. The "normal" vector of a polygon is a vector that stand perpendicular to the plane of the polygon. This is extremely easy to calculate for a triangle by simply calculating the cross product of two edges of the triangle and OpenGL is able to do this automatically for you by turning backface culling on:
  // enable and configure our backface culling
  glEnable(GL_CULL_FACE); // enable culling
  glFrontFace(GL_CW);     // clockwise
  glCullFace(GL_BACK);    // backface culling
The first command enables the culling logic. The second command tells OpenGL that if vertices are position in clockwise order they are front facing (the default is counter clockwise). The third command tells OpenGL to cull the 'back facing' polygons.

Now in the old days we would use the arrays we just defined (or loaded from disk, which we'll dive into in a later tutorial) directly but this would cause a lot of copying data from normal memory to our graphics card on each frame. Eventually OpenGL got smart enough to do this copy once and reuse the data loaded into the graphics card and in OpenGL 3 this now is the only way to go.

We've already used part of this in our previous examples but we're now going to use the full deal.
First we again need a Vertex Array Object or VAO. Here VAOs are beginning to shine as they will encapsulate all the data related to our model. With our one cube this isn't to special but if we have different objects all we need to do is bind the VAO for the model we wish to render and all state associated with that model is made current.

Second we need two Vertex Buffer Objects or VBOs that contain our actual data. We'll copy our vertices in the first VBO and our indices into our second VBO.

Last but not least we need to tell OpenGL about the two attributes our vertex VBO now contains. The whole code for loading our cube looks as follows:
void load_objects() {
  // we start with creating our vertex array object
  glGenVertexArrays(1, &VAO);
  glGenBuffers(2, VBOs);
  
  // select our VAO
  glBindVertexArray(VAO);
  
  // now load our vertices into our first VBO
  glBindBuffer(GL_ARRAY_BUFFER, VBOs[0]);
  glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), &vertices, GL_STATIC_DRAW);
  
  // now we need to configure our attributes, we use one for our position and one for our color attribute 
  glEnableVertexAttribArray(0);
  glEnableVertexAttribArray(1);
  glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid *) 0);
  glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid *) sizeof(vec3));
  
  // now we load our indices into our second VBO
  glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBOs[1]);
  glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(indices), &indices, GL_STATIC_DRAW);
  
  // at this point in time our two buffers are bound to our vertex array so any time we bind our vertex array
  // our two buffers are bound aswell

  // and clear our selected vertex array object
  glBindVertexArray(0);
};
Hopefully the comments are enough to explain the code. The tricky command in this is glVertexAttribPointer which tells OpenGL how the data loaded into our VBO is organized.

Note: as long as we wish to render this model we need to keep both the VAO and the two VBOs alive but if we had loaded the model from disk into temporary arrays we could free up this memory as we've copied it into memory managed by OpenGL. Freeing up our VAO and VBOs and thus freeing up the memory allocated by OpenGL is done in this function:
void unload_objects() {
  glDeleteBuffers(2, VBOs);
  glDeleteVertexArrays(1, &VAO);
};

Shaders

The final ingredient here is our shader but in this case it is deceptively simple. I didn't want to touch on the complex stuff just yet which is why I opted for adding a color attribute. The effect is fairly psychedelic as our shader simply interpolates the color value between each vertex as it draws the polygons.

Here however the difference between vertex shader and fragment shader becomes much more apparent. Our vertex shader is executed for each of our 8 vertices and at this point in time it has two goals:
  1. calculate the screen coordinates of each vertex by applying our model-view-projection matrix
  2. determine the color of each vertex
This is the code for our vertex shader:
#version 330

layout (location=0) in vec3 positions;
layout (location=1) in vec3 colors;

uniform mat4 mvp;              // our model-view-projection matrix

out vec3 color;

void main(void) {
  // load up our values
  vec4 V = vec4(positions, 1.0);
  color = colors;
  
  // projection of V
  gl_Position = mvp * V;
}
The first new thing we see are our attribute definitions, these map directly to our attribute definitions we declared when we loaded the data for our cube. Within our shader these variables will point directly to the entry in our array for which our vertex shader is being called.

We then define our color as a vec3 output variable that we can use in our fragment shader.
In our function we calculate V as a 4D vector for our position and simply copy our color.
We then apply our model-view-projection matrix (mvp) to V and store the result in gl_Position, one of the few build in OpenGL variables still supported in OpenGL 3.0.

Next OpenGL will use our indices to render the polygons taking the output of our vertex shader for each of the 3 vertices of our triangle and then interpolating our output variables. As we only output our color the color is nicely mixed over the surface of our cube. Our fragment shader is then called for each pixel we render out to screen.

As the interpolation is done before our fragment shader is called there is very little more to do and our fragment shader simply copies its input to our output:
#version 330

in vec3 color;
out vec4 fragcolor;

void main() {
  fragcolor = vec4(color, 1.0);  
}
This is where OpenGL gets a little silly. In the original shader implementation our fragment shader the output variable was a build in variable just like gl_Position but unlike gl_Position it was removed and instead our fragment shader must have a single output variable.
There is good reason for this however which we may come back to in a later tutorial. When we use techniques such as deferred rendering a fragment shader can have multiple outputs and this change starts making sense.

I've covered loading the shader into OpenGL in previous parts of this tutorial so I won't go over it again, the code can be found in the function load_shaders().

Rendering our cube

This is the bit where I really like VAOs. To render our cube we need to do 3 things:
  1. calculate our model-view-projection matrix
  2. select our shader
  3. render our VAO
For rendering one cube this may not seem very special but imagine rendering dozens of things you can see how little work there is to do in rendering them if you've loaded them into VBOs and setup a VAO for each of the objects to render.

The code for this is:
  // set our model view projection matrix
  mat4Copy(&mvp, &projection);
  mat4Multiply(&mvp, &view);
  mat4Translate(&mvp, vec3Set(&tmpvector, 0.0, 0.0, -30.0));   // move it back so we can see it
  mat4Rotate(&mvp, rotate, vec3Set(&tmpvector, 10.0, 5.0, 15.0)); // rotate our cube
  mat4Scale(&mvp, vec3Set(&tmpvector, 10.0, 10.0, 10.0));   // make our cube 10x10x10 big

  // select our shader
  glUseProgram(program);
  glUniformMatrix4fv(mvpId, 1, false, (const GLfloat *) mvp.m);  

  // now render our cube
  glBindVertexArray(VAO);
  glDrawElements(GL_TRIANGLES, 12 * 3, GL_UNSIGNED_INT, 0); 
For calculating our mvp I'm doing everything in one go instead of first creating a model matrix. The model matrix consists of 3 steps (they are in reverse order in the code):
  • Scale the cube to size
  • Rotate the cube based on an angle that we increase as time goes by
  • Move (translate) the cube into position
I've put the scale step in on purpose, I could have defined the cube at the correct size. We'll get back to that in due time.

Selecting the shader is a simple call to glUseProgram and then loading our mvp into our shader.

Rendering the VAO is now 2 simple steps: binding the VAO (which also binds the VBOs and calling glDrawElements to tell OpenGL to draw our triangles.



Now in this last bit we do see our first little hickup for those of you who are aware of how 3D modeling software often handles models especially when different materials are used with a single object. Often enough we'll have one array of vertices but then multiple arrays of indices which each need to be drawn using a different material (often different shaders, or the same shader with different settings). This mostly is a space saving issue as we can reuse vertices but as we'll later on find out this issue quickly becomes mute in a 3D engine as vertices will need to be duplicated for other reasons.

Now we can use the same VBO over multiple VAOs and thus reuse the same vertex array but personally I think it works much better in OpenGL if each VAO has its own vertex array and index array that renders (part of) an object with a single material.

Download the source code here

What's next?

In the next part we're going to make our cube a little less boring by applying a texture to it.
The part after that we'll discuss basic lighting.
After that we'll probably be ready to start applying the first part of the changes to our platform game.

Saturday, 26 December 2015

Interaction with the 2D environment (part 11)

Sorry for the long pause between this and the last article. I simply haven't had any time in the past months to spend on this project. But the holiday season has started and I finally found a few nights to take this to the next step.

This will be the last step in this 2D adventure but it leaves us with a fully functional 2D engine that allows our character to walk around a 2D map. While there are plenty of things to do from here it is a good starting point to go from.

A lot has changed in the source to get to this next level so this is a bit of a read. I'll try to handle each area one step at a time.

Housekeeping

Obviously some housekeeping had to be done to make some of the code written so far more generic and reusable. Some of these we'll look into in more depth but in broad strokes:
  • tilemap.h (and the associated shaders) has been updated to allow any sized map to be used without having to make changes to the code. We thus store the size of the map and size of our tiles image file in our structure and send this information to the shaders
  • spritesheep.h (and the associated shaders) has similarly been updated
  • all of the hardcoded data in engine.h and engine.c has been moved into a new header file called gamedata.h. Obviously in a real game this information should be stored in files that can be loaded as they are needed to create a game with multiple levels instead of stored inside source files but for our example code this is easier. I moved everything into a separate file to set this data apart.
I'm still using globals for various things though I have moved more into structures where this made sense for re-usability. As I've mentioned many times before this is mainly for readability of the code and to keep things simple but a few exceptions set aside this code could do with removing a few of the globals in the long run.

A new map

We're no longer using the sample desert map from our tiled map editor but a map I've flung together from some screenshots from the first level of the game flashback. This leaves a lot to be desired and really needs to be cleaned up but for our purpose it does well.
I've put the original maps you can edit in the tilemap editor within the folder structure in the github repository (Support/Flashback map) but they are not used directly. Instead the CSV exporter was used to export the maps and embed them in the gamedata.h header file.
Note that the map has two layers, one layer is the map we're actually rendering to screen in the game, the other allows us to mark where in our map our character can walk and where he can climb up or down. This is done by using a simple colored tileset so we can use the indexes it creates:
0 = empty space
1 = conrad can walk here
2 = conrad can climb up when facing left
3 = conrad can climb up when facing right
This is still very simplistic as I've limited the actions Conrad can do but if all his animations were implemented we'd probably need a few more to indicate where Conrad can only roll or where he can jump. More on this later.

Our map is now also scaled 1:1 so each tile is rendered as a 32.0 x 32.0 tile in our coordinate system. We also render Conrad at this scale which is very important because our animations line up with our tiles. Conrad will thus always be rendered at the right side of a tile.

The same applies to our currentPos global variable that holds Conrads position which is now a position within our tilemap (measured from the center of our tilemap). Our default position of -18, 9 (which is now set in our engineLoad function) puts Conrad at the left bottom of our map.

Enhancements to the action map

This is probably where the largest changes have taken place. The action map that was introduced in the last write-up has been changed drastically.

First it no longer is a single array through which we scan. Instead every animation has its own followup action map. This is simply a speed up because instead of having to scan through the entire map, we only scan through those entries that actually can be a followup action to our current animation.

Our animation structure now has a new member called followUpActions which is a pointer to an action_map array. 
// now define our animations
// note that movements are multiples of 32
animation conradAnim[] = {
    0,    0,  false,   0.0,  0.0, conradLookLeftActions,                        //  0 - look left
    0,    6,  false,   0.0,  0.0, conradLeftToRightActions,                     //  1 - turn from looking left to looking right
...
Note also that our moveX and moveY members now refer to our movement within our map and are multiplied by 32.0 to get our movement on screen. I've cheated a little with some of the animations which is something that should be cleaned up at some point.

Our first entry in our animation array refers to Conrad simply standing still and looking to the left. The conradLookLeftActions array being pointed to as the followUpActions member is an array that contains all followup actions possible when Conrad is looking to the left and looks as follows:
// Now we define all the followup actions for each of our actions
action_map conradLookLeftActions[] = {
  CR_ANIM_TURN_LEFT_TO_RIGHT, keysRight, conradCanStand,                        // turn left to right if our right key is pressed
  CR_ANIM_WALK_LEFT_1, keysLeft, conradCanWalkLeft,                             // start walking left
  CR_ANIM_GET_DOWN_LEFT, keysDown, conradCanCrouch,                             // start getting down facing left
  CR_ANIM_CLIMB_DOWN_LEFT, keysDown, conradCanGoDownLeft,                       // start climbing down facing left
  CR_ANIM_JUMP_UP_LEFT, keysUp, conradCanStand,                                 // jump up facing left
  CR_ANIM_LOOK_LEFT, NULL, conradCanStand,                                      // keep looking left if no key is pressed
  CR_ANIM_FALLING_LEFT, NULL, NULL,                                             // must be falling..
  CR_END, NULL, NULL,                                                           // end (we shouldn't get here)
};
Note that our array ends with an entry "CR_END, NULL, NULL". In theory we will never evaluate this but it is a safety net, kind of like a NULL terminator.

The first member is our followup action, the second member are the keypresses associated with our followup action (we had this already in a slightly different form) and our third member points to structures that check our second layer in our map to see if the movement is allowed (i.e. Conrad can't walk to the left if there is a tree in the way).
When our second or third member is NULL it is ignored and the action is possible.
We'll look at these two members more closely further on.

In our previous tutorial the loop that checked our action map was embedded in our engineUpdate function. I've moved the logic into new functions to make it easier to maintain. In our engineUpdate routine we know what the current animation is and thus have a pointer to the action map for that specific animation. We simply call a new function called getNextAnim with a pointer to that map which will return the next animation we should play:
...
  nextAnim = getNextAnim(conradAnim[currentAnim].followUpActions);
...
Couldn't be simpler. The magic happens in getNextAnim which simply looks at each entry in our action map until it finds the first one that can be used:
// obtain our next animation type
GLint getNextAnim(action_map *followUpActions) {
  GLint nextAnim = -1;

  if (followUpActions != NULL) {
    // check our action map
    while ((nextAnim == -1) & (followUpActions->startAnimation != CR_END)) {
      // assume this will be our next animation until proven differently
      nextAnim = followUpActions->startAnimation;
              
      // check if we're missing any keys
      if (!areKeysPressed(followUpActions->keys)) {
        // keep looking...
        nextAnim = -1;
        followUpActions++; // check next
      } else if (!canMoveThere(followUpActions->moveMap)) {
        // keep looking...
        nextAnim = -1;
        followUpActions++; // check next
      };
    };    
  };  
  
  return nextAnim;
};

We check our keypresses first and whether we can move to that location.
The keypress logic remains nearly the same as the previous implementation but instead of hardcoding up to 5 keypresses we now use a zero terminated array. Our first entry in our action map points to the array called keysRight which is defined as:
int keysRight[]     = { GLFW_KEY_RIGHT, 0 };
Our canMoveThere function works very similarly but has a slightly more complex structure it points at, here for example is our conradCanWalkLeft array and our idCanWalk array it references:
unsigned char idCanWalk[] = { 1, 2, 3, 0};
...
move_map conradCanWalkLeft[] = {
  -1,  0, idCanWalk,
   0,  0, NULL,
};
The first entry in conradCanWalkLeft checks if conrad can move one tile to the left. It does this by checking if the value of the tile in our second map appears in our idCanWalk array which is once again a zero terminated array.
Note that as a result we can't check for a zero value for our tile which is fine as we've defined zero to be empty space. 

Moving out of the screen

The last thing I hadn't addressed in the previous version was Conrad walking off screen. As Conrad is now constraint within our map I simply update the view when Conrad is about to leave the screen. There are nicer ways to handle this but for our purpose this will do fine. The code was added right after our followup animation check.



Where too from here?

We now have a working but very basic engine for a platform game. Obviously there is a need to complete the animations that are missing and tweak the existing ones. One good example is to break the roll up into 4 steps instead of rolling 4 tiles at once. The original game allowed you to roll even if there were only 2 tiles between Conrad and a wall but he would simply bounce off the wall. This could be easily triggered by using another tile type in the second map to indicate the wall and adding in that once you attempt to roll there, the bounce off the wall animation starts playing. Similarly we can't jump off a ledge or do long jumps yet which are really cool to add.

Another issue to tackle is that Conrad is currently drawn over the background map which isn't always correct:
Here Conrad is standing in front of the thingy on the ground while he should be standing behind it. This can be solved by either having a stencil for our image and tweaking the Z-value when drawing the tiled map or it could be done by creating multiple layers in our map having a background layer drawn behind Conrad and a foreground layer drawn in front of Conrad.

Finally completely missing here are the bad guys and other obstructions. For now I'll leave that up to your own imagination :)

Download the source code here

What's next?

As I mentioned up top, this is the last write-up I've planned on the subject of 2D within the confines of this tutorial. I probably will write additional blog posts to clear up certain things I've done here and I may tweak the examples as time goes by.

Next I wish to jump back into the world of 3D which I've left alone for far too long but we're not going to leave our platform example. We'll have to sidetrack a little to explain some basics but then we're coming right back to our platform game and we'll look into transforming it into a 3D environment.

Sunday, 20 September 2015

Rendering sprites #3 (part 10)

So it is time for the next step and get some basic user interaction going. Before we get there however, lets look at some of the mechanics involved.

In our previous post we looked at the individual animations that make up the movements of our main character. One thing you may have noticed is that the sprite sheet I'm using has actually been scaled up to 200%. I'm not sure if that was done in the original game or by the person who extracted the graphics. I've left it as is as this is just to demonstrate the techniques and with todays high resolutions screens we would probably want to use higher quality images anyway.
There are however two key observations we can make from the sprites and the animations:
  1. Each animation should play in full and is then followed up by the next animation to create fluid motion. Some animations are split into multiple ones. Walking for instance has 3 animations, taking the first step, stopping at the end of the first step and continuing with the 2nd step. Play the first and 3rd in a repeated sequence and our character keeps on walking. If our character needs to stop moving after the first animation we need to play the second one to round the animation off, but we can stop on a dime if we're stopping after the 2nd step.
  2. If the character moves, the character moves a fixed amount. The movement is always a multiple of 32 (or likely 16 in the original game).So after taking one step (1st + 2nd animation) we'll have moved 32 pixels. If we take two steps (1st + 3rd animation) we'll have moved 64 pixels. I've not checked the animation for running yet but my guess is the movement will be either double or triple that.
    Similarly climbing or falling we're moving up (or down) 96 pixels (3 * 32).
There is a very important reason for these fixed movements, they align with the background used in the original flashback game and this makes it far easier to check what movements can be done. For instance for climbing we can simply check if we're on a background tile where climbing makes sense. It also improves game play as the player doesn't need to go through the frustration of being a pixel to far to the left or right. In this part of the write-up we won't worry about this yet but it will become clear in the next part.

The one animation that stands out is rolling but I think that is caused by either missing a part of the animation or that I've gotten something wrong (I am trying to reverse engineer the animations used here). I've left it in the source code for now but I will likely remove it once we go to the next part to ensure we only have animations that work with our background.

Going from one animation to the next 

So looking at our animations it is important to realize that only certain animations follow others and the animation that should follow is controlled by two other parameters:
  • What is allowed, i.e. we can't walk left if there is a wall
  • What input is given by the user
We'll leave the interaction with the environment for the next part and focus on user input first. We'll also limit ourselve to based keyboard input.

As far as making our engine aware of the keyboard input I've taken out my temporary keyboard interface from the previous part and exposed the glfwGetKey method through a call back. Remember I didn't want to put any GLFW specific code in the game engine itself. I'm not 100% sure I'm going to keep this working the way it does right now but it will suffice for the time being. This function simply checks if a specific key is currently being pressed. Unlike our previous example we're not reacting on the keypress itself but on the current state of the keyboard. GLFW should simply be keeping a keymap of the keys currently pressed so this check should be pretty quick.

Instead of programming all the individual keypresses and reactions to them we're going to use a lookup table with a state engine. A state engine is nothing more then a fancy word for defining in what state our character currently is in i.e. if he's standing still, walking, jumping, facing left or right, etc.

Our state engine has three variables that track it:
  • currentAnim, the current animation playing, which is equal to what our character is doing. This relates directly to our animation enumeration we already introduced in the last post.
  • currentSprite, the current sprite within the animation that is currently being displayed.
  • currentPos, the current position within our game world our character occupies.
After our animation finishes playing we'll adjust the position according to the movement recorded in our animation table and check our action lookup table for the next action/animation we want to invoke.

This lookup table is defined using the following struct:
// structure to control our characters with
typedef struct action_map {
  GLint     animationEnding;        // when this animation ends
  GLint     startAnimation;         // start this animation
  int       keys[5];                // if these keys are pressed (ignoring zeroes)
} action_map;

As we can see our action lookup table is defined by the animation that has just finished playing, the animation that should follow and up to 5 keys that the user should be pressing to make this happen.
We'll be checking our table top down and the first entry that matches our current state wins.

Here is a part of the table:
// and define our action map, the first action that matches defines our next animation
action_map conradActions[] = {
  // looking left
  CR_ANIM_LOOK_LEFT, CR_ANIM_TURN_LEFT_TO_RIGHT, { GLFW_KEY_RIGHT , 0, 0, 0, 0},                  // turn left to right if our right key is pressed
  CR_ANIM_LOOK_LEFT, CR_ANIM_WALK_LEFT_1, { GLFW_KEY_LEFT, 0, 0, 0, 0},                           // start walking right
  CR_ANIM_LOOK_LEFT, CR_ANIM_LOOK_LEFT, { 0, 0, 0, 0, 0},                                         // keep looking left if no key is pressed
  CR_ANIM_TURN_LEFT_TO_RIGHT, CR_ANIM_LOOK_RIGHT, { 0, 0, 0, 0, 0},                               // turn to looking right once finished

  // walking left
  CR_ANIM_WALK_LEFT_1, CR_ANIM_WALK_LEFT_2, { GLFW_KEY_LEFT, 0, 0, 0, 0},                         // keep walking left
  CR_ANIM_WALK_LEFT_1, CR_ANIM_STOP_WALK_LEFT, { 0, 0, 0, 0, 0},                                  // stop walking left if no key is pressed
  CR_ANIM_WALK_LEFT_2, CR_ANIM_WALK_LEFT_1, { GLFW_KEY_LEFT, 0, 0, 0, 0},                         // keep walking left
  CR_ANIM_WALK_LEFT_2, CR_ANIM_LOOK_LEFT, { 0, 0, 0, 0, 0},                                       // finished walking left if no key is pressed
  CR_ANIM_STOP_WALK_LEFT, CR_ANIM_LOOK_LEFT, { 0, 0, 0, 0, 0},                                    // finished walking left if no key is pressed

Now this is only a small section of our table but lets look at these entries more closely to try and explain how this works.

The first entry applies when our current animation is CR_ANIM_LOOK_LEFT which is a single frame animation of our character looking left. The animation that follows is CR_ANIM_TURN_LEFT_TO_RIGHT which is an animation that makes the character turn around. The key listed is GLFW_KEY_RIGHT which means this animation is triggered if the user is pressing the right arrow key. Note that we don't care about any other keys but also that we don't check any other keys. I.e. if they user is pressing the right and up key this is still the first entry that matches and this is what will happen. The order in which the animations are specified in this table are thus very important.

The second entry triggers our walking animation. If the character is currently standing still looking left (CR_ANIM_LOOK_LEFT) and the user has the left arrow key pressed, we start our walking animation.

The third entry is the action if the character is looking left and the user has no (matching) keys pressed. We'll just keep playing the same standing still animation over and over again until something else happens.

The next set of entries define the walk itself. These go through the combinations of cycling through the 1st (CR_ANIM_WALK_LEFT_1) and 3rd (CR_ANIM_WALK_LEFT_2) animations we talked about in our introduction above. If the user lets go of the left key depending on which animation is currently playing we either directly got back to standing still (CR_ANIM_LOOK_LEFT) or we first play our stop walking animation (CR_ANUM_STOP_WALK_LEFT).

A lookup table like this lets us easily add in more animations and how we should react to the users input.

Note that I have no way of knowing if the original game uses a similar approach or something very different. Also I've kept things simple, there are a few easy optimizations to speed things up, especially animations defined later in the table mean we have to do a lot of tests before we end up at the right one but that is for another day.

Our new update routine


Our final step is actually reacting to our keyboard commands and making it all work. For this we've enhanced our engineUpdate routine. The start of our routine is still pretty similar. I've changed the animation speed to roughly 12 frames a second but other then that we simply cycle through the sprites. Once our animation recycles we set animReset and do our keyboard check.

This part has changed:
    // if our animation was reset we have a chance to change the animation
    if (animReset) {
      int nextAnim = -1;
      
      // if our current animation was played forward, we add our movement
      if (conradAnim[currentAnim].firstSprite<=conradAnim[currentAnim].lastSprite) {
        currentPos.x += conradAnim[currentAnim].moveX * SPRITE_SCALE;
        currentPos.y += conradAnim[currentAnim].moveY * SPRITE_SCALE;        
      };

Once our animation has finished we need to check if we need to add the movement to our current position. We only do this if our animation was played in the normal direction.

      // check our action map
      for (int a = 0; a < MAX_ACTIONS && nextAnim == -1; a++) {
        if (conradActions[a].animationEnding == currentAnim) {
          // assume this will be our next animation until proven differently
          nextAnim = conradActions[a].startAnimation;
                    
          // check if we're missing any keys
          for (int k = 0; k < 5 && nextAnim != -1; k++) {
            int key = conradActions[a].keys[k];
            if (key == 0) {
              // no need to check further
              k = 4;
            } else if (!engineKeyPressedCallback(key)) {
              nextAnim = -1;
            };
          };
        };
      };

      // if -1  We're missing something
      currentAnim = nextAnim == -1 ? 0 : nextAnim;
      currentSprite = conradAnim[currentAnim].firstSprite;

This is our main logic scanning our table until we found the correct animation and keypresses. We just loop through our entire table until we find the right one. If we can't find anything our table is incomplete, oops, so we just default back to our first animation. 

      // if our new animation is going to be played in reverse, we subtract our movement
      if (conradAnim[currentAnim].firstSprite>conradAnim[currentAnim].lastSprite) {
        currentPos.x -= conradAnim[currentAnim].moveX * SPRITE_SCALE;
        currentPos.y -= conradAnim[currentAnim].moveY * SPRITE_SCALE;        
      };

Finally, if we're playing an animation in reverse we need to subtract our movement first. I've only used this for backwards rolls.

If you compile and run the sample application at this point in time you should be able to make Conrad walk left and right, roll, and making him climb. He still lacks a way to get back down :)


Download the source code here

So where from here?

Well the next step is putting a proper background in place and make Conrad interact with it so he can only climb where it makes sense and drop back down.

It may take awhile before I get this done. First off I need to spend some time on getting the graphics sorted out but I've also got a few other things on my plate so I may not be working on this any time soon.






Saturday, 5 September 2015

Rendering sprites #2 (part 9)

So we're taking baby steps going forward. First because I want to detail how I'm slowly progressing to something that allows the user to fully control the character, but also because time is too short to do it all at once:) I'm trying to find and use every spare few minutes I can find between family, work and playing games. Owh and I'm still hard at work building a HALO suit, that's turning out a much larger project then I initially thought.

First off, I made a few more changes on the work done so far just to complete it. I ended up enhancing spritesheet.h so that sprites are centered but you add an offset to position it properly. This also allows the animation to give flexibility on the movement.

Our sprite struct thus is now defined as:
// structure to keep data about our spritesheet
typedef struct sprite {
  GLfloat left;                   // left position of our sprite in our texture
  GLfloat top;                    // top position of our sprite in our texture
  GLfloat width;                  // width of our sprite
  GLfloat height;                 // height of our sprite
  GLfloat offsetx;                // horizontal offset
  GLfloat offsety;                // vertical offset
} sprite;

Our array of sprites now defines about the first 4 rows in our spritesheet. That is as far as I'll go for now. Eventually I'll add the climb in because I need that for this little demo but I'll leave most of the other animations alone.

Defining animations

The next step is to bring some structure to our animations. We're going to break up all the animations into small pieces that can be sequenced depending on the users input. For now we'll just switch between them so we can see if we've got them right but eventually after each animation ends we'll check for user input and determine what the next animation is.

I've define a struct in engine.h that allows me to define one animations sequence:
typedef struct animation {
  GLint     firstSprite;            // first sprite of animation
  GLint     lastSprite;             // last sprite of animation
  bool      flip;                   // flip sprite horizontally
  GLfloat   moveX;                  // add this to our X after anim is finished
  GLfloat   moveY;                  // add this to our Y after anim is finished  
} animation;

To make it easier to use my animation I've also added an enum to engine.h that lists all the different animations:
enum conrad_animations {
  CR_ANIM_LOOK_LEFT,
  CR_ANIM_TURN_LEFT_TO_RIGHT,
  CR_ANIM_LOOK_RIGHT,
  CR_ANIM_TURN_RIGHT_TO_LEFT,

...

  CR_ANIM_JUMP_UP_LEFT,
  CR_ANIM_COME_DOWN_LEFT,

  CR_ANIM_COUNT
};

The animations themselves are then defined in an array in engine.c:
// now define our animations
animation conradAnim[] = {
    0,    0,  false,   0.0,  0.0,    //  0 - look left
    0,    6,  false,   0.0,  0.0,    //  1 - turn from looking left to looking right
    6,    6,  false,   0.0,  0.0,    //  2 - look right
    6,    0,  false,   0.0,  0.0,    //  3 - turn from looking right to looking left

...

   50,   66,  false,   0.0,  0.0,    // 16 - jump up (facing right)
   67,   78,  false,   0.0,  0.0,    // 17 - coming back down (facing right)   
};

The code for displaying the animation hasn't changed much. We're just cycling through a single animation instead of all defined sprites.

I've turned off the back ground for now and I'm displaying all sprites in the current selected animation on the left and right slightly overlapped to help with changing the offset of each sprite. Until we actually start with controlling the character we won't know if we've gotten these right.



The final change at this point is changing the animation. I've removed the camera control (this will be changes anyway) and for the time being am sending key presses to the engine (again I'm trying to leave any GLFW specific code out of the engine). In the engine I'm simply keeping a stack of unhandled keypresses which I check at the end of each animation. This won't be the final solution for when we start to control our character but it does the job for cycling through the animations without loosing keypresses when the animation hasn't finished yet.

Download the source code here

So where from here?

The next step is simple, to react on keyboard input and actually control the character.

At first we won't worry about any character interaction with his surroundings and implement basic walk left, walk right, roll left, roll right and straight jump logic. I may through in a few more animations just for the heck of it.

After that we'll add a more appropriate background and start looking at properly interacting with it.

Thursday, 3 September 2015

Making timer and worker events in Omnis safe

Timer events and the new worker objects in Omnis Studio are two very powerful functions in Omnis. Both use the same internal logic to kick of logic at a given time, for timers this will happen at either fixed intervals or when a specific amount of time has passed, for worker objects the run logic once results have been received from the database in a background thread and the data is ready for the application to deal with.

This however is not without consequence. These events can interrupt whatever code is currently running in the foreground thread and as a result introduce very hard to find bugs if the interruption changes something the code being interrupted wasn't expecting.

TigerLogic have done a good job to ensure these interruptions don't happen willy nilly as code can only be interrupted at specific times. It is however unclear what interruptions are possible. After some experimentation I've found the following scenarios will be interrupted by timer and worker code:
- when the main thread is paused on an enter data command
- at the end of a loop
- when constructing a new window
I'm sure I've forgotten some others.

Generally speaking this works great unless you rely on globals.

The main exception to that however are worker object. They have one big drawback, a worker can not interrupt another worker. Omnis tends to crash when that happens. If you have more then one worker active at a given time and you're in the middle of processing the results of the first worker object (which you'll likely do within a loop), the 2nd worker can interrupt that loop as it has finished retrieving its result set. 9 out of 10 times this will crash your application.

It is therefor important to check if you are interrupting code that could potentially lead to issues and delay the action till later. At the start of my $completed function for my worker I cache the result set and check whether I can process it. If so I process it right away, if not I start a timer to check if I can process it a short time later.

In both my worker $completed code and timer $timerproc code I call the following functions that I've placed in a code class:
cCode/ERR
  ; Bypass error handler, we ignore any errors while this is active
  SEA continue execution

cCode/CanRunTimer(pvExpectedLineCount = 2, pvSendToTraceLog = kFalse)
  Begin reversible block
    ;  Temporarily replace our error handler with a dummy one, sys(192) tends to send bogus issues to our error handler..
    Load error handler cCode/ERR
  End reversible block

  Calculate lvList as sys(192)
  If lvList.$linecount<=pvExpectedLineCount
    ;  We can run it, just $event/$completed/$timerproc and CanRunTimer are on the stack
    Quit method kTrue
  Else If pos('Enter data',lvList.[pvExpectedLineCount+1].linetext)=1
    ;  We can run it, we've interupted an enter data!
    Quit method kTrue
  Else If pos('ALLOW TIMER',lvList.[pvExpectedLineCount+1].linetext)<>0
    ;  Our comment says we are allowed to interupt here!!
    Quit method kTrue
  End If

  ;  !BAS! Timer can interupt END FOR, END WHILE, and some other situation. We DON'T want this as we may be in the middle of processing stuff!
  If pvSendtoTraceLog
    Calculate lvList.$line as pvExpectedLineCount+1
    Send to trace log (Diagnostic message) {Can't run queued timer, stackcount = [lvList.$linecount], interupts [lvList.classitem().$name].[lvList.method]/[lvList.line] = [lvList.linetext]}
  End If
  Quit method kFalse
This method will return true if I can safely process my timer code, and false if I should delay the action.

There are only 3 situations I deem safe:
- if all that is on the stack is my $event, $completed or $timerproc method and my CanRunTimer
- if we are interrupting an "enter data" command
- if we are interrupting a command that contains the "ALLOW TIMER" text (which I can put in a comment)

I've also made it optional to sent any other situation as a diagnostic message to the trace log. If a timer seemingly never runs I can check if I'm being too strict in letting timers run their course.

A timer method would then look something like this:
Timer.$event
  On evTimer
    Do code method cCode/CanRunTimer Returns lvCanRun
    If lvCanRun
      ;  Do your stuff
      ...
    Else
      ;  Restart our timer (omit this if you have $autoreset turned on)
      Calculate $cinst.$objs.Timer.$timervalue as 200 
      Do $cinst.$objs.Timer.$starttimer()
    End If

One last note on the subject. My workers tent to start a timer that delays processing of the results for 1ms regardless of any other running code. First of it means that I only need to implement this check in one place but more importantly it allows $completed to finish and clean up. Workers will also crash Omnis if you start another worker or do any other form of communication with the database even on completely unrelated session objects. By allowing $completed to finish and handling the result set when the timer object fires you gain a lot of stability in your application.