The XL Engine is and will remain free, donations are purely optional but greatly appreciated.

Recent Comments

Archive for July 2012

I missed the last XL Engine blog post and so delayed the Alien Awakening post from this week as well. Once the XL Engine progress is back on track, the Alien Awakening posts will resume. I wanted to show a movie for this post to show where things are now… but I’m not quite there yet due to some delays. However there has been a fair amount of progress that I’ll talk about in this post.


Logics are a system where behaviors can be defined and attached to objects, similar to the Logics that were used in DarkXL. Rather then having an object have fixed behavior – objects are basically containers to which components and behaviors can be attached. Logics are contained behavior modules that can be implemented in either code – in the game DLL – or in script. These logics are then referenced by name and can be added or removed from objects at runtime. In order for Logics to affect objects they have various callbacks that may be called, such as Update (if the object is currently active) and can also receive messages, though during setup the Logics must opt in to the messages that they wish to receive.

The basic Logic that a game needs is the player itself, which is currently implemented in the game DLL. The player Logic is no different to the Engine then any other Logic but it is where the player behavior is controlled – including input that may affect the player. Other, non-player affecting input is handled by the UI script that runs the “GameView.” For DaggerXL I’ve also implemented the Logics that control Doors, switches and other interactive objects inside the dungeons. Basically this means that the dungeon interaction is nearly finished and I’ll probably be moving on to exteriors in the coming week and finally getting everything back to where it should be. :)

Another area that I’ve been thinking about is the editor that will be coming after these sets of builds are complete, as you’ll see below.

Logics define the data that they need, per object, in order to operate. This data can simply be space to store state, such as the current animation time. However this data is oftentimes derived from the objects themselves or loaded data. In order to link data that the logic requests per-object to the engine data semantics are used. As data is loaded or defined, a particular variable may register a semantic with the Logic/script system. When a Logic is assigned to an object the semantics are matched up and the data setup accordingly. But how do objects receive Logics? For existing data, the data format loader manually assigns logics based on the object type. For example “Logic_Door” is used for door objects in Daggerfall. When the editor comes online, the user will be able to assign whatever Logic they want – it is simply stored as a name string. In fact an object can have any number of Logics, which simply are processed in the order that they are added. In addition there will be editor semantics for the Logic data declaration – and these semantics and UI meta data defined inside the scripts or code will be used to generate the editor UI needed to assign data to an object in the editor.

As an example, let’s say you wanted to add a new door type for DaggerXL. You would create a new Logic script file where you create the Logic. In the data declaration you define the data the Logic needs per-object and assign the appropriate semantics and UI meta data. You can make the UI use edit boxes, sliders, check boxes or other appropriate control including sensible limits. Then simply include this Logic file in the Core Logics UI file for your mod and the engine and editor will pick it up automatically. In this way behaviors can be added or edited in seconds and the editor immediately allows you to start using the Logics with an appropriate UI.



As I’m sure you’re all aware, I’ve been adding software rendering support for a variety of reasons. Among these reasons include: increased compatibility, closer emulation of the original look, as a reference for figuring out and matching up the rendering for all modes as close as possible. However I ran into a problem: there are a variety of feature combinations that I wish to support and I don’t want to add a bunch of code or conditionals to the scanline rendering – since it is the most time consuming part of the rasterizer. What I had been doing is hand coding a variety of scanline drawing functions that used only the features that were necessary but it became obvious that this wasn’t going to scale. For example, the 32 bit renderer became out of date with the 8 bit renderer that I had been focusing on.

The shader system (OpenGL, shader-based renderer) uses #defines to allow a few shaders to be coded, the shader code actually used is based on this base code with the appropriate defines set. For every set of features shader code is generated that fulfills the requirements without requiring a large number of if statements in the shader code or having to write a bunch of redundant code that may become out of date.

So I basically decided to implement the same system for scanline rendering in C/C++. There is one routine for scanline rendering with defines for the various feature permutations. When the engine renders triangles, the appropriate permutation of the scanline rendering routine is used to render the triangles with as little code in the inner loop as possible. So now adding new features to the renderer has become easier without becoming out of date. Now the 32 bit renderer has all the features of the 8 bit renderer and vice versa, for example.

One feature that I added is translucent surface rendering, which now works even in the 8 bit renderer. This will allow for certain effects for every renderer, from simple things such as translucent backgrounds for UI windows and the console to additive blending for things like torch flame. Some examples of alpha blending in the 8 bit software renderer:


The way it works in 8 bit is through the use of translucency tables. Basically the final source palette index and the destination pixel index are fed into a table which contains the closest palette index to the proper blended result. This functionality will allow me to add other blending modes, such as additive, in the future. This also means that colored lighting could be added to the software renderer in the future and other filtering affects, as much as the palette allows anyway.

For the 32-bit renderer, I’ve been working on a color table that will be able to match the 32 bit color lighting/shading results to the original colormap better. This remapping is working, though I have some tweaking to go before it really does the job properly. However I’m getting some pretty positive results for things like the colormap used for underwater and for fog, I’ll show these next time. There is still much more contrast in the 32 bit renderer then the 8 bit renderer though, however the quality is improved in some ways. You can see for yourself in the comparison shots below. Clearly I still have more work to do in this area. :)

Once the 32-bit color map is working properly, I will use the same colormap with the shader based OpenGL renderer as well, allowing for much of the effects afforded by the colormap and the quality and speed of the hardware rendering. This is one example of how the software renderer is helping me improve the hardware rendering as well.

Below are the comparison shots. Notice that there is still more tweaking to be done before it matches, though. But the beginnings are working. :)



In the next post should feature a video and exteriors. After that comes the DaggerXL part of the iterative XL Engine merger release.

One of the key elements of Alien Awakening is the portrayal of intelligent alien individuals who appear to think on their own and have the ability to converse with each other and the player. One of the most important aspects of this is language. One of the most important things that separate human beings from other, non-sentient animals such as dogs, is our language. Language is not just a tool for communication but a tool for thought itself. Our language defines the elemental concepts of conscious thought – that which lies beyond the basic qualia of feeling and need. By changing the language, not just the language we are speaking but the language that we use to form our conscious thoughts, we change the way we think. As an example, one could imagine a species like the Borg from Star Trek – their language would have no words forI or self. As a result they cannot form thoughts of individuality or self – it is a completely foreign concept. Before they could think of themselves as individuals, they would need to learn how to think of individuality.

It follows, then, that the language in Alien Awakening performs several very important tasks: it forms a means of communication between the player and aliens, it forms a visible form of communication between the aliens themselves to make them feel more alive and finally it forms a framework for the alien thought process itself. This thought process would inherently be alien – the concepts expressible in the alien language would certainly be different then our own – and yet understandable if you learn to think in the alien language (or at least understand the concepts that they can express) . This means that the alien AI must express “conscious” thought in their own language – if it is not expressible then it cannot be thought of at a conscious level. True conscious thought can not be achieved at this level, obviously but the goal is to give the appearance – at least in this constrained problem space. Of course, at first this seems crazy, but the AI would need to be scalable. When dealing with individuals, each individual has it’s own thoughts and perceptions. When talking about groups acting out of sight or far away, a group AI would take over. It would still track the individuals – similar to particles in a massive particle system – but work at a much higher level. This would allow individual behavior when needed as well as group behavior for the rest of the world.

So, then, the language should be relatively simple but must be complex enough to give the aliens adequate – even rich – expressiveness of thought and action. First thing, the basic rules of the language must be simplified. For example, in English, there are a large number of phonemes (a group of sounds that serve the same purpose in a language) which each have a number of allophones (individual sounds that belong to the same phoneme). To add additional complexity, there is a poor match between spelling and phonemes. Several things will be simplified here – each phoneme will have exactly one allophone, one sound. Each phoneme will also be represented by a single symbol or letter. In addition the number of unique phonemes will also be reduced, allowing players and aliens to “speak” directly in phonemes – greatly reducing the complexity at the base level.

There is a lot more to talk about when it comes to the language system but that will have to wait for future posts. Feel free to let me know what you think.

So I’m pretty close to being done with the lighting for DaggerXL. There is still some tweaking to be done but it is very minor at this point. Mostly I have to position the lights for different types of flats a bit differently and maybe tweak the animation and radius a bit more. The rendering for the OpenGL shader mode uses the refactored code from the old DaggerXL project, though tweaked for improved performance and to match the Daggerfall a bit better. However for this post I’ll be talking about the lighting for the software renderer.

Lighting in DaggerXL is rendered per-pixel, or close to it, for all lights even in the software renderer. The reason for this is simple, that was how it was handled in the original DOS game. Despite the numerous flaws with the X-Engine one of it’s coolest features was the real-time per-pixel lighting. Before Quake introduced lightmaps, lighting in software renderers was very simplistic at best. Games like Daggerfall were far too vast – especially in those days – for lightmaps to be practical. Per-vertex lighting, which was used primarily even with early GPU support, was also not very useful for Daggerfall due to the coarse nature of the geometry.

Obviously the number of lights per surface is very important to performance, so there are a couple of things done to keep this manageable:

Light Merging

When loading a dungeon block (or equivalent for other environments), any lights that are at nearly the same height are combined into a single light if their light volumes overlap enough. The position of the new “composite” light is simply the average of all the individual light positions. So, for example, if you have 3 torches right next to each other on one light is generated which would be on the center torch – or in the middle of the torches if they are arrayed in a triangular shape. This allows objects that generate lights, such as torches and candles, to be placed almost at-will without increasing the light count per-surface too much.

Here are a few examples:

3 torches in a line, one light is generated in the center. (8 bit software renderer, 1024×768)

Numerous candles in the rafters, this room actually looks much cleaner and runs much better with the light merging. All the candle lights are merged into a single light. (8 bit software renderer, 1024×768)



Aggressive Per-Polygon Culling and Light per-surface Limits

As suggested, the engine goes through a fair amount of work to cull lights aggressively to limit the number affecting a given surface. First lights that do not move are assigned to meshes during the load process. By doing this at load time the CPU doesn’t have to spend time doing this culling as the player moves around. When a mesh is rendered, this light list is sent to the low-level graphics system. As each polygon is being rendered only the lights from this list that may affect that polygon are used for the per-pixel lighting. The light sphere must overlap the polygon bounding sphere, the light must be above the polygon’s plane (otherwise it is behind the polygon and thus won’t have any visual affect anyway) and the true light to polygon distance must be within range to be affected by the light. From this final list of lights the 4 most likely to have the greatest impact are chosen to light the polygon. By limiting the maximum number of lights per-polygon, the worst case performance is known in advance.


Per-pixel Lighting in Software

It would probably be more accurate to call this per-span lighting, though it is very close to per-pixel. Anyway the relative world space position (i.e. world space – polygon vertex 0, in order to preserve precision with fixed point) is assigned as vertex attributes and is interpolated across the edges and per-span. Basically the rasterizer splits polygons into scanlines and each scanline is then split into Affine Spans, with proper perspective interpolation for the span edges. For resolutions 640×480 and smaller the Affine spans are 8 pixels in size or smaller (such as on edges). All the interpolaters are perspective correct at the span start and end position and are linearly interpolated in-between.

Anyway the relative world space position is perspective correct at the span ends and that is where the lighting is computed. A list of up to 4 lights is processed and then the result is modified by the fog, which is based on the Z distance – the same Z value used for z-buffering. For each light the linear attenuation is computed as well as the dotproduct of the polygon normal – which is a single value for Daggerfall since only polygon normals are used – and the normalized light direction. These values are then linearly interpolated across the spans. In order to make this fast, a table lookup is used. Since we use 16.16 fixed point, we know how much precision is needed and can thus use fast approximations that have sufficient precision.


Picture Time

320×200, 8 bit, software renderer


1024×768, 8 bit, software renderer


The Cave

You blink, attempting to shake off the grogginess, it feels as if you’ve been asleep for days. As your visions clears, it slowly dawns on you that you are not where you should be. You vaguely remember a flashing of lights and blaring alarms, people running about in confusion. Of what? You don’t remember, it’s such a blur. Then blackness, where you knocked out? Around you there is nothing but glistening rock illuminated by the soft glow of the strange looking platform on which you now lay. Something feels strange… you look down upon your body and scream – sound echoing off the cavern walls. Where your body once was, you see another body – you have 2 arms and 2 legs but they are completely alien, unlike any creature on Earth. After laying, confused and shocked, for what seems like an eternity you finally decide to try and figure out what’s going on. Surprisingly you have no problem controlling your new body and get up with little effort.

As you look around you see evidence of other machines, at least you think they are machines, but nothing you’re familiar with. You start looking for a way out, maybe you can get some answers outside, maybe even find out who’s responsible and get them to change you back. The cavern seems to have a simple layout, it doesn’t take long to find the lone tunnel that leads out of the room you started in. You see small machines and a few strange plants – well you assume they are plants anyway, no point in getting too close just in case – but nothing else other then the glittering stone walls. There’s not much light but enough thanks to the machines.

Eventually you see a single book lying on the ground, an oddly familiar yet strange sight in these caves. It’s a little damp but the pages are thick and whole. You open it to finally see something you understand – it appears to be a journal, though you suspect the author may not be entirely sane. He claims that this cave is not on Earth at all! Yeah, so I woke up on some alien world. But you continue to read… waking up in an alien body… cave with strange machines and even stranger plant life… It claims that there is a settlement not far from here. Then it goes on to describe some of the basics of their language. The aliens here have never heard any English, I have to speak their language for them to understand me… This must be some kind of mind game, you feel the need to get out of this cave and find out what is really going on.



You finally stumble into the sunlight, it takes your eyes a while to adjust but when they do… Where are you? You look around stunned, unable to move as you stare at the landscape… So alien… As you begin to regain your composure it dawns on you that the journal was right after all. You are on an alien world, in an alien body with no idea how you got there or how you can get home. With nothing else to go on, you head in the direction of the alien settlement.

And so begins the Alien Awakening.

The dungeon geometry refactor is almost complete. During this process several issues that are present in the current DaggerXL release are also being fixed – which should make DaggerXL much more polished and solid after upcoming release. Below I’ll list a few of the items that are being fixed:

* Textures – DaggerXL still had some textures assigned incorrectly. This is being fixed for the next release and should be finished soon.

Some areas that were incorrectly textured in DaggerXL which are now working correctly:


* Texture coordinates – There were some cases in DaggerXL where textures were not mapped correctly to surfaces. This oftentimes led to distortions that did not exist in the original.

In addition there was a sub-texel error in the mapping, basically the way coordinates are mapped to polygons led to an off by 1/2 texel offset problem. Look at the screenshots below to get an idea of what the effect is:

Before (click to see a larger view) – Notice the bright spots where the bottom of the door frame meets the wall of the door frame.

After (click to see a larger view)

* Flat scale and offset – There were errors in the scale and positioning of flats that is being fixed for the next release. This will fix things like flats being too big or small and flats that hovered over the ground instead of sitting on it.

* Object rotations – There were errors in DaggerXL with objects that were rotated on all 3 axes at the same time. This was caused by a different order of rotations then Daggerfall used.

There are a variety of other non-graphical issues that will be fixed as well including: bodies falling through the ground or disappearing, UI issues such as not being able to click on certain buttons and the in-game mouse cursor being offset from the real cursor, DaggerXL not always honoring focus correctly and more.

In the past DarkXL and DaggerXL releases, both games had custom “XL” title screens. These served multiple purposes: a place to display the version number – a very useful piece of information to know for a variety of reasons, a place for extra  XL specific options such as Mods and multiplayer setup and as a nice extra element of polish.

However as of the next release you’ll need to launch the games using the Launcher, which could already be done in the previous build as well. The Launcher is where you’ll be able to see the version number and can setup Mods or other XL specific features. To that end I think the XL specific title screens have served their purpose but are no longer needed. Starting from the next build when a game is launched the game menus will be displayed immediately, just like in the vanilla versions.

There are several reasons for this change, other then simply not needing the title screens any longer:

* Adding new games no longer requires new custom art work.

* Later I plan on adding the ability to switch between games without shutting down the XL Engine.

* Once you start a game, focus is entirely on the game itself and not on the XL Engine, unless XL Engine specific features are used such as the console or XL Engine settings menus.

* A unified XL Engine interface is presented, no longer being game-specific.


In addition to the Title Screen change, there will also be a change to the way versioning is done. Instead of having a version per game, there will be a global XL Engine version. Whenever the support for a given game is improved  the global version will be modified as well. Of course how complete support is for a given game will still be tracked but only one version number needs to be used for bug or feature tracking.


Alien Awakening is an original game that I plan on developing on the XL Engine. It is something I’ve thought about for years and I’ve already worked about some of the concepts and details, though no real development work has been done. I will continue to focus on the main XL Engine and finishing the support for the current games before even starting development on Alien Awakening. Even once started, this will be a long term project and the XL Engine will still be focused on supporting classic games.

That said I have a back log of material, concepts, thoughts, story, gameplay and so forth. I will be making posts about it, probably once a week or two. These posts won’t replace the regular XL Engine release news or affect the planned releases in any way. However it’ll supplement the normal content.

As a warning, this game may sound crazy and undoable – but I will chart my progress here as I develop it, so you can see for yourselves.  It will also require the XL Engine to be extended far beyond what it would normally need to be… but that’s ok – the new features will be useful for the other supported XL Engine games as extended features.

Alien Awakening

Alien Awakening is a game/simulation that I plan on building in the future and will be my most ambitious project ever, so I need to plan before I begin, as well as researching many systems before I begin to code the project itself. So while I work on my other projects, I will post my thoughts on Alien Awakening here – any research, test programs and other thoughts.

What is Alien Awakening?

Alien Awakening is a game/simulation of an alien world. You play as a human stranded on this world, somehow inhabiting the body of an alien. When you speak, the sounds are alien to your ears and completely foreign. You can spend your time in this world trying to figure out how you got there, what happened to your real body and how to get home. As you attempt to unravel those mysteries you discover at least one sentient alien race, primitive but unique. You’ll discover, though, that there is more to this world – to these people – then you’d ever expect. But this isn’t a linear game or adventure, at it’s heart it is a simulation. The aliens speak and understand their own language, you can learn to converse with them – even to live with them. You can lead or follow, help or hinder, learn or destroy. The goal is to make you feel like you are on a real world – everything is simulated or procedural. Everything changes, the seasons, animals migrate, alliances form and shatter, creatures grow old and die, new life is born and raised. The game never ends, even if you find out how to go home you may decide that you’d rather stay and continue to live with these people.

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.