The XL Engine is and will remain free, donations are purely optional but greatly appreciated.

Recent Comments

Archive for November 2013

Unfortunately time was tight this past week, reducing the number of updates I could write and the amount of work being done. I have loaded the character data, faction data, setup the current region and loaded the location data for my save game.

For the moment I’m working on getting dungeons fully loaded from the save games, which has had me revisiting the dungeon loading code again. It turns out this process has allowed me to clean up the code a bit, with better understanding gained by approaching the code from another angle.

So I’ll describe a bit of the way Daggerfall interprets dungeon data, which is a little different then DaggerXL did before this point. I’ll also explain a little about how objects are allocated and used.


Textures Revisited

First, you’ll recall the texture assignment code I showed before for dungeons when explaining how random numbers were involved. In that function there were calls to srand() and rand(), which were valid, and a call to RandByte(). I hadn’t fully fleshed out this function, thinking I’d revisit the random number generation stuff later. Well it turns out its both simpler and more complex then I originally thought.

It turns out that function is specific to dungeon textures, so I now call it DungeonTex_RandByte(). But basically another section of code needs to be called first to generate a “key,” which is derived from the map coordinates which happens during cell loading or save game loading depending on whether you’re loading from a save game, by starting a new game or by clicking on a door. It is generated like this:

dungeonLocTexKey = GetDungeonTexKey( mapX, mapZ, dungeonKeyTable )

In this case mapX and mapZ are not the exact map coordinates but offset slightly from normal:

mapX = worldX/32768 + 2
mapZ = 499 - (worldZ/32768)

if ( mapZ < 1 )     
    mapZ = 1;   
else if (mapZ > 499 )
    mapZ = 499;


Here is the code to build the texture key for the current location. I don’t show the key table because it is pretty big – 500 entries.

struct TexKey_XOffsetEntry
   word xOffset;
   byte key;

byte GetDungeonTexKey(int mapX, int mapZ, TexKey_XOffsetEntry *keyTable)
   TexKey_XOffsetEntry *keyEntry = &keyTable[mapZ];
   mapX -= keyEntry->xOffset;
   while (mapX > 0)
      mapX -= keyEntry->xOffset;
   return keyEntry->key;


Then the key is used DungeonTex_RandByte, as follows:

byte DungeonTex_RandByte()
    return dungeonTexTable[ dungeonLocTexKey ];


Which is used to actually generate the texture table (you’ve seen this code before):

word textureTableSrc[5]=;   //0x28617C
{ 119, 120, 122, 123, 124 };
word textureTableCur[5];   //0x286186

void InitializeTextureTable()
   int r0 = rand();
   srand( curLocationRec->locationID>>16 );
   byte r = DungeonTex_RandByte();
   int r2 = _texRandTable[ r ];

   memcpy(textureTableCur, textureTableSrc, 10);
   for (int i=0; i<5; i++)
      byte n = random_range(0, 4);
      if ( n == 2 )
         n += 2;
      n += textureTableSrc[r2];
      textureTableCur[i] = n;


… the process looks convoluted and well… it is.


Dungeon Blocks

As you already know dungeons are loaded as “blocks” – each of which is an RDB and RDI file that identifies all the objects, models, quest locations, monsters, loot piles and so on for the dungeon block. For the purposes of culling and rendering, each block is sub-divided into 2×2 sub-blocks. RDB files contain a grid of object root offsets – each of which maps to one of these 2×2 sub-blocks. As it turns out, the game always uses 2×2 sub-blocks, even though RDB files can (and usually do) specify more. Even though these blocks must be referenced based on grid width/height, only the upper left 2×2 block is actually used (the rest of the data is -1).

Daggerfall creates a sub-block game object for each sub-block, which is used as the parent object for the sub-block objects. For rendering, the game loops through each visible sub-block, then traverses the link list of child objects to render, collide with, etc.. Links between objects, for things like switches, actually occur within these sub-blocks, though the height difference can be immense, and the distance can seem quite large due to the windy nature of the corridors.

Each dungeon block is 2048 x ? x 2048 game units, each sub-block being 1024 x ? x 1024 units. Note that the height value is a question mark since there aren’t any hard-coded limits, as far as I’ve seen so far.


Game Objects

When Daggerfall starts up, a 3584000 byte (almost 3.42MB) object pool is allocated. Each object is 71 bytes is size, with extra data that can be allocated based on type. In addition 18 bytes of addition overhead is used for things like previous and next pointers.

Each game object in Daggerfall – things ranging from 3D objects, lights, flats, sub-block objects, the player, NPCs, enemies, etc. – use this pool, with the type specific data being allocated after the 71 bytes. This data contains information such as the world position (in “units” – basically inches), flags, unique identifier, link list data, type and so forth.

So, while the code is pure C, this type of structure allows for some object oriented style behavior – you can think of the Object as being the base class. And they are actually called objects in the original code, as evidenced by the following error message if no object can be found:

if ( !obj )
    printf( "Unable to allocate OBJECT memory." );

Anyway you can use the type parameter to determine which structure to cast the type-specific data to, at offset 71.

I’m hoping to be able to show screenshots soon, fortunately by coding support for save games now I can make sure that all locations of a given type work – so my first dungeon shots will not be of Privateer’s Hold. :D

The last couple of days have been slow due to work, Halloween and such, so I’m still working on loading save games. However, to keep the news flowing, I’ll talk about a specific element – factions.

When loading a save game or starting a new game the first thing that needs to happen is to load the base faction data from FACTION.TXT. The parser first frees all the previous faction data and then counts the number of factions – by counting the number of times ‘#’ shows up in the text file. At this point, the parser looks for certain symbols: ‘#’ for faction ID and ‘:’ for tags. Tags themselves are hashed, with a callback function being called for any matching table entry using the following table:

struct FactionTxtTag
   word tagID;
   factionTagCB func;

FactionTxtTag factionTags[19]=   //2865BC
   { 0x06C9, Faction_ProcessTypeTag   },
   { 0x0633, Faction_ProcessNameTag   },
   { 0x0302, Faction_ProcessRepTag    },
   { 0x1C18, Faction_ProcessSummonTag },
   { 0x1AB8, Faction_ProcessRegionTag },
   { 0x0D90, Faction_ProcessPowerTag  },
   { 0x0C85, Faction_ProcessFlagsTag  },
   { 0x0609, Faction_ProcessAllyTag   },
   { 0x0CA7, Faction_ProcessEnemyTag  },
   { 0x0DB4, Faction_ProcessRulerTag  },
   { 0x05DF, Faction_ProcessFaceTag   },
   { 0x0616, Faction_ProcessFlatTag   },
   { 0x063F, Faction_ProcessRaceTag   },
   { 0x1B76, Faction_ProcessSGroupTag },
   { 0x19F6, Faction_ProcessGGroupTag },
   { 0x064E, Faction_ProcessMinFTag   },
   { 0x0642, Faction_ProcessMaxFTag   },
   { 0x065B, Faction_ProcessRankTag   },
   { 0x0307, Faction_ProcessVamTag    },

A few of the entries are ignored – the callback functions actually just skip past the data without recording it, but most fill out the Faction structure. The following tags are ignored: Summon (the game determines this using other methods), MinFMaxF and Rank. Some tags can occur more then once – up to 2 flats, 3 allies and 3 enemies for example.

Then once all the factions are processed, enemy and ally IDs are converted to faction pointers. Once all this is complete, faction data is read from the save games and overwrites the data (but leaves the pointers as-is). These then have their pointers fixed up again.

So the factions in the text file determine the default faction data, including reputation (which doesn’t always start at 0). Only those factions that have been modified or that the player are directly a part of are actually tracked in the save game files. The rest stay at their default values – except for one thing. Each time the faction data is loaded, two random numbers are generated which are used to help determine certain behaviors (more on this later).

Anyway, here is the final Faction structure – I keep track of the actual hex offsets for each variable to make it easier for me to convert from offsets in the assembly code to the variable in the structure.

//sizeof(Faction) = 92
struct Faction
   byte type;                //0x00
   char region;              //0x01
   byte ruler;               //0x02
   char name[26];            //0x03
   short rep;                //0x1D
   short power;              //0x1F
   short id;                 //0x21
   short vam;                //0x23
   word flags;               //0x25
   dword rndValue1;          //0x27
   dword rndValue2;          //0x2B
   short flats[2];           //0x2F
   word face;                //0x33
   char race;                //0x35
   byte sgroup;              //0x36
   byte ggroup;              //0x37
   Faction *allies[3];       //0x38
   Faction *enemies[3];      //0x44
   //pointers used for traversing the list, these are not saved or loaded.
   Faction *child;           //0x50
   Faction *parent;          //0x54
   Faction *prev;            //0x58

Obviously I already know how Bethesda hashed the tag names, since I reverse engineered the code that does it, but as a fun puzzle let’s see if any of you guys can figure it out – given the data below. This is one kind of puzzle that is often involved with this work since I oftentimes get data that is referenced but understanding the code requires understanding what the data actually means – which is very often not explicit in the code (though it was this time). Unfortunately they are not always as simple as this one. :P


 INPUT          OUTPUT (hex)     OUTPUT (base-10 "normal")
   type           0x06C9               1737
   name           0x0633               1587
   rep            0x0302                770
(see the table above for more examples)


Important clues:
*case matters
*you’ll want to consult an ASCII table (search ASCII table on google)
*the number of characters has a fairly large impact on the result
*the transform is simple

*Convert from the tag name, such as type, to the number used in the lookup table using a mathematical transform on the characters.

Post here if you think you’ve figured it out. :)

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.