Translate
EnglishFrenchGermanItalianPortugueseRussianSpanish

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.

It has been a long time since my last update but my life has been undergoing a series of changes – for the better – that have occupied my time. I have started to settle in and have resumed working on the DaggerXL Beta. In this post I will talk a little about the Daggerfall Renderer, starting with some simple basics.

Below is the Object structure – the structure itself (i.e. size of variables and such) is fully known though a few members still do not have appropriate names yet, u1, u2, u3. As you can see the world space position is stored in (xPosition, yPosition, zPosition) in inches and  the rotation angles are stored in angles[3]. Positions are stored in absolute world space – this position is not relative but the actual position on the world map. The angles range from 0 – 2047 – which maps to 360 degrees. In other words 512 = 90 degrees, 2048 = 360 degrees.

struct Object
{
   //0x00
   byte type;
   //0x01
   word angles[3];
   //0x07
   int xPosition;
   //0x0B
   int yPosition;
   //0x0F
   int zPosition;
   //0x13
   word index2;
   //0x15
   word flags;
   //0x17
   word dataSize;
   //0x19
   word index;
   //0x1B
   word arrayIndex;
   //0x1D
   word model;
   //0x1F
   int ID;
   //0x23
   byte u1;
   word u2;
   byte u3;
   //0x27
   int curLocID;
   //0x2B
   dword time;
   //0x2F
   Object *target2;
   //0x33
   Object *target;
   //0x37
   Object *siblingNext;
   //0x3B
   Object *siblingPrev;
   //0x3F
   Object *child;
   //0x43
   Object *parent;
};

To generate a rotation transform for an object or for the camera Daggerfall uses the following matrix:

Given angles: x=xAngle, y=yAngle, z=zAngle – the Daggerfall rotation matrix =

[ cos(z)*cos(y) + sin(x)*sin(z)*sin(y)  -sin(z)*cos(x)     sin(z)*sin(x)*cos(y) ]
[-sin(x)*sin(y)*cos(z) + sin(z)*cos(y)   cos(x)*cos(z)    -cos(z)*sin(x)*cos(y) ]
[ sin(y)*cos(x)                          sin(x)            cos(x)*cos(y)	]

The 3×3 rotation matrix is stored in 1.3.28 fixed point.

Here is the function used to generate the 3×3 rotation matrix from the angles:

void Build3x3Matrix(int xAngle, int yAngle, int zAngle, int *matrix)
{
    int x = xAngle&0x07ff;
    int y = yAngle&0x07ff;
    int z = zAngle&0x07ff;

    const int64 half = 134217728LL;
    int64 c = (int64)sinTable[z] * (int64)sinTable[y] + half;
    camAngle0 = (int)( c >> 28LL );

    c = (int64)sinTable[y] * (int64)sinTable[512+z] + half;
    camAngle1 = (int)( c >> 28LL );

    c = (int64)sinTable[x] * (int64)sinTable[512+y] + half;
    camAngle2 = (int)( c >> 28LL );

    c = (int64)sinTable[512+z] * (int64)sinTable[512+y] + 
        (int64)sinTable[x] * (int64)camAngle0 + half;
    matrix[0] = (int)( c >> 28LL );

    c = -(int64)sinTable[z] * (int64)sinTable[x+512] + half;
    matrix[1] = (int)( c >> 28LL );

    c = (int64)sinTable[z] * (int64)camAngle2 + half;
    matrix[2] = (int)( c >> 28LL ) - camAngle1;

    c = -(int64)sinTable[x] * (int64)camAngle1 + 
         (int64)sinTable[z] * (int64)sinTable[y+512] + half;
    matrix[3] = (int)( c >> 28LL );

    c = (int64)sinTable[x+512] * (int64)sinTable[z+512] + half;
    matrix[4] = (int)( c >> 28LL );

    c = -(int64)sinTable[z+512] * camAngle2;
    matrix[5] = (int)( c >> 28LL ) - camAngle0;

    c = (int64)sinTable[y] * (int64)sinTable[x+512] + half;
    matrix[6] = (int)( c >> 28LL );

    matrix[7] = sinTable[x];

    c = (int64)sinTable[x+512] * (int64)sinTable[y+512] + half;
    matrix[8] = (int)( c >> 28LL );
}

A few things to note – the fixed point format is 1.3.28 so 64 bit math is required to avoid overflows, fortunately x86 assembly makes this fairly easy to do quickly. Also note that Daggerfall has a sine table that stores the sin values for all angles ranging from 0 to 2047 (remember that this maps to 0 to 360 degrees). As you can see the cosine values are computed as sinTable[angle+512] – since sine and cosine are out of phase by 90 degrees, we can compute cosine as cos(angle) = sin(angle+90degrees) which Daggerfall does to limit the size of the table.

The camera uses the same position and angles to generate the view and projection matrices. However the way positions are transformed into screenspace are a little different from most modern engines. The projection matrix is build directly from the view matrix by scaling by the x relative and y relative screen aspect ratios, built as follows:

void BuildAspectScaledMatrix(int *rotMatrix, int *outScaledMatrix)
{
    int64 c = (int64)rotMatrix[0] * (int64)screenAspectX;
    outScaledMatrix[0] = (int)( c >> 14LL );

    c = (int64)rotMatrix[1] * (int64)screenAspectX;
    outScaledMatrix[1] = (int)( c >> 14LL );

    c = (int64)rotMatrix[2] * (int64)screenAspectX;
    outScaledMatrix[2] = (int)( c >> 14LL );

    c = (int64)rotMatrix[3] * (int64)screenAspectY;
    outScaledMatrix[3] = (int)( c >> 14LL );

    c = (int64)rotMatrix[4] * (int64)screenAspectY;
    outScaledMatrix[4] = (int)( c >> 14LL );

    c = (int64)rotMatrix[5] * (int64)screenAspectY;
    outScaledMatrix[5] = (int)( c >> 14LL );

    outScaledMatrix[6] = rotMatrix[6];
    outScaledMatrix[7] = rotMatrix[7];
    outScaledMatrix[8] = rotMatrix[8];
}

Note that the screenAspectX and screenAspectY are stored in 1.17.14 fixed point but the resulting projection matrix is still stored in 1.3.28 fixed point.

Positions in view space are stored as 1.23.8 fixed point so a position can be transformed as follows:
Given (x,y,z) in absolute world space:
(Note s_xPosition, s_yPosition and s_zPosition are the camera positions extracted from the camera objects and used by the renderer).

//Convert to camera relative coordinates, stored in 1.23.8 fixed point), still in inches.
int viewX = (x - s_xPosition)*256;
int viewY = (y - s_yPosition)*256;
int viewZ = (z - s_zPosition)*256;
TransformPoint(&viewX, &viewY, &viewZ, projMatrix);

void TransformPoint(int *x, int *y, int *z, int *matrix)
{
    int64 xp = (int64)(*x)*16LL;
    int64 yp = (int64)(*y)*16LL;
    int64 zp = (int64)(*z)*16LL;

    *x = (int)( ( xp*(int64)matrix[0] + yp*(int64)matrix[1] + zp*(int64)matrix[2] ) >> 32LL );
    *y = (int)( ( xp*(int64)matrix[3] + yp*(int64)matrix[4] + zp*(int64)matrix[5] ) >> 32LL );
    *z = (int)( ( xp*(int64)matrix[6] + yp*(int64)matrix[7] + zp*(int64)matrix[8] ) >> 32LL );
}

Finally the following functions are used to cull the bounds, stored as spheres – center in relative camera coordinates and radius in inches.

enum ClipPlanes_e
{
    PLANE_NEGX =  1,
    PLANE_POSX =  2,
    PLANE_POSY =  4,
    PLANE_NEGY =  8,
    PLANE_NEAR = 16,
    PLANE_FAR  = 32,
    PLANES_ALL = PLANE_NEGX | PLANE_POSX | PLANE_POSY | PLANE_NEGY | PLANE_NEAR | PLANE_FAR
};
static int nearPlane=2560;	//[1E9CC0] :the near plane is 10 inches.
static int farPlane =393216;	//[1E9CC4] :the far plane is 128 feet 
                                //(at normal maximum settings).

int CullPlaneDistX(int xScaled, int yScaled, int radius)
{
    int r = (zRadius*xScaled)>>16 + (xRadius*yScaled)>>16;
    if ( r < 0 ) 
    { 
        r = -r; 
    }
    return r+radius;
}

int CullPlaneDistY(int xScaled, int yScaled, int radius)
{
    int r = (zRadius*xScaled)>>16 + (yRadius*yScaled)>>16;
    if ( r < 0 ) 
    { 
        r = -r; 
    }
    return r+radius;
}
//Is the sphere: center = (x,y,z), radius = r at least partially visible?
//Returns 0 if visible else 1.
//Note that the sphere position must be relative to the camera and be in 1.23.8 fixed point format.
int IsSphereVisible(int x, int y, int z, int r)
{
    //transform into projection space.
    TransformPoint(&x, &y, &z, projMatrix);

    //figure out which planes the sphere needs to be tested against.
    int clipFlags = PLANES_ALL;	//6 planes = 111111 binary = 63
    if ( z >= nearPlane )
    {
        clipFlags ^= PLANE_NEAR;
    }
    if ( z <= farPlane ) 	
    { 
        clipFlags ^= PLANE_FAR; 	
    } 	
    if ( x >= -z )
    {
        clipFlags ^= PLANE_NEGX;
    }
    if ( x <= z )
    {
        clipFlags ^= PLANE_POSX;
    }

    if ( y <= z ) 	
    { 
        clipFlags ^= PLANE_POSY; 	
    } 	
    if ( y >= -z )
    {
        clipFlags ^= PLANE_NEGY;
    }

    //compute plane data.
    xRadius = (int)( ((int64)x*(int64)x*(int64)ScreenAspectA   )>>32LL );
    yRadius = (int)( ((int64)y*(int64)y*(int64)ScreenAspectA_SC)>>32LL );
    zRadius = z;

    //test against each plane based on the clipflags set (see above).
    if ( clipFlags ) 
    {
        if ( s_clipFlags&PLANE_NEAR ) 
        {
            if ( z+r <= nearPlane ) 				
                return 1; 		
        } 		
        if ( s_clipFlags&PLANE_FAR ) 		
        { 			
            if ( z-r >= farPlane )
                return 1;
        }

        if ( s_clipFlags&PLANE_NEGX )
        {
            if ( CullPlaneDistX(ScreenX_Scaled, ScreenH_Scaled, r) <= 0 )
                return 1;
        }

        if ( s_clipFlags&PLANE_POSX )
        {
            if ( CullPlaneDistX(ScreenX_Scaled, -ScreenH_Scaled, r) <= 0 )
                return 1;
        }

        if ( s_clipFlags&PLANE_POSY )
        {
            if ( CullPlaneDistY(varD34, -varD30, r) <= 0 )
                return 1;
        }

        if ( s_clipFlags&PLANE_NEGY )
        {
            if ( CullPlaneDistY(varD34, varD30, r) <= 0 )
                return 1;
        }
    }

    return 0;
}

Anyway that is enough about the renderer for this post but I will talk about the lighting system, more about culling objects and other topics in future blog posts. If you want to see the original assembly for these functions, visit the Renderer Part I – Original Functions in Assembly page. As you will see I have rearranged the code a little to make some things more clear.

I have resumed work on the DaggerXL Beta after taking a break for Christmas. I don’t have much new to show yet but I will talk about a few topics.

WebUI

On the XL Engine forums there has been mod work towards using the WebUI system that will be present in the Beta release. I talked about this system before in a previous blog post but now Lazaroth has been busy working on a UI mod, using existing web technologies to prototype. I dropped the files into the WebUI folder under the XL Engine and modified a bit of code (that will be externalized for the release) and was able to view the new UI in the current version of the XL Engine. There are a few issues to work out but its a great start and will allow me to work out kinks in the WebUI system. It also shows that using HTML5/CSS/Javascript and a browser is a great way of prototyping new UI ideas and the results can be nearly “drop-in” for use in the XL Engine.

LazarothUI_XLEngine

 

 

If you want more information on the mod or to help out, check out Lazaroth’s thread on the forums.

 

Anatomy of a For-Loop in Daggerfall

So what does a for-loop look like in Daggerfall? It turns out that recognizing for-loops generated by the compiler is rather simple as you’ll see shortly. I have copied some of the actual assembly code with comments.

mov dword [ebp-0018],00000000
[19A283]				;for (int i=0; i<27; i++) {
cmp dword [ebp-0018],001B               ;//comparison block
jl 0019A293
jmp 0019A2D1
[19A28B]			        ;//for-loop counter block
mov eax,[ebp-0018]
inc dword [ebp-0018]
jmp 0019A283
[19A293]			        ;//code block
...
jmp 0019A28B
[19A2D1]				;} //for (int i=0; i<27; i++)

The actual assembly code is on the left, with code addresses shown in brackets [ ]. As you can see the local variable, ‘i’ as I named it, is at ebp-0×18. So the first thing that happens is to fill the value with 0, basically the i=0; part of the for-loop. Next it enters the comparison block where the local variable is compared to a value – in this case 0x1B = 27, and if the comparison is successful the execution jumps to the code block, otherwise it jumps to the end of the loop (the last line shown above).

When you see the C/C++ code

for (int i=0; i<27; i++)
{
    //code inside the loop
}

The “code inside the loop” is the code block above. Once the code is executed or a continue is hit – then the execution jumps to the for-loop counter block. Here the counter is incremented, decremented or otherwise modified before jumping back to the comparison block. Obviously if a break is encountered in the code block, the jump will lead directly to the end of the loop or to a another address which will have a jump to the end of the loop if the difference in address is too big for a “short” jump (usually). Remember that for-loops look different with modern compilers and sometimes the format is tweaked a bit even in Daggerfall depending on what the optimizer does. But this is essentially what a for-loop looks like, even nested loops have a similar, though recursive, structure.

The Beta

I have recently continued work towards the Beta release and will start posting updates again as additional progress is made.

Unfortunately I have a lot on my plate for this Christmas season, so I will be taking a break from XL Engine work until after Christmas. This has been happening for the last week or so, which is why I haven’t been posting updates. Anyway this is not caused by roadblocks or even stressful issues but rather by obligations I have to meet by the holiday.

Once Christmas is over I should have some extra time for a while, allowing me to make up for lost time a little. :)

Happy Holidays.

 
DXL_8bpp_Lighting3

Unfortunately time was tight this past week, reducing the number of updates I could write and the amount of work being done. I have loaded the character data, faction data, setup the current region and loaded the location data for my save game.

For the moment I’m working on getting dungeons fully loaded from the save games, which has had me revisiting the dungeon loading code again. It turns out this process has allowed me to clean up the code a bit, with better understanding gained by approaching the code from another angle.

So I’ll describe a bit of the way Daggerfall interprets dungeon data, which is a little different then DaggerXL did before this point. I’ll also explain a little about how objects are allocated and used.

 

Textures Revisited

First, you’ll recall the texture assignment code I showed before for dungeons when explaining how random numbers were involved. In that function there were calls to srand() and rand(), which were valid, and a call to RandByte(). I hadn’t fully fleshed out this function, thinking I’d revisit the random number generation stuff later. Well it turns out its both simpler and more complex then I originally thought.

It turns out that function is specific to dungeon textures, so I now call it DungeonTex_RandByte(). But basically another section of code needs to be called first to generate a “key,” which is derived from the map coordinates which happens during cell loading or save game loading depending on whether you’re loading from a save game, by starting a new game or by clicking on a door. It is generated like this:

dungeonLocTexKey = GetDungeonTexKey( mapX, mapZ, dungeonKeyTable )

In this case mapX and mapZ are not the exact map coordinates but offset slightly from normal:

mapX = worldX/32768 + 2
mapZ = 499 - (worldZ/32768)

if ( mapZ < 1 )     
    mapZ = 1;   
else if (mapZ > 499 )
    mapZ = 499;

 

Here is the code to build the texture key for the current location. I don’t show the key table because it is pretty big – 500 entries.

struct TexKey_XOffsetEntry
{
   word xOffset;
   byte key;
};

byte GetDungeonTexKey(int mapX, int mapZ, TexKey_XOffsetEntry *keyTable)
{
   TexKey_XOffsetEntry *keyEntry = &keyTable[mapZ];
   mapX -= keyEntry->xOffset;
   while (mapX > 0)
   {
      keyEntry++;
      mapX -= keyEntry->xOffset;
   };
   return keyEntry->key;
}

 

Then the key is used DungeonTex_RandByte, as follows:

byte DungeonTex_RandByte()
{
    return dungeonTexTable[ dungeonLocTexKey ];
}

 

Which is used to actually generate the texture table (you’ve seen this code before):

word textureTableSrc[5]=;   //0x28617C
{ 119, 120, 122, 123, 124 };
word textureTableCur[5];   //0x286186

void InitializeTextureTable()
{
   int r0 = rand();
   srand( curLocationRec->locationID>>16 );
   byte r = DungeonTex_RandByte();
   int r2 = _texRandTable[ r ];

   memcpy(textureTableCur, textureTableSrc, 10);
   for (int i=0; i<5; i++)
   {
      byte n = random_range(0, 4);
      if ( n == 2 )
      {
         n += 2;
      }
      n += textureTableSrc[r2];
      textureTableCur[i] = n;
   }

   srand(r0);
}

… the process looks convoluted and well… it is.

 

Dungeon Blocks

As you already know dungeons are loaded as “blocks” – each of which is an RDB and RDI file that identifies all the objects, models, quest locations, monsters, loot piles and so on for the dungeon block. For the purposes of culling and rendering, each block is sub-divided into 2×2 sub-blocks. RDB files contain a grid of object root offsets – each of which maps to one of these 2×2 sub-blocks. As it turns out, the game always uses 2×2 sub-blocks, even though RDB files can (and usually do) specify more. Even though these blocks must be referenced based on grid width/height, only the upper left 2×2 block is actually used (the rest of the data is -1).

Daggerfall creates a sub-block game object for each sub-block, which is used as the parent object for the sub-block objects. For rendering, the game loops through each visible sub-block, then traverses the link list of child objects to render, collide with, etc.. Links between objects, for things like switches, actually occur within these sub-blocks, though the height difference can be immense, and the distance can seem quite large due to the windy nature of the corridors.

Each dungeon block is 2048 x ? x 2048 game units, each sub-block being 1024 x ? x 1024 units. Note that the height value is a question mark since there aren’t any hard-coded limits, as far as I’ve seen so far.

 

Game Objects

When Daggerfall starts up, a 3584000 byte (almost 3.42MB) object pool is allocated. Each object is 71 bytes is size, with extra data that can be allocated based on type. In addition 18 bytes of addition overhead is used for things like previous and next pointers.

Each game object in Daggerfall – things ranging from 3D objects, lights, flats, sub-block objects, the player, NPCs, enemies, etc. – use this pool, with the type specific data being allocated after the 71 bytes. This data contains information such as the world position (in “units” – basically inches), flags, unique identifier, link list data, type and so forth.

So, while the code is pure C, this type of structure allows for some object oriented style behavior – you can think of the Object as being the base class. And they are actually called objects in the original code, as evidenced by the following error message if no object can be found:

if ( !obj )
{
    printf( "Unable to allocate OBJECT memory." );
}

Anyway you can use the type parameter to determine which structure to cast the type-specific data to, at offset 71.

I’m hoping to be able to show screenshots soon, fortunately by coding support for save games now I can make sure that all locations of a given type work – so my first dungeon shots will not be of Privateer’s Hold. :D

The last couple of days have been slow due to work, Halloween and such, so I’m still working on loading save games. However, to keep the news flowing, I’ll talk about a specific element - factions.

When loading a save game or starting a new game the first thing that needs to happen is to load the base faction data from FACTION.TXT. The parser first frees all the previous faction data and then counts the number of factions – by counting the number of times ‘#’ shows up in the text file. At this point, the parser looks for certain symbols: ‘#’ for faction ID and ‘:’ for tags. Tags themselves are hashed, with a callback function being called for any matching table entry using the following table:

struct FactionTxtTag
{
   word tagID;
   factionTagCB func;
};

FactionTxtTag factionTags[19]=   //2865BC
{
   { 0x06C9, Faction_ProcessTypeTag   },
   { 0x0633, Faction_ProcessNameTag   },
   { 0x0302, Faction_ProcessRepTag    },
   { 0x1C18, Faction_ProcessSummonTag },
   { 0x1AB8, Faction_ProcessRegionTag },
   { 0x0D90, Faction_ProcessPowerTag  },
   { 0x0C85, Faction_ProcessFlagsTag  },
   { 0x0609, Faction_ProcessAllyTag   },
   { 0x0CA7, Faction_ProcessEnemyTag  },
   { 0x0DB4, Faction_ProcessRulerTag  },
   { 0x05DF, Faction_ProcessFaceTag   },
   { 0x0616, Faction_ProcessFlatTag   },
   { 0x063F, Faction_ProcessRaceTag   },
   { 0x1B76, Faction_ProcessSGroupTag },
   { 0x19F6, Faction_ProcessGGroupTag },
   { 0x064E, Faction_ProcessMinFTag   },
   { 0x0642, Faction_ProcessMaxFTag   },
   { 0x065B, Faction_ProcessRankTag   },
   { 0x0307, Faction_ProcessVamTag    },
};

A few of the entries are ignored – the callback functions actually just skip past the data without recording it, but most fill out the Faction structure. The following tags are ignored: Summon (the game determines this using other methods), MinFMaxF and Rank. Some tags can occur more then once – up to 2 flats, 3 allies and 3 enemies for example.

Then once all the factions are processed, enemy and ally IDs are converted to faction pointers. Once all this is complete, faction data is read from the save games and overwrites the data (but leaves the pointers as-is). These then have their pointers fixed up again.

So the factions in the text file determine the default faction data, including reputation (which doesn’t always start at 0). Only those factions that have been modified or that the player are directly a part of are actually tracked in the save game files. The rest stay at their default values – except for one thing. Each time the faction data is loaded, two random numbers are generated which are used to help determine certain behaviors (more on this later).

Anyway, here is the final Faction structure – I keep track of the actual hex offsets for each variable to make it easier for me to convert from offsets in the assembly code to the variable in the structure.

//sizeof(Faction) = 92
struct Faction
{
   byte type;                //0x00
   char region;              //0x01
   byte ruler;               //0x02
   char name[26];            //0x03
   short rep;                //0x1D
   short power;              //0x1F
   short id;                 //0x21
   short vam;                //0x23
   word flags;               //0x25
   dword rndValue1;          //0x27
   dword rndValue2;          //0x2B
   short flats[2];           //0x2F
   word face;                //0x33
   char race;                //0x35
   byte sgroup;              //0x36
   byte ggroup;              //0x37
   Faction *allies[3];       //0x38
   Faction *enemies[3];      //0x44
   //pointers used for traversing the list, these are not saved or loaded.
   Faction *child;           //0x50
   Faction *parent;          //0x54
   Faction *prev;            //0x58
};

Obviously I already know how Bethesda hashed the tag names, since I reverse engineered the code that does it, but as a fun puzzle let’s see if any of you guys can figure it out – given the data below. This is one kind of puzzle that is often involved with this work since I oftentimes get data that is referenced but understanding the code requires understanding what the data actually means – which is very often not explicit in the code (though it was this time). Unfortunately they are not always as simple as this one. :P

 

 INPUT          OUTPUT (hex)     OUTPUT (base-10 "normal")
   type           0x06C9               1737
   name           0x0633               1587
   rep            0x0302                770
(see the table above for more examples)

 

Important clues:
*case matters
*you’ll want to consult an ASCII table (search ASCII table on google)
*the number of characters has a fairly large impact on the result
*the transform is simple

Goal:
*Convert from the tag name, such as type, to the number used in the lookup table using a mathematical transform on the characters.

Post here if you think you’ve figured it out. :)

As I discussed in previous updates, I have the cell loading code in place. Prior to that I already had the save/load menu fully functional, except for the actual saving and loading – selecting saves, sounds, double-clicking, etc.. So now its time to put those things together and fully support the Daggerfall save games.

Here is a screenshot of the menu, I’ve shown this screen before but ;) You’ll see my current save games, which are different then last time – and setup to make testing different kinds of areas easier. And, of course, this is taken from my DaggerfallDOS executable – where I test the reverse engineered code as-is before moving (with refactoring and/or rewritting) into DaggerXL.

DaggerfallDOS_Load2

Currently I’m working on parsing the faction data in faction.c and then loading the appropriate faction data from the save. I’ve already gone through most of the other setup – clearing out object links, deleting existing *.atf and *.amf files from the arena2/ folder, copying those files from the current save folder to arena2/, as well as copying rumor.datbio.dat, mapsave.sav. I’ve also loaded in all the data from savevars.dat.

Once that is complete I’ll start by making sure I can load various dungeons – I’ll be sure to show some in-dungeon screenshots when I that is done, then work on exterior and interiors. Once one type of area is fully functional, all areas of that type should be. Then I can start connecting completed pieces together (and completing pieces as needed) to get a fully functional game – quests, faction data, etc.

As I was working on finishing the loading code, see the Missing Code topic for more information, I was able to verify a few things about the coordinate systems used in Daggerfall.

Each location has a unique position on the world map – which, as you already know, is 1000×500 pixels in size. Each “pixel” on this map is exactly 32768 Daggerfall units on each side. Given the height of the character, 75 Daggerfall units, the original estimate of 1 inch per unit seems accurate – making the character exactly 6 1/4 feet tall. Thus a pixel is 32768 inches x 32768 inches, making it 51.7% of a mile (832.3072 meters) on each side. Finally “map pixels” are split into 256 unit sub-tiles – meaning each pixel has 128×128, 256×256 unit subtiles – each being 21 1/3 feet on each size. When outside, these sub-tiles are the terrain chunks.

Daggerfall uses an interesting way to determine which locations are near the player. Given the world space coordinates of the player (X and Z), the map coordinates can be calculated as follows:

const int maxWorldSpaceZCoord = 16384000;

mapPosX = playerObj->xPosition / 32768;
mapPosZ = (maxWorldSpaceZCoord - playerObj->zPosition) / 32768;

 

Then, given the position on the map, an index is computed:

const int mapWidth = 1000;

mapIndex = mapPosZ * mapWidth + mapPosX;

 

This index corresponds directly to the location ID’s. In other words the location ID is actually it’s map position encoded as shown above. This also means that its map position could also be derived directly from its ID.

Note the code was taken from reverse engineered functions, I just decided to show code snippets this time for brevity. Also Daggerfall used shifts instead of divides, but the divides make the code much clearer for the purpose of showing here.

In the spirit of providing small but more frequent updates as progress is made, here is another – smaller update.

It turns out that I missed some code for region loading, so I’ve been filling in the blanks for the last few days.

The core missing functionality was the “map table” which contains simple data for all the locations inside a specific region – loading using the following function:

LoadMapTable(int region) – which loads data from “MAPTABLE.XXX”, where XXX = region number.

Its part of functionality that loads a new region of the map – such as the Daggerfall province (region 17). First it frees all the data from the old region then loads the maptable, then applies flags which are supplied from MAPSAVE.XXX, again XXX = region number. Interestingly MAPSAVE files are contained inside “mapsave.sav” which turns out to have the same archive format as the BSA files. Why not just call it mapsave.BSA then? I have no idea, lol. Anyway its a bit more complicated then that – but you get the idea. :)

Now that the map table is in place, several missing variables can be filled out allowing me to completely finish the Cell and Region loading code, which was only partially complete before. Which means that tomorrow I can go back to working on the actual use of that data.

Here’s some useful code for you:

/*************************************************
** func_13DF8F(newRegion)   //eax
   LoadMapTable(int region) //eax
*************************************************/
push ebp
mov ebp,esp
push ebx,ecx,edx,esi,edi
sub esp,00000004
mov [ebp-0018],eax   <- Loads the "region" into a local variable
mov eax,[ebp-0018]   <- Copies the local variable back to the register
call 00133DD49       <- Calls the useful function: _LoadMapTable(region)
lea esp,[ebp-0014]
pop edi,esi,edx,ecx,ebx,ebp
ret

Things like this in the Daggerfall code just make me shake my head. :lol:

I’ve been working feverishly on the Beta lately and have decided to take a step back and look and what’s been accomplished. While it’s hard to say, linearly, how far along things are – going through the code just doesn’t work that way – I can show a “map” of the code as seen so far.

For many of the files (21 of them) I know the actual name used, which I will provide. For other files, I don’t know the real name but do know about the functionality provided (to varying levels).

Numerical values shown are the first 3 numbers of the in-memory hex addresses – for example 0x16F4C7 “LoadCell()” would be shown as 16F. This is a high level map, so only file names are shown.

MAP:
Start Address | Code File or Description
————————————————————–
130 | main game loop (3D view) – file name unknown (yet)
132 | archive.c
134 | engsupp.c
13D | maploads.c
144 | career.c
147 | automap.c
155 | fs2df.c
15A | intro.c
15D | init.c
15D | text.c
166 | parse.c
16B | quests.c
16E | LoadCell and support – file name unknown (yet)
178 | char.c
18C | disk.c
192 | weapon.c
19B | loadsave.c
19C | support.c
1A0 | interface.c
1A5 | objlib.c
1A6 | maplogic.c
——————————–
Files names below this point are unknown but functionality is.
——————————–
1B0 | c-lib (malloc, free, fopen, fread, fclose, memset, rand,etc.)
1E0 | 3D rendering functionality
240 | mouse and font functionality

This includes sound support but I’ve re-written most of that already to support OpenAL.

I should also mention that most c-lib functions are not being written based off of the Daggerfall code – instead the modern equivalents are used (DOS based file system support, for example, isn’t very useful on Windows).

The major exception to this is rand() – since it must act exactly like vanilla Daggerfall, otherwise dungeon textures will be wrong. Yes, you read that right – here’s why:

word textureTableSrc[5]=	//0x28617C
{ 119, 120, 122, 123, 124 };
word textureTableCur[5];	//0x286186

void InitializeTextureTable()
{
	int r0 = rand();
	srand( curLocationRec->locationID>>16 );
	byte r = randByte();
	int r2 = _texRandTable[ r ];

	memcpy(textureTableCur, textureTableSrc, 10);
	for (int i=0; i<5; i++)
	{
		byte n = random_range(0, 4);
		if ( n == 2 )
		{
			n += 2;
		}
		n += textureTableSrc[r2];
		textureTableCur[i] = n;
	}

	srand(r0);
}

 

textureTableCur[] now contains texture file indices which are used to texture the various dungeon sections (modulo 5). As you can tell, if the random number generator is even slightly different, the dungeons will be textured completely differently. As it is, it works because the random number seed is based off the location ID – but note that the generator is re-seeded at the end so things like loot, random encounters, etc. are not affected by the locationID as well. :)

I originally promised an update this week. Unfortunately there isn’t a lot to tell, I’m still hammering away at the goal of completing the gameplay functionality for the Beta. I should make some good progress next week with the closed testing starting shortly afterwards – I already have enough testers lined up for this (see the previous blog post). My plan is to have a gameplay video up next week, showing the “Pure Mode” in action – showing things like proper NPC behavior, quests and so forth. :)

This is a short post this time but I’m planning on having a blog post per week from here on out (or more when warranted), though some may be short and informal like this one.

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.