It has been a long time since my last update but my life has been undergoing a series of changes – for the better – that have occupied my time. I have started to settle in and have resumed working on the DaggerXL Beta. In this post I will talk a little about the Daggerfall Renderer, starting with some simple basics.

Below is the Object structure – the structure itself (i.e. size of variables and such) is fully known though a few members still do not have appropriate names yet, u1, u2, u3. As you can see the world space position is stored in** (xPosition, yPosition, zPosition)** in inches and the rotation angles are stored in **angles[3]**. Positions are stored in absolute world space – this position is not relative but the actual position on the world map. The angles range from **0 – 2047** – which maps to 360 degrees. In other words **512 = 90 degrees**, **2048 = 360 degrees**.

struct Object { //0x00 byte type; //0x01 word angles[3]; //0x07 int xPosition; //0x0B int yPosition; //0x0F int zPosition; //0x13 word index2; //0x15 word flags; //0x17 word dataSize; //0x19 word index; //0x1B word arrayIndex; //0x1D word model; //0x1F int ID; //0x23 byte u1; word u2; byte u3; //0x27 int curLocID; //0x2B dword time; //0x2F Object *target2; //0x33 Object *target; //0x37 Object *siblingNext; //0x3B Object *siblingPrev; //0x3F Object *child; //0x43 Object *parent; };

To generate a rotation transform for an object or for the camera Daggerfall uses the following matrix:

Given angles: x=xAngle, y=yAngle, z=zAngle – the Daggerfall rotation matrix =

[ cos(z)*cos(y) + sin(x)*sin(z)*sin(y) -sin(z)*cos(x) sin(z)*sin(x)*cos(y) ] [-sin(x)*sin(y)*cos(z) + sin(z)*cos(y) cos(x)*cos(z) -cos(z)*sin(x)*cos(y) ] [ sin(y)*cos(x) sin(x) cos(x)*cos(y) ]

The 3×3 rotation matrix is stored in **1.3.28 fixed point**.

Here is the function used to generate the 3×3 rotation matrix from the angles:

void Build3x3Matrix(int xAngle, int yAngle, int zAngle, int *matrix) { int x = xAngle&0x07ff; int y = yAngle&0x07ff; int z = zAngle&0x07ff; const int64 half = 134217728LL; int64 c = (int64)sinTable[z] * (int64)sinTable[y] + half; camAngle0 = (int)( c >> 28LL ); c = (int64)sinTable[y] * (int64)sinTable[512+z] + half; camAngle1 = (int)( c >> 28LL ); c = (int64)sinTable[x] * (int64)sinTable[512+y] + half; camAngle2 = (int)( c >> 28LL ); c = (int64)sinTable[512+z] * (int64)sinTable[512+y] + (int64)sinTable[x] * (int64)camAngle0 + half; matrix[0] = (int)( c >> 28LL ); c = -(int64)sinTable[z] * (int64)sinTable[x+512] + half; matrix[1] = (int)( c >> 28LL ); c = (int64)sinTable[z] * (int64)camAngle2 + half; matrix[2] = (int)( c >> 28LL ) - camAngle1; c = -(int64)sinTable[x] * (int64)camAngle1 + (int64)sinTable[z] * (int64)sinTable[y+512] + half; matrix[3] = (int)( c >> 28LL ); c = (int64)sinTable[x+512] * (int64)sinTable[z+512] + half; matrix[4] = (int)( c >> 28LL ); c = -(int64)sinTable[z+512] * camAngle2; matrix[5] = (int)( c >> 28LL ) - camAngle0; c = (int64)sinTable[y] * (int64)sinTable[x+512] + half; matrix[6] = (int)( c >> 28LL ); matrix[7] = sinTable[x]; c = (int64)sinTable[x+512] * (int64)sinTable[y+512] + half; matrix[8] = (int)( c >> 28LL ); }

A few things to note – the fixed point format is **1.3.28** so 64 bit math is required to avoid overflows, fortunately x86 assembly makes this fairly easy to do quickly. Also note that Daggerfall has a sine table that stores the sin values for all angles ranging from **0 to 2047** (remember that this maps to 0 to 360 degrees). As you can see the cosine values are computed as **sinTable[angle+512]** – since sine and cosine are out of phase by 90 degrees, we can compute cosine as **cos(angle) = sin(angle+90degrees)** which Daggerfall does to limit the size of the table.

The camera uses the same position and angles to generate the view and projection matrices. However the way positions are transformed into screenspace are a little different from most modern engines. The projection matrix is build directly from the view matrix by scaling by the x relative and y relative screen aspect ratios, built as follows:

void BuildAspectScaledMatrix(int *rotMatrix, int *outScaledMatrix) { int64 c = (int64)rotMatrix[0] * (int64)screenAspectX; outScaledMatrix[0] = (int)( c >> 14LL ); c = (int64)rotMatrix[1] * (int64)screenAspectX; outScaledMatrix[1] = (int)( c >> 14LL ); c = (int64)rotMatrix[2] * (int64)screenAspectX; outScaledMatrix[2] = (int)( c >> 14LL ); c = (int64)rotMatrix[3] * (int64)screenAspectY; outScaledMatrix[3] = (int)( c >> 14LL ); c = (int64)rotMatrix[4] * (int64)screenAspectY; outScaledMatrix[4] = (int)( c >> 14LL ); c = (int64)rotMatrix[5] * (int64)screenAspectY; outScaledMatrix[5] = (int)( c >> 14LL ); outScaledMatrix[6] = rotMatrix[6]; outScaledMatrix[7] = rotMatrix[7]; outScaledMatrix[8] = rotMatrix[8]; }

Note that the **screenAspectX** and **screenAspectY** are stored in **1.17.14** fixed point but the resulting projection matrix is still stored in **1.3.28** fixed point.

Positions in view space are stored as 1.23.8 fixed point so a position can be transformed as follows:

Given (x,y,z) in absolute world space:

(Note **s_xPosition, s_yPosition and s_zPosition** are the camera positions extracted from the camera objects and used by the renderer).

//Convert to camera relative coordinates, stored in 1.23.8 fixed point), still in inches. int viewX = (x - s_xPosition)*256; int viewY = (y - s_yPosition)*256; int viewZ = (z - s_zPosition)*256; TransformPoint(&viewX, &viewY, &viewZ, projMatrix); void TransformPoint(int *x, int *y, int *z, int *matrix) { int64 xp = (int64)(*x)*16LL; int64 yp = (int64)(*y)*16LL; int64 zp = (int64)(*z)*16LL; *x = (int)( ( xp*(int64)matrix[0] + yp*(int64)matrix[1] + zp*(int64)matrix[2] ) >> 32LL ); *y = (int)( ( xp*(int64)matrix[3] + yp*(int64)matrix[4] + zp*(int64)matrix[5] ) >> 32LL ); *z = (int)( ( xp*(int64)matrix[6] + yp*(int64)matrix[7] + zp*(int64)matrix[8] ) >> 32LL ); }

Finally the following functions are used to cull the bounds, stored as spheres – center in relative camera coordinates and radius in inches.

enum ClipPlanes_e { PLANE_NEGX = 1, PLANE_POSX = 2, PLANE_POSY = 4, PLANE_NEGY = 8, PLANE_NEAR = 16, PLANE_FAR = 32, PLANES_ALL = PLANE_NEGX | PLANE_POSX | PLANE_POSY | PLANE_NEGY | PLANE_NEAR | PLANE_FAR }; static int nearPlane=2560; //[1E9CC0] :the near plane is 10 inches. static int farPlane =393216; //[1E9CC4] :the far plane is 128 feet //(at normal maximum settings). int CullPlaneDistX(int xScaled, int yScaled, int radius) { int r = (zRadius*xScaled)>>16 + (xRadius*yScaled)>>16; if ( r < 0 ) { r = -r; } return r+radius; } int CullPlaneDistY(int xScaled, int yScaled, int radius) { int r = (zRadius*xScaled)>>16 + (yRadius*yScaled)>>16; if ( r < 0 ) { r = -r; } return r+radius; }

//Is the sphere: center = (x,y,z), radius = r at least partially visible? //Returns 0 if visible else 1. //Note that the sphere position must be relative to the camera and be in 1.23.8 fixed point format. int IsSphereVisible(int x, int y, int z, int r) { //transform into projection space. TransformPoint(&x, &y, &z, projMatrix); //figure out which planes the sphere needs to be tested against. int clipFlags = PLANES_ALL; //6 planes = 111111 binary = 63 if ( z >= nearPlane ) { clipFlags ^= PLANE_NEAR; } if ( z <= farPlane ) { clipFlags ^= PLANE_FAR; } if ( x >= -z ) { clipFlags ^= PLANE_NEGX; } if ( x <= z ) { clipFlags ^= PLANE_POSX; } if ( y <= z ) { clipFlags ^= PLANE_POSY; } if ( y >= -z ) { clipFlags ^= PLANE_NEGY; } //compute plane data. xRadius = (int)( ((int64)x*(int64)x*(int64)ScreenAspectA )>>32LL ); yRadius = (int)( ((int64)y*(int64)y*(int64)ScreenAspectA_SC)>>32LL ); zRadius = z; //test against each plane based on the clipflags set (see above). if ( clipFlags ) { if ( s_clipFlags&PLANE_NEAR ) { if ( z+r <= nearPlane ) return 1; } if ( s_clipFlags&PLANE_FAR ) { if ( z-r >= farPlane ) return 1; } if ( s_clipFlags&PLANE_NEGX ) { if ( CullPlaneDistX(ScreenX_Scaled, ScreenH_Scaled, r) <= 0 ) return 1; } if ( s_clipFlags&PLANE_POSX ) { if ( CullPlaneDistX(ScreenX_Scaled, -ScreenH_Scaled, r) <= 0 ) return 1; } if ( s_clipFlags&PLANE_POSY ) { if ( CullPlaneDistY(varD34, -varD30, r) <= 0 ) return 1; } if ( s_clipFlags&PLANE_NEGY ) { if ( CullPlaneDistY(varD34, varD30, r) <= 0 ) return 1; } } return 0; }

Anyway that is enough about the renderer for this post but I will talk about the lighting system, more about culling objects and other topics in future blog posts. If you want to see the original assembly for these functions, visit the Renderer Part I – Original Functions in Assembly page. As you will see I have rearranged the code a little to make some things more clear.

I know jack shit about coding, but I am glad to see this post.

Keep up the good work, man.

Nice, but since you settled in does that mean you’ll be posting more often?

Hey you’re not dead! Glad to hear it’s not for bad reasons, seems to be pretty rare for the people I’m following.

I would describe the angles as a 0.16 fixed point unit angle – even before you mentioned fixed point further down ;). I’ve not seen the ‘1.’ prefix for sign before – I’ve always seen it assumed in the integral size and explicitly ‘unsigned fixed point’ if it’s not present (though ‘0.x’ makes that pretty obvious).

I’ve noticed that you stick to C (ish?) in your decompilations: I can see the appeal of not getting distance between it and the assembly you’re comparing it to, but I can’t help but cringe seeing all those >> 17 and * 256’s where a quick template <int MantissaBits, typename BaseInteger=int> struct fixed_point would make it all pretty :).

I’m at an annoying point with matrices where I know enough to try to use them, but not enough to be efficient at using them – I can never keep straight which side of the multiplication is “first”, and as soon as there is a projection matrix involved my head explodes. Almost certainly just because I’ve not had enough practice using them, and the mathematical matrices being layed out differently (often with different axes!) probably doesn’t help.

The code is intentionally in C in order to try and replicate the original structure and keep things simple. The game code will probably remain in C, even though the XL Engine itself is in C++.

That really capetrus the spirit of it. Thanks for posting.

Great, good news!!! Hope this year will see a Daggerfall Beta!!!

I lost hope for such things long ago, but still hope that the rebuild will be good enough to come back to Daggerfall.

I know, man. I’m still hopeful, but shit dude I’ve been waiting for this like I waited for Oblivion!

Always exciting to see an update on DaggerXL!

I think my brain exploded.

(First seconds in.) Okay!

(Thirty seconds in.) Mmmmmmm…

(5 minutes in.) Uhhhhhhh…

(After Reading it all) Holy #%&$ what a post!

I must say, this is truly an art form of its own and you are like the Star Wars Rock Band picture, just completely awesome with reverse-engineering and coding. :O

Welcome back!

Glad to see you’re back, and great blog post!

It’s pretty weird that they mapped angles from 0-2048 to 0-360, and even weirder that they used fixed point representation — were either of those practices at all “standard” when Daggerfall was made back in the 90s? Or are they just weird, Daggerfall-specific quirks?

In those days floating point co-processors could not be relied to be in the system and operating with pure integers, using fixed point, was standard – especially in the core engine code. To store angles as integers, obviously 360 discreet angles are far too few for smooth rotation and mouse control. With 2048 discreet levels then a 320×200 view gives approximately 1 angular unit percision per pixel, allowing for smooth mouse control of the camera. At higher resolutions the angular precision would need to be increased to continue feeling smooth – 640×400/640×480 will probably still feel good, as evidenced by SkyNet – but the feeling would quickly degrade as the resolution increased beyond that point.

Floating point arithmetic is a historical curiosity. No, floats still are noticeably slower & obviously dirtier, sloppier than a pure integer, combinatorics approach (the mere float/int conversion is already unbearable). Yes, there are empires built on the inefficiency of floating point arithmetic e.g., NVIDIA & such. You don’t need floats, / or * in a 3-D renderer. You don’t even need the weaklings’ parallelism. There are existence proofs.

I’m very glad you understand this so well. Even with 3D experience most of those details are unfamiliar to me. Keep up the good work!

Glad to see you’re not dead!

Love this!

Please contiune !

Life is short, and this article saved vablaule time on this Earth.

DaggerfallXL, I hope to play you one day… when you are ready. Great work so far.

This post is a test.

….

Itching for an update soon….

I have no idea what you just said, but I’m glad to hear that you’re alive and well!

So when do you estimate that quests and NPCs will work in DXL?

Is this project dead? Very glad your life has changed for the better but it would be superb to know if you’re still working on it or not.

update?

Where are you, Lucius? You’re our only hope.

Is this project dead or just freezed for a while?

Kinda sad that this was his first project yet it’s the one furthest behind. I’m guessing he’s getting burnt out on it and that’s the reason he started the others.

Been watching this for years, hoping…guess I’ll check back in another year or two.

eep! I’m wrong…DaggerXL wasn’t his first and I apologize for the mistake.

Talk to us, Lucius, i gotta have something to show the Codex soon before they loose hope!

It is very quiet at the moment.

I hope Lucius will keep up his good work.

I read a lot of interesting content here.

Probably you spend a lot of time writing,

i know how to save you a lot of work, there is an online tool that creates high quality, SEO friendly articles in minutes, just

type in google – laranitas free content source

I’ve been checking this blog every single day for over 2 years (maybe even 3). I hope to hear again from you soon Lucius, even if it’s just an “I’m still alive” post!

Hi. Is the source code of the XL engine open source? I would lvoe to see how the engine works If you want you could send it to me (panda1996@gmx.de). THX Eric

So acording to the faq the code is not open until the project reach the beta state. Dont be upset but since the project is running now for about 4 years I am a little bit worried that it will take another long amount of time since the project reaches the beta. I know that it is very hard to work at a such big project without a team as a single person. I hope you share your great work with us.

Wow. 8 months past by the last Post already. I just want a new little post to know, this game’s still alive.

I’m kinda worried about this project. I’ve been following it for years since it was at a very early stage with Dark Forces. Time flew by and i was mesmerized with current progress with that game. I was even more excited with Blood, Daggerfall and Outlaws future support, since all of these games were part of my youth. Taking into account: personal life of Lucius, also being a programmer and realising how this is gonna be hard work and the last 8 months of silence from him, let me ask: Is this project still alive? Just to be sure.

I really hope that Lucius greets us with good news, so we can at least know that things are going forward at their own pace. Cheers.

I really hope lucius comes back to this soon. Or if he can’t finish the project he hands it off to someone who can. This is something that needs to see the light of day.

Great to see a relatively recent update on this site. I love classic Dark Forces and Dark XL is the best idea ever.

Daggerxl confirmed dead?

Can we get an update please Lucius!!!!

Dead again it seems..

Some day. Some day luciusDXL will return.

this whole project is a hoax

It’s been one year since the last update. Is the project still alive in any way?

Hi Lucius. Do you have any updates?

I wish you luck and patience.

So, where is he?

Hey there — without wanting to tweak your tail, do you still intend to complete this Daggerfall project? I’ve been holding off on a playthrough for years now hoping to use your engine, but if life has taken you in other directions, I’ll give up. Thanks! And good luck in whatever your current projects are.

It would be funny if he had just completed all of this over the last year.

Well… It’s been a while since anything has been posted here…

Still lurking, still hoping….

ARE U DED :(?

Cancelled ?

It’s been over a year since anything has been said, seems like it’s dead.

I’ve been following this for years and years and it’s extremely depressing to visit this site hoping for an update just to see the same rendering post from 2014.

2 options: Lucius is dead or the project is canceled :c

Hey, any news/updates on this?

I can throw a boat load of money your way for motivation if that will help as money dont mean a damn thing to me, hell maybe even open a donate button people can send you “precious $” as then you can work on it full time as alot of people want to see this complete as do i since Daggerfall is the best Elder(est) Scroll game (a little joke in there ;). Hell even make it a Kickstarter for cash round up, maybe you can also get help/staff on the project.

Food for thought.

Hey, any news/updates on this?

I can throw a boat load of money your way for motivation if that will help as money dont mean a damn thing to me, hell even make it a Kickstarter for cash round up, maybe you can also get help/staff on the project, as then you can work on it full time as alot of people want to see this complete as do i since Daggerfall is the best Elder(est) Scroll game (a little joke in there ;).

Food for thought.

is this still ongoing?

STILL WAITING FOR PART 2 BRO

lol what happened with this¿¿

I already moved onto Daggerfall Workshops attempts at doing something about remaking Daggerfall, so far, they seem to be most transparent with the process.

Please, come back!

At the times of press-x-to-win games with The-Chosen-To-Genocide-Billions-Of-Weaklings (games must be friendly!)-One plot style and 9000 achievements for being idiot it is too sad to see almost 0 games with real roleplaying and freedom. And the project that could remake the most powerful RPG ever… is dead? Noooooooooooooo!

Please, reborn Daggerfall!