Translate
EnglishFrenchGermanItalianPortugueseRussianSpanish

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.

Recent Comments

As many of you may have guessed, I have been working long hours the last few weeks – delaying my plans for posting an update and temporarily slowing progress. Fortunately the pace has died down, so progress can continue more smoothly again. As you can guess, however, this won’t be the last time there are hiccups in the road. Unfortunately I’m way past the point I originally thought I was going to be done – a good reason to take any estimates I give with a giant grain of salt (to be fair though, this is true for most developers and most large scale projects – especially for hobby projects).

 

Progress

As you might imagine, most of my time lately on getting the game code ready. Most of this has been spent refining my tools, adjusting the resulting code – which then goes back to refining the tools again. The good news is that the game code is between 80 – 90% complete. There is a fair bit of work left but things have been progressing well. I expect to have the games starting to become playable within the next few weeks – “real life” permitting. There is a sizable amount of work to go from playable to the whole package being ready for release, including bug fixes – so this is not a release estimate, merely an indication regarding the state of the release and progress.

 

Process

As I talked about before, there are several stages that allow me to go from the original executable and allow me to start piecing the code together and moving towards a fully decompiled result that runs natively on the XL Engine. I thought it would be interesting, in this post, to give a “behind the scenes” look at the intermediate format, between the disassembled code and decompiled code and give a little insight on that process.

This skips the processing steps that discovers the 32 bit code (for now 16 bit code is ignored), disassembles the code, figures out how the functions are arranged, removes dead code, converts the assembly into an internal format for processing and other required work. For now we are going to focus on the processing of a single function. Function processing occurs in several global passes and each pass has several steps, such as figuring out function inputs and outputs but for now we focus on only a few steps.

Listed here is a simple function, at this point in the process the purpose of the function isn’t known.

The first step is to disassemble the function:

000B094C 55               push	ebp
000B094D 89E5             mov	ebp,esp
000B094F 53               push	ebx
000B0950 51               push	ecx
000B0951 52               push	edx
000B0952 56               push	esi
000B0953 57               push	edi
000B0954 81EC00000000     sub	esp,dword 00000000h
000B095A B8603F0300       mov	eax,00033F60h
000B095F E83CF60200       call	0000DFFA0h
000B0964 A1783F0300       mov	eax,[000033F78h]
000B0969 2B05743F0300     sub	eax,[00033F74h]
000B096F C1E002           shl	eax,byte 02h
000B0972 A3983F0300       mov	[000033F98h],eax
000B0977 8B157C3F0300     mov	edx,[00033F7Ch]
000B097D C1E202           shl	edx,byte 02h
000B0980 A1983F0300       mov	eax,[000033F98h]
000B0985 01C2             add	edx,eax
000B0987 8915903F0300     mov	[00033F90h],edx
000B098D A1783F0300       mov	eax,[000033F78h]
000B0992 C1E002           shl	eax,byte 02h
000B0995 A3943F0300       mov	[000033F94h],eax
000B099A A1983F0300       mov	eax,[000033F98h]
000B099F A39C3F0300       mov	[000033F9Ch],eax
000B09A4 8D65EC           lea	esp,[ebp-14h]
000B09A7 5F               pop	edi
000B09A8 5E               pop	esi
000B09A9 5A               pop	edx
000B09AA 59               pop	ecx
000B09AB 5B               pop	ebx
000B09AC 5D               pop	ebp
000B09AD C3               ret

Next the function is converted to an intermediate format. At this point variables are assigned as inputs (%i#), locals (%l#) or globals (%g#). At this stage the tool is only looking at a single function at a time, later variables will be matched up and merged as appropriate and renamed.

void func_000b094c()
{
	/* variables
	Locals (7)
	EBX, ECX, EDX, ESI, EDI, EAX, [ebp-20]
	Globals (7)
	[33f78], [33f74], [33f98], [33f7c], [33f90], [33f94], [33f9c]
	*/
	%l5 = 0x00033f60;
	func_000dffa0();
	%l5 = %g0;
	%l5 -= %g1;
	%l5 <<= 2;
	%g2 = %l5;
	%l2 = %g3;
	%l2 <<= 2;
	%l5 = %g2;
	%l2 += %l5;
	%g4 = %l2;
	%l5 = %g0;
	%l5 <<= 2;
	%g5 = %l5;
	%l5 = %g2;
	%g6 = %l5;
	return;
}

Then the code is simplified by merging statements into blocks, as discussed in a previous blog post. Block boundaries are defined by certain instructions such as calling a function or writing to global memory. At this point the tool already knows what the function inputs and outputs are and can modify the intermediate format to reflect that.

void func_000b094c()
{
	func_000dffa0( 0x00033f60 );
        %g2 = (%g0 - %g1) << 2;
        %g4 = (%g3<<2) + %g2;
        %g5 = (%g0<<2);
	%g6 = %g2;
}

Additional passes are then required to define all of the global variables and types, to try to match strings to variables, functions and/or files and then to finally assign better names to everything. The end result (when everything works) is compiling code which then must be debugged and cleaned up. Once that is done, parts are modified to use the XL Engine services or features (for example using the XL Engine for rendering, input and sound).

15 Responses to “Delays and Progress”

  • Lyle Warren Castro:

    Im really excited for this, keep up the good work man!

  • 01010101:

    Thanks for the update. I hope this is released by June.

  • Axl:

    Thanks for the update… take your time… quality above all.

  • TGE:

    Thanks for the update. Remember history taught us that rushed releases ALWAYS suck. Good luck.

  • POLE7645:

    I was honestly a bit worried since there hasn’t been any updates on Github for more than three months.

    Thanks for the update and keep up the good work.

    • luciusDXL:

      Unfortunately most of the work lately hasn’t been related to the engine. Though I do have a bunch of engine changes that I really should push.

  • Zeneric:

    Looking forward to finally playing Dark Forces (and the other games) on modern PC properly. Take all the time you need, no rush. Thanks for doing the amazing work.

  • Luke:

    Hi,

    Want to thank you for your efforts. I know a good Endless Virtual Project when I see one :)
    I have paused playing Daggerfall to await your release; I am guessing I should restart (as in: old saved games, will they work?).

    Oh ah…and don’t you just really dislike how this silly thing named Real Life keeps interfering with our serious/fun Gametime?

    Gl!
    Luke

    * I say, there is no
    * darkness but ignorance.

    * Twelfth-night; or, What You Will (William Shakespeare)

  • Solie:

    I’ve stopped my playthrough as soon as I’ve heard from your work. It’s really exciting to be able to play this game with brand new graphics !

    So many thanks for the work you’ve already done, keep up the good work !

  • SubCog:

    Awesome work as always. Time for another blog update though, me thinks.

  • toxicshadow:

    @luciusDXL

    Any chance of a quick update? Even if its just a one liner :)

  • Ocean D. Morgan:

    Delays are just Absolute Progress minus Progress. Divide and conquer. The more you share, the faster this’ll be done. I know you want XL to be you’re own, but XL is bigger than you, therefore you will never complete it. It’s the translation of all that was awesome about 90’s PC gaming packed into one.
    Don’t treat this like a programming exercise, treat it like a story with a beginning middle and end. The best stories are written by one author and proof read by 10 more.

  • Juan Valdez:

    Did you create the software to do this yourself?

  • DarkVader:

    I’ll be dead before this thing gets anywhere. You really should open up the project and let some new blood get this back on track.

  • Andree:

    Can’t wait to play Daggerfall on the XL engine. Any guess on when the game will be fully playable? Im assuming a year or so but let us know please.

Leave a Reply for Axl

The XL Engine is and will remain free, donations are purely optional but greatly appreciated.