XLC Documentation

From XL Engine Wiki
Jump to: navigation, search

Note that this system is still a work in progress so not all of the features discussed here are available yet - especially in terms of modding. However it is important to discuss the goals as well as the current state. Features not yet available will be clearly marked and are subject to change.

Introduction

XLC stands for XL Engine “C”, which uses JIT compiled “C99″ as the basis for the scripting system. In order to improve the scripting experience, the environment is sandboxed (only engine provided functions and services are available) and the API is written in a way that avoids or hides the use of pointers and avoids user memory management. The language is not hobbled, however, and advanced users can use the full power of C. The compiler is built into the engine and compiles code, as needed, directly into memory. This means that the engine can call script functions with very little overhead, memory and data can be shared between scripts and the engine and scripts can call into the engine with the same cost as calling any C or C++ function using a pointer.

The only tool required to write XLC scripts is a text editor. If the engine is running and you edit a script, the engine will automatically hot-reload and recompile the script – which takes a fraction of a second – and you can see the changes immediately. The compilation is fast enough that scripts can easily be included as part of the data – so levels and game areas can have their own scripts in mods. Scripts can also include other scripts in order to import their functionality, variables and structures – allowing scripts to be broken up into files and allowing people to provide encapsulated functionality that anyone can use in their scripts for any game (unless game specific functions are used).

Obviously extremely fast compilation times come at a cost, the generated code isn’t as fast as Visual Studio C++ or GCC optimized code. However performance is much better than debug code, interpreted languages and even a decent improvement over other JIT compiled scripting languages in most cases. In addition the overhead of passing the “script barrier” is much better then the alternatives – you would need to call many tens of thousands of script functions per frame before it starts to become a problem.

For more information on the reasoning behind XLC and why other options were not chosen, please read the original blog post: Introducing XLC - XL Engine Scripting System.

Script Modules

Scripts are grouped into "modules" - these are "xlc" files that contain several scripts that serve a shared purpose. For example, a user made Dark Forces level may have a module that contains scripts that are executed during play. These individual scripts may be initiated by the user entering or leaving a sector, some in-game event such as the player killing a certain enemy and various other triggers. Or it may be the UI module that runs the XL Engine UI that you see when selecting games and changing settings.

A script provided by a module is really just a special function - it acts the same way as any other function except that it is callable by the engine and game code. Script introspection will reveal all available scripts that the module provides, allowing you to load up XLC files in tools which will be able to provide a list of available scripts and the arguments that they take.

Take a look at this toy example, you will see that scripts are identified by the public keyword in front of a function. This module has three scripts available - perfTest which takes no arguments, simpleInc which takes 4 arguments and simple_main which takes 3. As you can see, internally scripts are merely functions - so one script function can call into any others that exist in the same module. Within the module these functions are exactly the same as the others.

Oftentimes you will write common code that you wish to use in several modules or provide to others so they can use it in their scripts. In XLC this is accomplished by simply "including" those files which makes all of their script global variables, structures, constants and functions available to the module. In the example above test2.xlc is included which provides the sqr() function and someVar variable. Note, however, that included variables are in the scope of the current module only - meaning that multiple modules that include the same file each have their own copy of those variables. While it is possible to share data between modules that is handled through a different mechanism that will be discussed later.

Language

If you don't know C and just want to be able to use XLC you can skip this section.

The language is C with some extensions to allow it to work as a scripting language - such as the public keyword. It supports almost all C99 features with the exception of complex types (which were made optional in C11 anyway due to their rare use). Some of the syntax may look slightly different but usually the original C syntax will still work.

For example creating a new structure type in C is usually done as follows:

typedef struct StructName
{
    ...
} TypedefName;

But in the example you will see

Struct(TypedefName)
{
    ...
};

instead, which is really just typedef struct TypedefName {...} TypedefName;. This was done to simplify the common task of defining structures and to avoid struct Type name in code, since all structures made in this way are typedefs. However, like I mentioned above, the regular C syntax still works.

Getting Started

Here I will go over some of the basic knowledge needed to write simple scripts. ...Under Construction...


Script Execution and Threading

Each Script within a Module is executed from a single thread but different Modules may be executed on different threads, allowing the script system to take advantage of multi-core systems. Within each module, scripts are executed as coroutines which essentially allows the script to return control to the system and then later start up from the same spot again while preserving the values of all local and script global variables.

Yielding and Timing

Scripts may yield execution back to the system and be resumed after a specific amount of time. This can serve multiple purposes - the first is to allow other scripts to run and to avoid taking up too much CPU time and the second is to allow for easy timing since you can wait for a specific amount of time (or until the next frame). This may be useful during animations, displaying text for a specific amount of time and other situations where you want code to be executed with certain timing. The first example ("playerlight"), below, shows an example of a script that keeps processing every frame - you will note that the xlYield(0) will cause the while loop to run exactly once per frame. The second example ("brokenDoor") shows how to use yields to control animation timing - in this case to animate a sector. Also note that the second example only plays the animation 5 times and then exits whereas the first will keep going until the module is shutdown.

Suspending Scripts

Scripts may also suspend execution indefinitely. In this scenario the script returns control to the system and will pause until the script is executed again. This may be used, for example, to allow switches to have multiple states as shown in the third example ("multistateSwitch"). Suspended scripts consume almost no extra processing time so this is, in general, preferred over loops that check state variables and similar constructs.

Signals and Synchronization

There are times when scripts need to synchronize with each other or even system events. The best way to handle this is to wait on the signal, so the script does not consume any CPU time while waiting for the signal to be fired. Note, however, that a script waiting on a signal will resume the frame after the signal is fired - this is done to avoid "cascading" effects that can effect performance (imagine signals activate scripts that fire more signals that activate more functions that also fire signals...). This is great if you don't know how long something is going to take or when something is going to happen. See the "generatorSwitch" and "giantGear" scripts in the example below. Note that these two scripts work together by synchronizing using signals.

Coroutine execution functions

  • xlYield(dt) : this will return control to the system, the script will resume from the next instruction after dt seconds has past. A value of 0 will cause the function to resume on the following frame.
  • xlSuspend() : this will suspend the script execution, it will resume from the next instruction if/when the script is activated again.
  • xlSignal(name) : this will trigger a signal, which will cause all script functions waiting on the same signal to resume executing on the next frame.
  • xlWaitOnSignal(name) : this will cause the script to suspend execution until another script or the engine triggers the same signal.

The coroutine will exit once a return is hit or the end of the function is reached. Beware that scripts that execute for too long will be forcibly shut down. This means that you must return control to the system by calling xlYield(), xlSuspend(), xlWait functions or returning from the script function in a reasonable amount of time.

Coroutine Examples

Note, the "API" shown below is not final. They are merely here to represent some of the features you may have access to.

//This shows how to run a script that will last until the module is shutdown. xlYield(0) at the end of the loop causes 
//the loop to iterate exactly once per update frame.
public void playerlight(int r, int g, int b)
{
    //light values
    u32 color    = PACK_RGBA(r, g, b, 255);
    Vec3 pos     = Vec3(0, 0, 0);
    float radius = 10.0;
    float brightness = 1.0;

    //object identifiers
    int playerID = xlGetObjectID("player");
    int lightID  = xlCreateLight("playerlight", color, brightness, pos, radius);

    //loop: this function will keep running until the module is shutdown.
    while (1)
    {
        //obviously this isn't the most efficient way to do this, better to attach the light instead but...
        //this is for demonstration purposes
        Vec3 playerPos   = xlGetObjectPos(playerID);
        Vec3 newLightPos = { playerPos.x, playerPos.y, playerPos.z + 2 };
        xlSetObjectPos(lightID, newLightPos);

        //a 0 delay means that the function returns control to the system but will be executed again as soon as possible -
        //which will be the next frame. It will continue from the next instruction, in this case the start of the loop.
        xlYield(0);
    };
}

//This shows using the xlYield() function for timing. Each movement is started and then yields execution to the system, 
//continuing the animation after the yield time has elasped. Each loop of the animation takes 5.75 seconds and the 
//door animates 5 times before the script ends. So the script takes a total of 28.75 seconds to run.

//For the final API I will also have functions that will not only move sectors but also have a built-in yield until
//the animation is complete. However this still serves as an example of timing since the yields are longer then
//the animation duration. They could also be shorter, maybe you want to apply some additional animation at the
//same time a sector is moving up or down.
public void brokenDoor(int sectorID)
{
    //animate a broken door - but go through the animation 5 times.
    for (int i=0; i<5; i++)
    {
        //start moving it down 10 units - it will move for 2 seconds.
        xlSector_MoveCeiling(sectorID, -10, 2);
        //then yield for 3 seconds, which means the script will resume 1 second after the ceiling stops moving.
        xlYield(3);
    
        //start moving it up 5 units - it will move 1 second.
        xlSector_MoveCeiling(sectorID, 5, 1);
        //then yield for 1.5 seconds, which means the script will resume 0.5 seconds after the ceiling stops moving.
        xlYield(1.5f);

        //finally start moving it up to the top again - it will move for 1 second again.
        xlSector_MoveCeiling(sectorID, 5, 1);
        //then yield for 1.25 seconds, which means the script will resume 0.25 seconds after the ceiling stops moving.
        xlYield(1.25f);
    };
}

//This shows how to use xlSuspend() as a way of handling multi-state switches.
public void multistateSwitch(int switchID)
{
    //this switch is never disabled.
    while (1)
    {
        //the first time the switch is triggered, the message "the first state" will be printed.
        //then the script will to return control to the system and pause until it is hit again.
        xlDebugMessage("the first state");
        xlSuspend();

        //the second time the switch is triggered, the message "the second state" will be printed.
        //then the script will to return control to the system and pause until it is hit again.
        xlDebugMessage("the second state");
        xlSuspend();

        //the third time the switch is triggered, the message "the third state" will be printed.
        //then the script will to return control to the system and pause until it is hit again.
        xlDebugMessage("the third state");
        xlSuspend();

        //next time the switch is triggered, it will go back to the first state, then the second
        //and so on.
    };
}

//These scripts show how to use signals to synchronize between scripts. The "giantGear" scripts will
//wait on the "Generator Activated" signal. The switch will fire off that signal once it is activated
//causing the giantGear script to be resumed - which will run until the module is shutdown.
public void generatorSwitch(int switchID)
{
    //fire off the signal "Generator Activated" - any scripts waiting on that signal will be resumed on the next frame.
    xlSignal("Generator Activated");

    //disable the switch so it can't be pressed again.
    xlDisableSwitch( switchID );
}

//Note that xlYield(0) still must be called to return control to the system, the loop will be executed exactly once
//per update frame.
//It is probably obvious that something must call this script before the switch can be hit. There is a
//mechanism to execute certain scripts, with proper arguments, when a module is started (see [TO-DO]).
public void giantGear(int sectorID)
{
    //wait for the signal "Generator Activated" to be fired, this will cause the script to be suspended until the signal is fired.
    xlWaitOnSignal("Generator Activated");

    //once the power is on, rotate!
    while (1)
    {
        //rotate by 5 degrees every frame.
        xlSector_Rotate(sectorID, 5);
        //yield control to the system until the script is resumed next frame.
        xlYield(0);
    };
}

WIP - more information to come soon...