Core MSIL Assembler

This post contains instructions for downloading/installing the current prototype of the Core MSIL Assembler, and will be updated as new versions are published; explanations of what the assembler is and what it does will appear in later posts.

Prereqs

The assembler is built on version 4 of the .NET Framework. This is automatically installed by Windows Update on Windows 7 & Vista, and is an optional update for Windows XP; you can also download it here.

Download & Install

You can download the assembler here. There's currently no installation step, just unzip and run the CoreAssembler.exe app where it is by dragging a xxx.core file onto it. There's also a CompileAll.bat file that you can run to automatically compile all the example code.

AxumMUD

Just a quick note to say that the code for AxumMUD is online at CodePlex. The network layer is basically complete, though I'll likely go back and refactor when I've got some time, but I'm currently working on the Character/Location design and event communication; hopefully I'll get that checked in later in the week.

New keyboard shortcuts

The fn key on my new laptop has shifted all the ctrl/alt/et cetera keys one to the right, so I keep hitting the wrong ones and discovering new keyboard shortcuts. So I'll note them here as I discover them:

In Visual Studio 2008
Shift+Alt+Left/Right Arrow: Same as Ctrl+Shift+Left/Right Arrow (selects whole tokens at a time), except it selects words within the token, delimited by capitals. E.g. if the carat were at the end of the token "BuildNewConnection", pressing Shift+Alt+Left Arrow would select "Connection".

In Windows 7
Win + Left Arrow: Aero Snaps the current window to the left of the screen, then rotates through Aero Snap Right, Restore, Aero Snap Left et cetera.
Win + Right Arrow: Aero Snaps the current window to the right of the screen, then rotates through Aero Snap Left, Restore, Aero Snap Right et cetera.
Win + Up Arrow: Maximises the current window.
Win + Down Arrow: Minimises the current window.
Shift + Win + Up Arrow: Stretches the current window to the top and bottom of the screen.
Win + Home: Aero Shake (minimises all windows except the current window).

Axum!

I've been on a bit of a hiatus, both from this blog and from doing any major work on my Codeplex projects, since around April; guess the day job's been taking up all my valuable coding time :-D.

But a few days ago, I downloaded the latest version of Axum (now v0.2), Microsoft's new .NET language/incubation project for introducing actor-based, highly-scalable and massively concurrent code to the .NET framework, and thought I'd post some of my experiences. I first heard about this back in November 2008 at TechEd EMEA, when it was called Maestro, and I was really excited - the work I'd been doing on Lite was in the same vein, trying to make distributed, concurrent code both trivial, implicit and powerful for programmers to use, whilst retaining all the good stuff that you get with .NET. Since the first CTP came out, I've been converting a few of my projects and writing some sandbox code to find out what works, and what doesn't, in the Axum programming model.

I don't want to get to much into the design philosophy, as it's fully explored elsewhere on the web, particularly on the Axum blog (here); instead, I'll be uploading code snippets that do interesting things or exploit the features of the language, and useful code patterns for developing in Axum. So yeah, expect some Axum-related goodness in the next few posts. In the meantime, I'd recommend downloading the preview, trying out some experiments of your own and giving the dev team feedback; as an incubation project, they need all the community support they can get.

Lazy Evaluation Gotcha!

Note: I'm assuming a vague knowledge of LINQ, and I'd recommend reading up on Lazy Evaluation on other blogs as I only skim over the concept here.

Code-writing for the Conquest project is currently paused while I do some research around the subject area (RTS games), particularly around the unit AI, which is my immediate bugbear; in the meantime, I've been focusing my efforts on Setun, a project that I started early last year. I'll go into more detail on exactly what I'm aiming to achieve with the project in a later post, but for now I can summarise it as a Balanced Ternary virtual machine simulated at the logical electronics level; that is, I'm simulating And and Or gates, Decoders and other similar-sized components, but not going all the way down to transistors and shifting voltages. This is lower-level than a typical VM, but not a strict physical simulation.

I've been increasingly using LINQ in this project; each large-scale component (like registers and the ALU) is typically joined to the next by a bus, or a collection of electronic lines (i.e. wires) and I'm simulating this with an IEnumerable of ILine. Now, when you're handling lots of IEnumerable objects, concatenating them, picking out individual items, taking subsets and so-on, then LINQ is perfect; manipulation of sets is exactly what it's designed for.

However, one of the features of LINQ that often catches people out is Lazy Evaluation, which, if you're not familiar with it, simply means that any LINQ query you create (e.g. pick the fifth object out a list of strings) is not executed (or evaluated) until the results of the query are actually needed. In most cases, this is really useful; queries that are declared but never used are never evaluated and optimisations are performed once the full query is declared and applied, which is better than optimising piecemeal as the query is constructed.

But...

If the timing of the query is important, Lazy Evaluation can catch you out. If you declare a query to pick the first three items of a List, clear the list then try and display the items that you just queried for, you'll find that your query returns nothing. This is because the query now points to an empty list, as it was evaluated after the list was cleared.

Now, I've pretty heavily summarised Lazy Evaluation, because there's plenty of blogs that explain it in enough detail, so instead I'll focus on the behaviour that caught me out; that is, multiple evaluation.

I've been using the Select method to create sets of objects from other sets of objects - for example, I could take an IEnumerable of ILines, and wrap them all in Not gates, like this:

var lines = new List<ILine>();

//Insert some ILines here!

var invertedLines = lines.Select(line => new NotGate(line));

Which gives me a set of NotGates, one for each input line - this is all well and good. But, I tried the same trick with registers; unlike Not gates, which are pretty simple (inverting whatever their input is) registers store a value, only changing on clock ticks. When I tried to display the contents of the set of registers, they always came out as 0 - the initial value. No matter what input I gave, they wouldn't change.

A little investigation later, and I realised the problem: because Lazy Evaluation occurs when you use the result of a query, it happens every time you use it. Each time I iterated over the IEnumerable of registers to output the contents, the LINQ query was creating a new set of register objects, which of course contained the initial value.

So remember, if you're using Select to create sets of objects (for which it is very useful), always remember to stick a ToArray() call at the end; this forces the query to evaluate immediately, and only once.

Buffer chains in GDI

Today's lesson: you can't implement back-buffer flipping with the DrawImage method in GDI. Well, you can, but it takes longer than the actual rendering, isn't multi-threadable and it still causes tearing. The original idea I'd had while implementing multi-monitor support in Synchron was to render everything to an off-screen buffer once and then copy it onto each screen using DrawImage, but a quick run through performance analysis put paid to that.

In the fullness of time I think I'll move to DirectX, but there's no rush.

Conquer & PageManager

Whilst I make the last changes to Synchron - I think the last thing remaining to implement is the mini-preview - I'm starting up a new project called Conquer, which is a fairly simple implementation of an RTS game. I've a few ideas around lines of communication, chains of command and other, real-world military concepts that tend to get glossed over in current RTSs but the truth is, I'm not really doing this to explore strategic game mechanics. Conquer is actually a vessel for me to test out a style of state management that I've called PageManager.

Let's say that we're simulating an object in 2D space: that object has a position (X & Y coordinates) and a velocity (change in X per second and change in Y per second) and we're updating the position of the object in real-time and then rendering it to the screen. The main thread of the simulation loops through two steps; first, it updates the position of the object by finding how far it has moved in this iteration, and second it renders the object in its new position. The distance the object has moved is the velocity of the object multiplied by the time span of this iteration; that is, the difference between the current time, and the time at which the last update took place. So if each update loop takes a second, then the position changes by velocity * 1; if each update takes half a second, then the position changes half as much per iteration, but there's twice as many iterations per second so the rate of change is constant. Ideally, we'd want at least 60 iterations per second to maintain a good frame rate.

So, that's the basics; the next step is to consider interacting values in the simulation. For example, let's say that our object is a spaceship; it has the properties of Position and Velocity (as before), but in this case it's propelling itself using an internal fuel source that steadily declines. As the fuel source is depleted, the power supplied is reduced and the velocity of our ship decreases - to simulate this, we'll include a property of Power, which has a Rate of Depletion.
In this example, the position is calculated as before, but velocity is now proportional to power and power decreases by "Rate of Depletion" amount per second. This means that there are several steps to the update and, more interestingly, the order in which they are determined has an effect on the result. If you update the position before you update the velocity, then your ship will always fly further on the same amount of fuel, because you'll use the earlier, faster velocity rather than the later, slower value. The same is true of updating velocity & power - if you update velocity before power, you get a better deal.

This gets a lot more complicated when you start simulating lots of objects interacting in various ways and becomes difficult to ensure that everything is evaluated in a specific order; this is especially true when you start working with multiple thread, where this situation is technically a race condition. So what's the solution?

A very neat and simple solution is to store every property on every object (at least, every property that changes over time) in pairs - a current value and a new value. During the update step, your code should always read the current value and write to the new value (e.g. NewPosition = CurrentVelocity * Time) - by doing this, you ensure that the current value remains constant and the calculations will come out the same no matter which order you evaluate them in. Once everything's done, you copy the new values into the current value variables (e.g. set CurrentPosition to the value of NewPosition) and repeat.

Now, I've not described anything new here, but we've reached the point at which I started thinking. The first thought that occurred to me is that copying all the values from New to Current is an expensive operation; instead, leave the values where they are, but use a boolean flag to indicate which variable is the current, and which is the new, then flip that flag for each iteration. For example, I have Position1 & Position2 properties on my object and I begin my update step with the flag set to false. This means that I update the variables numbered 2 from the variables numbered 1 (e.g. Position2 = Velocity1 * Time) - I also need to render from the properties marked 2, as these are the most up-to-date. In the next loop, the flag is set to true, so I update in the reverse direction (Position1 = Velocity2 * Time) and render from the opposite set of properties. This way, you get the advantage of separating the properties into a "current" and "new" pair, but without having to copy the values back and forth.

Expanding on this idea presents another possibility: splitting each property into several variables allows several threads (typically, an update thread and a render thread) to act on the same set of objects without interfering. So long as you ensure that the update thread never writes to the same copy of a property as the render thread is currently reading from, and that the update thread always reads from the newest copy, you can have both threads running at different rates - this is actually preferable, as the process of updating the simulated objects can vary drastically in length, whilst you want to keep the visual framerate fairly constant.

PageManager does this by dealing in pages & contexts. A page is simply a term for a copy of a property - for example, if there is a new and current copy, then there are two pages. A context provides two things: firstly, it provides a page number (similar to the boolean flag described above) that indicates which page the context is referring to and secondly, it locks the page for reading or writing.
There are two types of context: a Read Context, which locks a single page for reading (for use by render threads), and a Write Context, which locks one page for reading and one for writing (used by update threads). The PageManager is responsible for coordinating requests for and releases of contexts - when a Read Context is requested, the manager picks the newest available page (that is, the page that was updated last) and locks it. When a Write Context is requested, the manager locks a page for reading in the same way as for a Read Context, but also locks the oldest available page for writing. A page may be locked for reading multiple times, but a write lock is exclusive; neither read-locked or write-locked pages can be write-locked.

In this way, a render thread can request and hold a Read Context and take as long as it likes to render; meanwhile, the update thread requests and releases Write Contexts, reading from one page and writing new values to the other.

Hopefully that explains what the idea is about; next time, some code!