Lander - Dev Blog 3 and a bit

So the last week or so of work has been in fits and starts, with RL interruptions and Easter holidays and all that jazz, but this is the first update with actual vehicles in the game! There's a set of different "tools" now which the player can scroll through (for digging, placing blocks, building vehicle parts, etc), I've been making more tweaks to the player movement and voxel loading, and you can place launch clamps and vehicle parts (just fuel tanks for the moment). So, without further ado, the summary screenshot:

The launch clamp serves several purposes - from the game's perspective, it's the join between the dynamic, mobile vehicle model and the static terrain, so I don't have to be constantly checking if the player's done something daft like removed the bottom part and left a vehicle floating in mid-air - or at least, there's only one place I have to check. It also gives somewhere for the player to move and rotate the vehicle while they're working on it, rather than pretend the player is some sort of superhuman that can lift and rotate spacecraft at will. Finally, any interface activities the player needs (like saving or loading vehicle designs) will happen through the launch clamp.

The next chunk of work is to develop the vehicle object so that it's more than just a set of parts - mechanically it needs to merge all the parts into a smaller set of render & collision meshes, add up all the fuel tanks, mass and engines so it can simulate the actual flight, and possibly track more complex details like which areas of the vehicle are air-tight.

Lander - Dev Blog 2.5

Alight, this is Friday's and Monday's update rolled in to one...

My original plan was to get planet terrain rendering out to the horizon (with a sort of adaptive scaling as you gain altitude, or move around the map) but for a plain, flat heightmap, and then get the vehicle building interface working over the weekend; but it ended up making more sense to work on the terrain generation so that it scaled both in first-person view, and also up to continents and land-masses. There's a bit of bodging in there to get it just right, but you can see that the local terrain's come out pretty nicely:

You can also see a Mercator projection of the planet map (about the size of Earth), so it's distorted around the poles but you can see the continents pretty clearly. That's one continuous scaled heightmap, so the current terrain is smack in the middle of that map.

Most of my weekend went into that, and a bunch of optimising the voxel generator so it loads the current scene (262,144 voxels) in ~0.36 seconds and ~7Mb, which is good enough for the load I'm putting on it right now. I'm partway done on the heightmesh - in theory, it's just a surface mesh heading out to the visible horizon, scaled so it fits into the camera's clipping planes and follows the same heightmap (at a lower resolution) as the voxel nodes - but I've a few ideas to try out for it, based around how to progressively load it into the scene as the player walks or flies around the map.

Objective for Wednesday, then, is the launch clamp - a special "voxel" which you attach to the ground, and onto which you build your vehicles.

Lander - Dev Blog 1

I'm actually in my fourth week of development on Lander (excluding a week's holiday), but 90% of the work done so far is reworking the 3D coordinate system to work on an interplanetary scale, so there's not been much by way of shareable images.

But no more! I'm finally getting stuck in to actual gameplay mechanics, and the basic voxel engine is working:

You can walk around the surface, pick up and put down blocks, and the whole thing is correctly positioned 6371km above the centre of a spinning (though presently invisible) planet that's whipping round a distant sun every 365.25 days (hence the shadows). You'd be surprised how complicated that last bit actually is (here's the Wikipedia summary).

I'm polishing the digging/inventory/placing mechanics tonight, then for Friday's feature I need to extend the terrain model out to the horizon - using voxel nodes (like the 10x10 square you can see) out to about 1km, then a height-mapped curvature mesh out to the visible horizon, in a way that will adapt as you fly into the upper atmosphere.

Patreon - Dev Notes, the First

So, for the past few weeks I've been working towards an inaugural game for the Patreon page I'm aiming to set up; it's one part demoing to patrons how much I can deliver in the 4-6 weeks I expect each game to take, and two parts showing myself how much I can get through in 4-6 weeks. The longest (unpaid) projects I've actually run to completion are 48hr game jams, and as those usually involve less than an hour's sleep a day, they're probably not the best yardstick...

The current project is nicknamed "Lander", having started as a remake of the classic Atari arcade game Lunar Lander and spiralled into the illegitimate lovechild of Minecraft and Kerbal Space Program.
Long story short, it's a sandbox game in which you build rockets, fly into space and explore some PCG planets; but unlike (for example) KSP, you do everything from a first-person perspective - the terrain is a big, voxelly map that you can rebuild into scaffolds and launch pads, and (if I hit enough milestones) convert into resources for building your rockets. I think the lack of central build & launch location (unless you make one), with the ability to build & rebuild vehicle wherever you are, should give this a very different atmosphere to some of the existing space flight simulators.

The plan is to get a workable feature finished every Monday, Wednesday & Friday (with corresponding blog update) to stop me getting lost down a rabbit-hole of quaternions like I have with Lander - anyhow, that's the quick intro, next up is the current progress report.

Ludum Dare 26

It was Ludum Dare 26 last week - wasn't terribly keen on the theme of "Minimalism", which seemed to push people in three directions - a literal interpretation, with minimalist art & music; token gestures, where they did whatever the hell they felt like and put "minimal" in the title; and abstract, where minimalism was a feature of the game setting - e.g. the villain you're fighting is a minimalist. Didn't see many where the theme inspired a novel game design; especially annoying is that they've already had this theme for a previous LD.

Anyhow, my offering can be found at http://www.ludumdare.com/compo/ludum-dare-26/?action=preview&uid=8404

I also wrote a quick post-mortem here http://www.ludumdare.com/compo/2013/04/29/ludum-dare-26-post-mortem/

Algorithm for Dividing a 2D Space into Convex Polygons

I'm writing this down mainly as an aide-mémoire, and because I can't find the solution online anywhere - I can't imagine it isn't out there, because I'm trying to solve a fairly basic problem, so I'm possibly using the wrong terminology in my search.

In any case, I have a simple engine for moving a player character round a 2D game map which is split into convex polygons/cells linked by portals/doors; I'm also working on a game that, in essence, allows a player to build and walk round their own spaceship. To fit these two together, I need an algorithm that will take the outline of an area of the ship (itself a convex polygon), a set of walls that the player has added (straight edges within that area) and divide the outline into a set of convex cells such that none overlap a wall, and that any edges between cells that are not walls are marked as portals between them.

My five-minute, scribbled on a PostIt note algorithm is:

• Every wall lies along a line that bisects the outline area into two parts. Filter out all walls that bisect the area such that any other wall overlaps both parts.
• For each remaining wall, select the set of walls that most evenly divide the other walls (from the full set) between the two parts.
• Pick a wall at random from that set and split the outline area into two new areas, adding portals along the dividing line where appropriate, and group the remaining walls from the full set into each area.
• Repeat this algorithm for both new areas with their reduced wall set.

Notes:

There are two special cases - where two walls are in line with each other, and two walls directly intersect. The latter case is probably easiest to prohibit, and the former case is handled by only including one of the two walls in the selection process, but remembering to include the second wall as a non-portal edge when splitting the area.

The Social Graph Pt3: Inversion of Control

(This post is part three of a series: here's part one & part two)

Okay, I've had my rant and ramble about the current situation with Web 2.0 social networks, what I think the Social Graph is and why I think the existing implementations don't match up to the reality; so now I'm going to propose my idea for where we should go next.

The core of this idea is that a social network is comprised of many different services, each of which is atomic¹ (though potentially inter-dependent). The central, quintessential service is the identity provider - a service that can vouch for you, presenting an online identity. When you use Sign In with Twitter or Facebook Connect, that's an identity provider in action.

An extension of the identity provider is the claims provider; this service handles the permissions that you give to the services you use. For example, a service may wish to send you notifications (via a notification service), but to do so it requires permission from you. So that it doesn't have to contact you every time to ask permission (which would just be ironic, given that it's asking permission to contact you in the first place) that service can contact the claims service and be issued a claim token (a digitally signed piece of data essentially saying "Service X may perform action Y") that it keeps hold of and presents to other services (like the notification service) to prove that you gave permission back when you registered. This is the same as the permission options that you get for Facebook apps, allowing an app to post on your timeline, send you emails, read your profile data, et cetera. In essence, the claims service controls which other services can perform tasks on your behalf or access your data.

This leads on to some of the other requisite services for a functional social network; you'll need a profile service that shares your personal information (given appropriate permissions, of course); a notification service that feeds information back to you via SMS, email or in-app notifications; a contacts services, which stores your list of friends, follows, "circles" and so on, acting as a claims service but for other identities, and as a routing map, saying who can see what parts of your profile, follow your feed or receive updates from you (storing the structures of the Social Graph); a feed/publishing service, that lets you share those statuses and updates; a photo-sharing/file-sharing service; a tagging service; and so on.

Of course, none of this is especially revolutionary, and already exists explicitly (as in Facebook) or implicitly (as in the ecosystem around Twitter). Where it gets interesting is when we consider the case of services interacting with each other in the abstract terms above.

Let's say I sign up to one of the newspaper apps that seem to have sprung up on Facebook recently. The app wants to know who I am, presumably to track my usage and which articles I read, so I type in my identity name (an example might be my email address, or my blog URL) then click the big sign-in button; it goes off to the identity server and verifies my identity (using an established process like OpenAuth or OpenID), then lets me in and I can go read articles. However, the newspaper app also wants to contact me; perhaps to let me know when new articles are published, or that someone has replied to a comment I posted. To do this, it needs to know how I want to be contacted; in this example, we're using the notification service which holds information about my contact details and preferences - the notification service I'm using (in this example) batches up my notifications and sends them in an hourly email. Someone else, using a different notification service, signs up to the newspaper app; they receive their notifications as SMSs. Both notification services implement a standard NotificationAPI v1.0, so all the newspaper app requires from me and the guy who prefers SMSs is the URL of our distinct notification services, and we then each get our customised notification behaviours. This is not unlike the drop-down in TweetDeck that allows me to pick which photo-sharing service I want to use (as mentioned back in part 2).

The downside to this approach is that every time I sign up to a new service or application, I have to configure it with all the different services in my customised social network that it happens to need; if it depends on a lot of services, this would be a nightmare. Equally, if I decide that actually, my notification service provider sucks and want to switch, I'd have to go round all my other services & reconfigure them. This leads, logically, to the need for a service provider service (mmm, metaservices) which keeps track of the different services that I use for different purposes. My identity service will likely know which service provider service (and claims provider service) I use, and all the other services I sign up to can then find out, via the identity service, which other service to use for each different purpose. In our newspaper app example, the app would contact my identity service to verify who I am & find out where my claims service & service provider service are; contact the claims provider to get permission to contact me; then contact the service provider service to find out which service to use for notifications. All of this happens entirely out of sight of the user, excepting the specific moments when user input is required - when the identity service needs the user to input their password, and the claims service checks with the user that the newspaper app is legit.

In a general sense, again, none of this is revolutionary. This pattern is already commonly used in software development - see dependency injection and inversion of control; and it's that latter concept that I think is so important. By controlling the injection of dependencies (e.g. providing your own choice of notification service) into the applications & services you're using, control of your social network switches from the service providers (Twitter, Facebook, Google, etc.) back to you. With interoperability via an open, published set of API standards, by returning control of each user's social networking platform to the user, it is possible to produce a free (or, at least, freer) market of services that users can combine to suit their needs and preferences; and diminish the barriers to data flow through the Social Graph.

There's plenty more detail to the design than that, of course; how different types of service can be described and denoted, how evolving standards (does your notification service support NotificationAPI v2.0?) are handled, how we bootstrap new users without presenting them with a hundred drop-downs (which notification service do you want? And your file-sharing service? And your URL shortener? Which flavour of sommelflange takes your fancy?), but I've covered the core of the concept. The next stage is to build a proof of concept, a small ecosystem of services from which you can devise your own social networking platform - another item for my long list of projects - so I'll update as that matures and hopefully have something to demonstrate as summer progresses. Feel free to get in touch if this sounds like something you'd like to help with, as there's plenty of work to do... ;-)

[1] Atomic in this sense meaning that a service is not coupled to or bundled with another service; and that it provides one (type of) service only. A service host might provide multiple services, but there's no requirement to use them all if you only need one. e.g. a photo-hosting service just hosts photos - it doesn't provide tagging, and a separate photo-tagging service is required for that.

Non-Euclidean Software

I've finally moved into this contracting gig and formed my own company: Non-Euclidean Software. Yes, the name is Lovecraft in-joke. No, it's not a very funny joke. Yes, the website is just a placeholder. I'll get round to adding some relevant bits like access to my open-source projects and the like when I have a free evening.

The Social Graph Pt2: Facebook is a Big Truck (in a Walled Garden)

And we're just dumping all our data on it.

(This post is my (somewhat belated) follow-up to an earlier ramble about viewing the Social Graph as an active medium and not just dataset.)

Imagine you wanted to send an email to someone; you have a GMail account and they have a Hotmail account. Sadly, you can't; you have to sign up with Hotmail (or, perhaps, convince your friend to switch to GMail) because in this world, email providers don't talk to each other. To be able to talk with everyone, you'd have to have multiple accounts - then again, you may already, because each email provider bundles additional value-add services along with email. Of course it's not actually integrated with the email service, so much as it is bound to the identity that your email address represents, but it's still an integral part of one service competing with another.

You will likely have realised I'm drawing an analogy of the current ecosystem of monolithic social networks - you cannot send a Direct Message from Twitter to a friend on Facebook¹; the more astute (and elderly, and probably also American) among you, however, will realise that I'm describing something that actually happened, back in the early days of the public Internet (80s to early 90s). The early ISP-equivalents peddled proprietary email, forums and file transfer services that only functioned within their networks; my experience of this was with CompuServe in '94, though by this point the ISPs were integrating more and it was only a couple of years before Internet access was opened up fully - I think I got my first Hotmail address in '96/97.

There are various flaws to this approach, some more obvious than others. An immediate issue is the network effect - if all my friends are on service A, I'm likely to join that service; but if a new service starts up with an amazing new feature, I need to drag all my friends across to make it worthwhile. And, as Google+ is discovering, the nature of the Social Graph as a dataflow network means that having friends on a network isn't enough - there has to be content being generated and flowing through the graph to make it worthwhile.
I've always been a believer² in the free market and would argue that customer lock-in stifles innovation & choice, while choice empowers users and enables them to fight back against abuses of power; and when we're talking about services that facilitate the flow of information, I would argue that partitioning dataflow networks can have as great an impact on the development of society as putting up physical barriers³.

But there are other, more integral factors. The issue of security is well-explained in this XKCD comic; essentially, the more services I sign up to the more risk there is of my identity being compromised. The solution to this is already in place - Facebook, Twitter and other services are allowing integration through Facebook Connect, OpenAuth & OpenID that opens their identity service to other service providers, resulting in an improved user experience (one-click "sign-ups"), improved security (as your username & password is only held by one service) & reduced fragmentation (as your identity on different services remains constant across all). This openness empowers and enables users to switch freely between identity providers without impacting their experience of a given service; just as I was able to switch between BT & VirginMedia as my internet providers without my access to the Internet changing (beyond, of course, an increase in speed on a cable connection - the distinction here being between quality of service & nature of service), I was able to switch from an LJ login to a GMail login when using StackOverflow, and my experience of the service remained the same (note that here we must distinguish between my identity as an individual (which is what StackOverflow cares about) and the identities presented by LJ & GMail distinctly, which function more as aspects or façades).

Indeed, the question of choice is central to the problem with modern social networks. If you don't like Facebook's handling of private data or their use of advertising to monetise their service, your choice is to stay or to leave; to have or have not. If you have concerns about Twitter's censorship policies or Google's data gathering, your choice is to have or have not. There are numerous issues, from constantly changing UIs to the ever-waiting Fail Whale to freedom of speech, that can only be tackled when users have the choice to have this or have that.

There is (and has been, slowly developing, over the past year or two) a movement to decentralise & distribute the functionality provided by the monolithic service providers. Consider this blog post on the move to federated social networks, the Diaspora* project (and on Wikipedia), a distributed, user-hosted social network, or YaCy, a decentralised search engine. It an important step, and with enough time (and a good UX) these projects should help us to dislodge the grip of monolithic Web 2.0 services; there is, however, a further step required in this process, and you can see it developing in the ecosystem of services around Twitter.

In addition to the core micro-blogging functionality of Twitter, references to other media are now possible through the emergence of dedicated value-add services (e.g. TwitPic, TwitVid, Deck.ly) or the filtering of existing media hosting (YouTube, web-comics, blogs, online news, etc.) through URL-shorteners⁴. Twitter clients are now integrating with a range of these to provide wider (and yet seamless) functionality to users, but without necessarily constraining users to a specific service. This quick snapshot of the TweetDeck options page shows this choice in action; there's two types of service that TweetDeck can consume, but the choice of which specific service provider is up to me. We're seeing two parallel developments in progress here:

• The ability for software (in this case a client, but potentially services) to consume other services⁵ on behalf of a user (and their associated identity)
• The ability for users to build & customise a conglomerate social networking service that provides the specific functionality that they need, whilst remaining completely compatible with everyone else's custom social network

The move to integrate services in this way is amply demonstrated by If This Then That, a Yahoo-Pipes-Lite that allows users to build active links between their services. IFFT uses a set of purpose-built adaptors for each type of interaction, but by using the standardised interfaces & APIs for content sharing & notifications that are in development by the federated social network projects, this can all be generalised and made freely interoperable.

---

Right, it's taken nearly two months of talking to people and rewriting to get this post this far, so I'll pause here and promise a third & final part with my thoughts on the ideal solution...well, it'll take less than two months, anyway.

---

[1] Indeed, a "friend" or a "subscribe" on Facebook is an entirely different and incompatible concept to a "follow" on Twitter.

[2] In the same sense that I believe free speech is generally beneficial if you're not a dick about it, as opposed to the way I believe in dinosaurs (they existed, but I doubt they'd be beneficial to the economy).

[3] Hmmm, that's probably worth a rant of its own. Adam Smith's Division of Digital Labour?

[4]The passing of data references (Pass/Call by Sharing) is a familiar concept to programmers, enabling developers to manipulate and exchange heavyweight objects (large data files like images & videos) using only lightweight references (URLs in this context).

[5]Oh, my goodness! Shut me down! Machines making machines. How perverse.

The Social Graph is a Series of Tubes

Okay, this one is a bit of a stream of consciousness, so bear with me.

So I was reading this, a rant on why the Social Graph is something of a misnomer, and it stirred up some thoughts that I've been mulling for the past year or so. Note that I don't actually agree with several things the article had to say, but that's a post for another time; instead I'll respond to some specific points that were made and the train of thought those kicked off.

"In order to model something as a graph, you have to have a clear definition of what its nodes and edges represent. In most social sites, this does not pose a problem. The nodes are users, while edges means something like 'accepted a connection request from', or 'followed', or 'exchanged email with', depending on where you are.

The way you interpret this is another matter - does clicking 'follow' imply you're friends with someone in real life?"

There's a few points in the article where the author questions the nature of the arcs or links in the social graph, and what those represent - sadly missing the equally complex discussion of what a node represents, and how this relates to issues of personal identity online and the many façades we use - and while the discussion oversimplifies somewhat, the point is generally sound: the links between people cannot be simplified to or interpreted as Is Friend/Is Not Friend (or some variant thereon) and they are intrinsically dynamic (indeed, it is a lack of activity that can cause the greatest change in interpersonal relationships). A point that the author skirts but fails to deliver specifically is that the Social Graph as espoused by, for example, Facebook, deliberately (and vocally) misinterprets the links that its users create and destroy. When I "friend" someone, it's because I want to see the information they're sharing; I want to follow them. When I "unfriend" someone, it's not usually because I'm no longer friends with them - there's plenty of people linked to my Facebook account with whom I have no interaction at all - it's because I'm tired of listening to their shit. The status updates you've been posting for the past few weeks are actually really offensive, and I don't want to hear them any more; it's just a difference of opinion, and I'm not going to stop being your "real life" friend because we disagree on something, but I don't want to subscribe to your thoughts on the matter. You only use Twitter to play Echo Bazaar; there's nothing wrong with that and we're still friends, but there's no signal and all noise in your feed. We've broken up, and whilst we've resolved to stay friends, I don't especially want to see the pictures of your annoyingly handsome new boyfriend.

Changing the links between myself and the people I "follow", "friend" or otherwise is, to me, not a matter of keeping my data up-to-date in the same way as changing my relationship status, my address or my profile picture, it's about changing my experience of the service. I'm not expressing the status quo, I'm not describing the actual Social Graph (which is definitely an actual thing, even if we've not found it yet) I'm building a dataflow network. Google+ has started to grasp this with the Circles mechanism, allowing me to distinguish between the people I trust to see my information (friends), the people with something interesting to say (followees), and the overlaps between them. But I contest the article's implicit assertion that the links in the Social Graph (as present in social networks) are nouns, "friend of", "lover of", "enemy of", "employer of"; instead, I say they are actions, verbs. My links say things like "trust", "follow", "ignore", "inform". I'm telling the social network to do something, I'm building a set of pipes through which I can broadcast, send, receive & transform data.

Maybe what I'm trying to say is that the Social Graph is not the data, it's the medium. A friendship, a relationship, an emnity, these are not tags or metadata, they're processes, they're impellers, they're agents of change and movement.
Is that the difference between Amazon Recommendations/Google Ad-Targeting and viral marketing? Because the former uses the Social Graph as data, churns it through an engine and tries to drive itself along those lines, while the latter lets the natural forces of human society carry the message?

"One big sticking point is privacy. Do I really want to find out that my pastor and I share the same dominatrix? If not, then who is going to be in charge of maintaining all the access control lists for every node and edge so that some information is not shared? You can either have a decentralized, communally owned social graph (like Fitzpatrick envisioned) or good privacy controls, but not the two together."

Yes. And also, no. But mainly, sort of.

The present system of social networks lumps all the data in one, big central pile; there's a service out there, somewhere, on the net to which you send all your private details. This service is also the identity authority; you get a username and password to identify yourself, but it's up to Facebook, Twitter or whoever to decide if you are who you say you are. As an identity, you then ask the central service to allow or deny other identities access to your data. It's an entire world in a box, massively centralised and under the absolute authoritarian control of the service. This means the privacy policy of that particular service can be absolutely enforced, but it does mean sending all your private data to Mark Zuckerberg.

I'm loathe to comment overmuch about the author's apparent assumptions around privacy and control in the Semantic Web alternative; I need to read the essay by Fitzpatrick in more detail (though at first glance it seems to explicitly leave the topic of private data as an known requirement to be resolved in future) and the article provides no justification for the statements quoted above. But I do believe that a better standard of privacy and control can be achieved if the responsibility for managing data and identities is transferred to (and made practical for) users. I still believe that as Web2.0 features user-generated content, Web3.0 will feature user-published content; just as content generation has decentralised, so will content publishing; that we will all be our own YouTubes, Facebooks et cetera.

Okay, I've got to go do something social, so I'll be back in a bit to finish of this post with the actual endpoint of my train of thought; but in the meantime, here's a list of the all functionality I could think of that a social network would need to provide:

• A Feed: I want to share a stream of short statuses/tweets, links to articles, videos, et cetera. I also want to reshare things that have been shared with me.
• Content hosting: I've generated some of the content that I'm sharing, and I need to host it. Further, the act of sharing content is content in and of itself.
• Data sharing: I want to share information about myself. That may be my age, address, relationship status, or my dating proclivities, or where I am in the world, or my CV, or whatever. In a sense, this is just another form of content, but it's a specific format of content to which social networks cater.
• Identity: I need my content to be identifiable and attributable as mine. If someone re-shares my content, I want my identity to stay attached to it. Note that identity does not implicitly equate to me as a person; I may have several online identities, or an identity may belong to several people (like a company or brand).
• Privacy: I need to control who gets to see what content, and - so far as possible - the period of time for which they have that access. My friends get to see all my personal data, but a potential employer can see my contact details and CV for a few weeks after I submit a job application. That control should not be affected by the channels on which I share content; especially, it should not be affected by who chooses to reshare it.

Anything I've missed?

What I did with my Weekend (a one-man Game Jam retrospective)

Having missed yet another game jam (TIGJam UK 5, this time) by being at a LARP event, and suddenly finding myself with an unexpectedly free weekend, I decided to have a gamejam of my own (because that's just how cool I am). I'd been mulling over some ideas for a Dwarf Fortress-meets-The Sims-meets-Day of the Triffids game - a sort of Survival Strategy - since watching the 2009 remake of Triffids, so I sat myself down on Friday night and started coding - and here's what I learned from the experience:

Writing Design Notes

It's kind of obvious that writing a design is useful, but the hard part (I've always found) is knowing what to write down. When I have a code architecture in mind, it's tricky to try and work out which bits are obvious and which bits I'll forget and wish I'd written down. Equally, it's often as quick for me to write the code framework, and then use the code as the documentation; which is fine for me, because I've more practice of reading and writing code than reading and writing English, but other people less so.

Over the course of the weekend, I was making the whole thing up as I went; there was no grand design in mind, I just wrote down a list of features and tackled them one at a time. That level of focus (and the particular focus on How Do I Do This Quickly over How Do I Mix All These Into One Unified Symphony Of Code) meant those individual notes flowed more organically into a fleshed-out design, even if that design was just a tree of This Feature needs ==> This Features needs ==> This Feature; and probably This Feature, too. It was also more obvious which design decisions needed to be recorded for future reference when the code was less structured and self-describing.

Coder's Block

Realisation of the weekend: The more you know about something, the harder it is to code it. I know all about AI & Pathfinding - it was the topic of my degree - but somehow the language in which I understand those academic topics is very different from the language in which I express code. Equally, when I'm writing game code (I've hit this block a few times before) my mind jumps ahead, because I already know the (implementation) answer to the questions I've not quite reached yet; but that means that rather than smoothly grow my code base, I'm stuck trying to work out how to plug together the code I've written, and the code I know I'm going to have to write in the near future.

So I stopped thinking ahead. The emphasis on "just get it working, it doesn't have to be pretty" finally pushed me through the block, as I stopped thinking in terms of a grander design and just focused on the bit of code right in front of me. Of course, just getting it to work means that the pathfinding code I have now is nothing like as efficient as it could be, but it works, and it'll be easier now to go back and polish it up, now that I've tied it in to the rest of the code.

Take better care of yourself

Staying up too late on Friday night (by which I mean 8am Saturday) threw my whole sleep cycle off; I wasn't properly focused on Saturday/Sunday & would estimate I hit a 50% velocity for the whole weekend, when I should really have managed 70-80%. What I got right was my eating habits, for once; plenty of caffeine shots to keep me alert, glucose drinks and "energy" snacks (mmm pistachios) to replenish my blood sugar (continuous mental activity can really drop your blood sugar levels) and only eat "heavy" foods with complex carbs at main meals (I'll confess I cheated and ordered in pizzas, but hey, time saved cooking, right?) because the effort of digestion makes you sluggish.

Prep your materials in advance

Ye gods, the time I lost searching for isometric sprites. I'm not an artist by any stretch of the imagination, so all the art I used is downloaded from very generous people on the 'net; but there's a lot of dross out there, and to make things worse, many of the sites that do publish free sprites are horrific to navigate and search. I was really lucky to find OpenGameArt, from where I got most of the artwork that I used in the game, but ultimately the time I spent searching for and prepping sprites (especially given my lack of skill or experience) was time I couldn't spend building features. So next time, I'm working out a shopping list of the artwork I'm likely to need and getting it together before I start.

Play games before you write games

The first thing I did was to sit down with a notebook and write out a few key phrases and ideas about the game. The second thing I did was to think of all the games that those reminded me of and see if I could have a quick playthough of each; which meant that the first hour or so of Friday evening was spent playing X-COM & The Sims, re-learning the interfaces and "vocab" of an isometric UI. I guess this point (for me) is more about user experience and interface; I don't have anything new to bring to the table in those fields, so I'd rather point to previous games and say "They got it right, so I followed their example." and focus on gameplay, which is what I'm trying to improve.

Of course, having talked about all this I should mention what I actually achieved over the weekend, which is admittedly not a great deal. The game idea I had was quite grand for two-day sprint, so the result is more of a simple tech demo than a playable game, but I've uploaded it for now and will see about turning into something actually fun at the next jam; or maybe at the next CB2 Indies meet (there's one tomorrow, actually, will see about heading along).

Go here to get the current download; when you run it up, you'll see the example map with a single survivor. There is code in place for ambling/hunting/killing zombies (hence the red health bar about the survivor's head) but I need to wire up a button or something to spawn one on the map. A task for tomorrow, I guess.

Anyhow, middle-click somewhere on the map to refocus the camera on that spot, and the scroll wheel moves the camera up and down levels - note the basement under the house.
Left click the survivor to select them, then right-click to tell them to move; sorry about the lack of animations...
With the survivor selected, click the wall button in the top-left to select the Build Wall tool; then right-click on two locations on the map (two clicks, not click & drag) (also, two locations in a straight line) to tell the survivor to start building a new wall. It is a slow action, so don't worry if they seem to move to the site of the new wall and stare blankly at it for a few seconds - they're working, honest!

I'm fairly pleased to have got all that working in a single weekend; it's not exactly fun, but it's broken the back of building a simple Tower Defense vs Zombies game, and from there is just a matter of adding ever more awesome.

Epilogue install

Prereqs

The game is built on v4 of the .NET Framework (Client Profile) & the XNA Framework - that makes it Windows-only, I'm afraid (XP SP3 or later).

Download & Install

You can download the game from here. There's no installer, so just run the Expilogue.exe file in situ.

Core MSIL Assembler

This post contains instructions for downloading/installing the current prototype of the Core MSIL Assembler, and will be updated as new versions are published; explanations of what the assembler is and what it does will appear in later posts.

Prereqs

The assembler is built on version 4 of the .NET Framework. This is automatically installed by Windows Update on Windows 7 & Vista, and is an optional update for Windows XP; you can also download it here.

Download & Install

You can download the assembler here. There's currently no installation step, just unzip and run the CoreAssembler.exe app where it is by dragging a xxx.core file onto it. There's also a CompileAll.bat file that you can run to automatically compile all the example code.

AxumMUD

Just a quick note to say that the code for AxumMUD is online at CodePlex. The network layer is basically complete, though I'll likely go back and refactor when I've got some time, but I'm currently working on the Character/Location design and event communication; hopefully I'll get that checked in later in the week.

New keyboard shortcuts

The fn key on my new laptop has shifted all the ctrl/alt/et cetera keys one to the right, so I keep hitting the wrong ones and discovering new keyboard shortcuts. So I'll note them here as I discover them:

In Visual Studio 2008
Shift+Alt+Left/Right Arrow: Same as Ctrl+Shift+Left/Right Arrow (selects whole tokens at a time), except it selects words within the token, delimited by capitals. E.g. if the carat were at the end of the token "BuildNewConnection", pressing Shift+Alt+Left Arrow would select "Connection".

In Windows 7
Win + Left Arrow: Aero Snaps the current window to the left of the screen, then rotates through Aero Snap Right, Restore, Aero Snap Left et cetera.
Win + Right Arrow: Aero Snaps the current window to the right of the screen, then rotates through Aero Snap Left, Restore, Aero Snap Right et cetera.
Win + Up Arrow: Maximises the current window.
Win + Down Arrow: Minimises the current window.
Shift + Win + Up Arrow: Stretches the current window to the top and bottom of the screen.
Win + Home: Aero Shake (minimises all windows except the current window).

Axum!

I've been on a bit of a hiatus, both from this blog and from doing any major work on my Codeplex projects, since around April; guess the day job's been taking up all my valuable coding time :-D.

But a few days ago, I downloaded the latest version of Axum (now v0.2), Microsoft's new .NET language/incubation project for introducing actor-based, highly-scalable and massively concurrent code to the .NET framework, and thought I'd post some of my experiences. I first heard about this back in November 2008 at TechEd EMEA, when it was called Maestro, and I was really excited - the work I'd been doing on Lite was in the same vein, trying to make distributed, concurrent code both trivial, implicit and powerful for programmers to use, whilst retaining all the good stuff that you get with .NET. Since the first CTP came out, I've been converting a few of my projects and writing some sandbox code to find out what works, and what doesn't, in the Axum programming model.

I don't want to get to much into the design philosophy, as it's fully explored elsewhere on the web, particularly on the Axum blog (here); instead, I'll be uploading code snippets that do interesting things or exploit the features of the language, and useful code patterns for developing in Axum. So yeah, expect some Axum-related goodness in the next few posts. In the meantime, I'd recommend downloading the preview, trying out some experiments of your own and giving the dev team feedback; as an incubation project, they need all the community support they can get.

Lazy Evaluation Gotcha!

Note: I'm assuming a vague knowledge of LINQ, and I'd recommend reading up on Lazy Evaluation on other blogs as I only skim over the concept here.

Code-writing for the Conquest project is currently paused while I do some research around the subject area (RTS games), particularly around the unit AI, which is my immediate bugbear; in the meantime, I've been focusing my efforts on Setun, a project that I started early last year. I'll go into more detail on exactly what I'm aiming to achieve with the project in a later post, but for now I can summarise it as a Balanced Ternary virtual machine simulated at the logical electronics level; that is, I'm simulating And and Or gates, Decoders and other similar-sized components, but not going all the way down to transistors and shifting voltages. This is lower-level than a typical VM, but not a strict physical simulation.

I've been increasingly using LINQ in this project; each large-scale component (like registers and the ALU) is typically joined to the next by a bus, or a collection of electronic lines (i.e. wires) and I'm simulating this with an IEnumerable of ILine. Now, when you're handling lots of IEnumerable objects, concatenating them, picking out individual items, taking subsets and so-on, then LINQ is perfect; manipulation of sets is exactly what it's designed for.

However, one of the features of LINQ that often catches people out is Lazy Evaluation, which, if you're not familiar with it, simply means that any LINQ query you create (e.g. pick the fifth object out a list of strings) is not executed (or evaluated) until the results of the query are actually needed. In most cases, this is really useful; queries that are declared but never used are never evaluated and optimisations are performed once the full query is declared and applied, which is better than optimising piecemeal as the query is constructed.

But...

If the timing of the query is important, Lazy Evaluation can catch you out. If you declare a query to pick the first three items of a List, clear the list then try and display the items that you just queried for, you'll find that your query returns nothing. This is because the query now points to an empty list, as it was evaluated after the list was cleared.

Now, I've pretty heavily summarised Lazy Evaluation, because there's plenty of blogs that explain it in enough detail, so instead I'll focus on the behaviour that caught me out; that is, multiple evaluation.

I've been using the Select method to create sets of objects from other sets of objects - for example, I could take an IEnumerable of ILines, and wrap them all in Not gates, like this:

var lines = new List<ILine>();

//Insert some ILines here!

var invertedLines = lines.Select(line => new NotGate(line));

Which gives me a set of NotGates, one for each input line - this is all well and good. But, I tried the same trick with registers; unlike Not gates, which are pretty simple (inverting whatever their input is) registers store a value, only changing on clock ticks. When I tried to display the contents of the set of registers, they always came out as 0 - the initial value. No matter what input I gave, they wouldn't change.

A little investigation later, and I realised the problem: because Lazy Evaluation occurs when you use the result of a query, it happens every time you use it. Each time I iterated over the IEnumerable of registers to output the contents, the LINQ query was creating a new set of register objects, which of course contained the initial value.

So remember, if you're using Select to create sets of objects (for which it is very useful), always remember to stick a ToArray() call at the end; this forces the query to evaluate immediately, and only once.

Buffer chains in GDI

Today's lesson: you can't implement back-buffer flipping with the DrawImage method in GDI. Well, you can, but it takes longer than the actual rendering, isn't multi-threadable and it still causes tearing. The original idea I'd had while implementing multi-monitor support in Synchron was to render everything to an off-screen buffer once and then copy it onto each screen using DrawImage, but a quick run through performance analysis put paid to that.

In the fullness of time I think I'll move to DirectX, but there's no rush.

Conquer & PageManager

Whilst I make the last changes to Synchron - I think the last thing remaining to implement is the mini-preview - I'm starting up a new project called Conquer, which is a fairly simple implementation of an RTS game. I've a few ideas around lines of communication, chains of command and other, real-world military concepts that tend to get glossed over in current RTSs but the truth is, I'm not really doing this to explore strategic game mechanics. Conquer is actually a vessel for me to test out a style of state management that I've called PageManager.

Let's say that we're simulating an object in 2D space: that object has a position (X & Y coordinates) and a velocity (change in X per second and change in Y per second) and we're updating the position of the object in real-time and then rendering it to the screen. The main thread of the simulation loops through two steps; first, it updates the position of the object by finding how far it has moved in this iteration, and second it renders the object in its new position. The distance the object has moved is the velocity of the object multiplied by the time span of this iteration; that is, the difference between the current time, and the time at which the last update took place. So if each update loop takes a second, then the position changes by velocity * 1; if each update takes half a second, then the position changes half as much per iteration, but there's twice as many iterations per second so the rate of change is constant. Ideally, we'd want at least 60 iterations per second to maintain a good frame rate.

So, that's the basics; the next step is to consider interacting values in the simulation. For example, let's say that our object is a spaceship; it has the properties of Position and Velocity (as before), but in this case it's propelling itself using an internal fuel source that steadily declines. As the fuel source is depleted, the power supplied is reduced and the velocity of our ship decreases - to simulate this, we'll include a property of Power, which has a Rate of Depletion.
In this example, the position is calculated as before, but velocity is now proportional to power and power decreases by "Rate of Depletion" amount per second. This means that there are several steps to the update and, more interestingly, the order in which they are determined has an effect on the result. If you update the position before you update the velocity, then your ship will always fly further on the same amount of fuel, because you'll use the earlier, faster velocity rather than the later, slower value. The same is true of updating velocity & power - if you update velocity before power, you get a better deal.

This gets a lot more complicated when you start simulating lots of objects interacting in various ways and becomes difficult to ensure that everything is evaluated in a specific order; this is especially true when you start working with multiple thread, where this situation is technically a race condition. So what's the solution?

A very neat and simple solution is to store every property on every object (at least, every property that changes over time) in pairs - a current value and a new value. During the update step, your code should always read the current value and write to the new value (e.g. NewPosition = CurrentVelocity * Time) - by doing this, you ensure that the current value remains constant and the calculations will come out the same no matter which order you evaluate them in. Once everything's done, you copy the new values into the current value variables (e.g. set CurrentPosition to the value of NewPosition) and repeat.

Now, I've not described anything new here, but we've reached the point at which I started thinking. The first thought that occurred to me is that copying all the values from New to Current is an expensive operation; instead, leave the values where they are, but use a boolean flag to indicate which variable is the current, and which is the new, then flip that flag for each iteration. For example, I have Position1 & Position2 properties on my object and I begin my update step with the flag set to false. This means that I update the variables numbered 2 from the variables numbered 1 (e.g. Position2 = Velocity1 * Time) - I also need to render from the properties marked 2, as these are the most up-to-date. In the next loop, the flag is set to true, so I update in the reverse direction (Position1 = Velocity2 * Time) and render from the opposite set of properties. This way, you get the advantage of separating the properties into a "current" and "new" pair, but without having to copy the values back and forth.

Expanding on this idea presents another possibility: splitting each property into several variables allows several threads (typically, an update thread and a render thread) to act on the same set of objects without interfering. So long as you ensure that the update thread never writes to the same copy of a property as the render thread is currently reading from, and that the update thread always reads from the newest copy, you can have both threads running at different rates - this is actually preferable, as the process of updating the simulated objects can vary drastically in length, whilst you want to keep the visual framerate fairly constant.

PageManager does this by dealing in pages & contexts. A page is simply a term for a copy of a property - for example, if there is a new and current copy, then there are two pages. A context provides two things: firstly, it provides a page number (similar to the boolean flag described above) that indicates which page the context is referring to and secondly, it locks the page for reading or writing.
There are two types of context: a Read Context, which locks a single page for reading (for use by render threads), and a Write Context, which locks one page for reading and one for writing (used by update threads). The PageManager is responsible for coordinating requests for and releases of contexts - when a Read Context is requested, the manager picks the newest available page (that is, the page that was updated last) and locks it. When a Write Context is requested, the manager locks a page for reading in the same way as for a Read Context, but also locks the oldest available page for writing. A page may be locked for reading multiple times, but a write lock is exclusive; neither read-locked or write-locked pages can be write-locked.

In this way, a render thread can request and hold a Read Context and take as long as it likes to render; meanwhile, the update thread requests and releases Write Contexts, reading from one page and writing new values to the other.

Hopefully that explains what the idea is about; next time, some code!

Synchron (continued)

I guess the important thing to know about Synchron is how to tell the time with it!

Each unit of time (hours, minutes, seconds) is represented by a separate wave of differing amplitude, much like each hand on a (24hr) clock; the hours wave (the largest wave) is the hours in a single day, the minutes wave (the next largest) is the minutes in a single hour and so on. As time progresses, those waves move past a central line that marks the current time. The trough of the wave (left and right edges on the example image) is the crossover from one day/hour/minute to the next (for example, midnight), whilst the peak of the wave (the centre of the image) is half way through (e.g. midday). So whilst it's difficult to tell the exact time, you can get a reasonable estimate by looking at where each wave meets the centre line and where it is between the peak and the trough of the wave.