Non-Euclidean Software

I've finally moved into this contracting gig and formed my own company: Non-Euclidean Software. Yes, the name is Lovecraft in-joke. No, it's not a very funny joke. Yes, the website is just a placeholder. I'll get round to adding some relevant bits like access to my open-source projects and the like when I have a free evening.

The Social Graph Pt2: Facebook is a Big Truck (in a Walled Garden)

And we're just dumping all our data on it.

(This post is my (somewhat belated) follow-up to an earlier ramble about viewing the Social Graph as an active medium and not just dataset.)

Imagine you wanted to send an email to someone; you have a GMail account and they have a Hotmail account. Sadly, you can't; you have to sign up with Hotmail (or, perhaps, convince your friend to switch to GMail) because in this world, email providers don't talk to each other. To be able to talk with everyone, you'd have to have multiple accounts - then again, you may already, because each email provider bundles additional value-add services along with email. Of course it's not actually integrated with the email service, so much as it is bound to the identity that your email address represents, but it's still an integral part of one service competing with another.

You will likely have realised I'm drawing an analogy of the current ecosystem of monolithic social networks - you cannot send a Direct Message from Twitter to a friend on Facebook¹; the more astute (and elderly, and probably also American) among you, however, will realise that I'm describing something that actually happened, back in the early days of the public Internet (80s to early 90s). The early ISP-equivalents peddled proprietary email, forums and file transfer services that only functioned within their networks; my experience of this was with CompuServe in '94, though by this point the ISPs were integrating more and it was only a couple of years before Internet access was opened up fully - I think I got my first Hotmail address in '96/97.

There are various flaws to this approach, some more obvious than others. An immediate issue is the network effect - if all my friends are on service A, I'm likely to join that service; but if a new service starts up with an amazing new feature, I need to drag all my friends across to make it worthwhile. And, as Google+ is discovering, the nature of the Social Graph as a dataflow network means that having friends on a network isn't enough - there has to be content being generated and flowing through the graph to make it worthwhile.
I've always been a believer² in the free market and would argue that customer lock-in stifles innovation & choice, while choice empowers users and enables them to fight back against abuses of power; and when we're talking about services that facilitate the flow of information, I would argue that partitioning dataflow networks can have as great an impact on the development of society as putting up physical barriers³.

But there are other, more integral factors. The issue of security is well-explained in this XKCD comic; essentially, the more services I sign up to the more risk there is of my identity being compromised. The solution to this is already in place - Facebook, Twitter and other services are allowing integration through Facebook Connect, OpenAuth & OpenID that opens their identity service to other service providers, resulting in an improved user experience (one-click "sign-ups"), improved security (as your username & password is only held by one service) & reduced fragmentation (as your identity on different services remains constant across all). This openness empowers and enables users to switch freely between identity providers without impacting their experience of a given service; just as I was able to switch between BT & VirginMedia as my internet providers without my access to the Internet changing (beyond, of course, an increase in speed on a cable connection - the distinction here being between quality of service & nature of service), I was able to switch from an LJ login to a GMail login when using StackOverflow, and my experience of the service remained the same (note that here we must distinguish between my identity as an individual (which is what StackOverflow cares about) and the identities presented by LJ & GMail distinctly, which function more as aspects or façades).

Indeed, the question of choice is central to the problem with modern social networks. If you don't like Facebook's handling of private data or their use of advertising to monetise their service, your choice is to stay or to leave; to have or have not. If you have concerns about Twitter's censorship policies or Google's data gathering, your choice is to have or have not. There are numerous issues, from constantly changing UIs to the ever-waiting Fail Whale to freedom of speech, that can only be tackled when users have the choice to have this or have that.

There is (and has been, slowly developing, over the past year or two) a movement to decentralise & distribute the functionality provided by the monolithic service providers. Consider this blog post on the move to federated social networks, the Diaspora* project (and on Wikipedia), a distributed, user-hosted social network, or YaCy, a decentralised search engine. It an important step, and with enough time (and a good UX) these projects should help us to dislodge the grip of monolithic Web 2.0 services; there is, however, a further step required in this process, and you can see it developing in the ecosystem of services around Twitter.

In addition to the core micro-blogging functionality of Twitter, references to other media are now possible through the emergence of dedicated value-add services (e.g. TwitPic, TwitVid, Deck.ly) or the filtering of existing media hosting (YouTube, web-comics, blogs, online news, etc.) through URL-shorteners⁴. Twitter clients are now integrating with a range of these to provide wider (and yet seamless) functionality to users, but without necessarily constraining users to a specific service. This quick snapshot of the TweetDeck options page shows this choice in action; there's two types of service that TweetDeck can consume, but the choice of which specific service provider is up to me. We're seeing two parallel developments in progress here:

• The ability for software (in this case a client, but potentially services) to consume other services⁵ on behalf of a user (and their associated identity)
• The ability for users to build & customise a conglomerate social networking service that provides the specific functionality that they need, whilst remaining completely compatible with everyone else's custom social network

The move to integrate services in this way is amply demonstrated by If This Then That, a Yahoo-Pipes-Lite that allows users to build active links between their services. IFFT uses a set of purpose-built adaptors for each type of interaction, but by using the standardised interfaces & APIs for content sharing & notifications that are in development by the federated social network projects, this can all be generalised and made freely interoperable.

---

Right, it's taken nearly two months of talking to people and rewriting to get this post this far, so I'll pause here and promise a third & final part with my thoughts on the ideal solution...well, it'll take less than two months, anyway.

---

[1] Indeed, a "friend" or a "subscribe" on Facebook is an entirely different and incompatible concept to a "follow" on Twitter.

[2] In the same sense that I believe free speech is generally beneficial if you're not a dick about it, as opposed to the way I believe in dinosaurs (they existed, but I doubt they'd be beneficial to the economy).

[3] Hmmm, that's probably worth a rant of its own. Adam Smith's Division of Digital Labour?

[4]The passing of data references (Pass/Call by Sharing) is a familiar concept to programmers, enabling developers to manipulate and exchange heavyweight objects (large data files like images & videos) using only lightweight references (URLs in this context).

[5]Oh, my goodness! Shut me down! Machines making machines. How perverse.

The Social Graph is a Series of Tubes

Okay, this one is a bit of a stream of consciousness, so bear with me.

So I was reading this, a rant on why the Social Graph is something of a misnomer, and it stirred up some thoughts that I've been mulling for the past year or so. Note that I don't actually agree with several things the article had to say, but that's a post for another time; instead I'll respond to some specific points that were made and the train of thought those kicked off.

"In order to model something as a graph, you have to have a clear definition of what its nodes and edges represent. In most social sites, this does not pose a problem. The nodes are users, while edges means something like 'accepted a connection request from', or 'followed', or 'exchanged email with', depending on where you are.

The way you interpret this is another matter - does clicking 'follow' imply you're friends with someone in real life?"

There's a few points in the article where the author questions the nature of the arcs or links in the social graph, and what those represent - sadly missing the equally complex discussion of what a node represents, and how this relates to issues of personal identity online and the many façades we use - and while the discussion oversimplifies somewhat, the point is generally sound: the links between people cannot be simplified to or interpreted as Is Friend/Is Not Friend (or some variant thereon) and they are intrinsically dynamic (indeed, it is a lack of activity that can cause the greatest change in interpersonal relationships). A point that the author skirts but fails to deliver specifically is that the Social Graph as espoused by, for example, Facebook, deliberately (and vocally) misinterprets the links that its users create and destroy. When I "friend" someone, it's because I want to see the information they're sharing; I want to follow them. When I "unfriend" someone, it's not usually because I'm no longer friends with them - there's plenty of people linked to my Facebook account with whom I have no interaction at all - it's because I'm tired of listening to their shit. The status updates you've been posting for the past few weeks are actually really offensive, and I don't want to hear them any more; it's just a difference of opinion, and I'm not going to stop being your "real life" friend because we disagree on something, but I don't want to subscribe to your thoughts on the matter. You only use Twitter to play Echo Bazaar; there's nothing wrong with that and we're still friends, but there's no signal and all noise in your feed. We've broken up, and whilst we've resolved to stay friends, I don't especially want to see the pictures of your annoyingly handsome new boyfriend.

Changing the links between myself and the people I "follow", "friend" or otherwise is, to me, not a matter of keeping my data up-to-date in the same way as changing my relationship status, my address or my profile picture, it's about changing my experience of the service. I'm not expressing the status quo, I'm not describing the actual Social Graph (which is definitely an actual thing, even if we've not found it yet) I'm building a dataflow network. Google+ has started to grasp this with the Circles mechanism, allowing me to distinguish between the people I trust to see my information (friends), the people with something interesting to say (followees), and the overlaps between them. But I contest the article's implicit assertion that the links in the Social Graph (as present in social networks) are nouns, "friend of", "lover of", "enemy of", "employer of"; instead, I say they are actions, verbs. My links say things like "trust", "follow", "ignore", "inform". I'm telling the social network to do something, I'm building a set of pipes through which I can broadcast, send, receive & transform data.

Maybe what I'm trying to say is that the Social Graph is not the data, it's the medium. A friendship, a relationship, an emnity, these are not tags or metadata, they're processes, they're impellers, they're agents of change and movement.
Is that the difference between Amazon Recommendations/Google Ad-Targeting and viral marketing? Because the former uses the Social Graph as data, churns it through an engine and tries to drive itself along those lines, while the latter lets the natural forces of human society carry the message?

"One big sticking point is privacy. Do I really want to find out that my pastor and I share the same dominatrix? If not, then who is going to be in charge of maintaining all the access control lists for every node and edge so that some information is not shared? You can either have a decentralized, communally owned social graph (like Fitzpatrick envisioned) or good privacy controls, but not the two together."

Yes. And also, no. But mainly, sort of.

The present system of social networks lumps all the data in one, big central pile; there's a service out there, somewhere, on the net to which you send all your private details. This service is also the identity authority; you get a username and password to identify yourself, but it's up to Facebook, Twitter or whoever to decide if you are who you say you are. As an identity, you then ask the central service to allow or deny other identities access to your data. It's an entire world in a box, massively centralised and under the absolute authoritarian control of the service. This means the privacy policy of that particular service can be absolutely enforced, but it does mean sending all your private data to Mark Zuckerberg.

I'm loathe to comment overmuch about the author's apparent assumptions around privacy and control in the Semantic Web alternative; I need to read the essay by Fitzpatrick in more detail (though at first glance it seems to explicitly leave the topic of private data as an known requirement to be resolved in future) and the article provides no justification for the statements quoted above. But I do believe that a better standard of privacy and control can be achieved if the responsibility for managing data and identities is transferred to (and made practical for) users. I still believe that as Web2.0 features user-generated content, Web3.0 will feature user-published content; just as content generation has decentralised, so will content publishing; that we will all be our own YouTubes, Facebooks et cetera.

Okay, I've got to go do something social, so I'll be back in a bit to finish of this post with the actual endpoint of my train of thought; but in the meantime, here's a list of the all functionality I could think of that a social network would need to provide:

• A Feed: I want to share a stream of short statuses/tweets, links to articles, videos, et cetera. I also want to reshare things that have been shared with me.
• Content hosting: I've generated some of the content that I'm sharing, and I need to host it. Further, the act of sharing content is content in and of itself.
• Data sharing: I want to share information about myself. That may be my age, address, relationship status, or my dating proclivities, or where I am in the world, or my CV, or whatever. In a sense, this is just another form of content, but it's a specific format of content to which social networks cater.
• Identity: I need my content to be identifiable and attributable as mine. If someone re-shares my content, I want my identity to stay attached to it. Note that identity does not implicitly equate to me as a person; I may have several online identities, or an identity may belong to several people (like a company or brand).
• Privacy: I need to control who gets to see what content, and - so far as possible - the period of time for which they have that access. My friends get to see all my personal data, but a potential employer can see my contact details and CV for a few weeks after I submit a job application. That control should not be affected by the channels on which I share content; especially, it should not be affected by who chooses to reshare it.

Anything I've missed?

What I did with my Weekend (a one-man Game Jam retrospective)

Having missed yet another game jam (TIGJam UK 5, this time) by being at a LARP event, and suddenly finding myself with an unexpectedly free weekend, I decided to have a gamejam of my own (because that's just how cool I am). I'd been mulling over some ideas for a Dwarf Fortress-meets-The Sims-meets-Day of the Triffids game - a sort of Survival Strategy - since watching the 2009 remake of Triffids, so I sat myself down on Friday night and started coding - and here's what I learned from the experience:

Writing Design Notes

It's kind of obvious that writing a design is useful, but the hard part (I've always found) is knowing what to write down. When I have a code architecture in mind, it's tricky to try and work out which bits are obvious and which bits I'll forget and wish I'd written down. Equally, it's often as quick for me to write the code framework, and then use the code as the documentation; which is fine for me, because I've more practice of reading and writing code than reading and writing English, but other people less so.

Over the course of the weekend, I was making the whole thing up as I went; there was no grand design in mind, I just wrote down a list of features and tackled them one at a time. That level of focus (and the particular focus on How Do I Do This Quickly over How Do I Mix All These Into One Unified Symphony Of Code) meant those individual notes flowed more organically into a fleshed-out design, even if that design was just a tree of This Feature needs ==> This Features needs ==> This Feature; and probably This Feature, too. It was also more obvious which design decisions needed to be recorded for future reference when the code was less structured and self-describing.

Coder's Block

Realisation of the weekend: The more you know about something, the harder it is to code it. I know all about AI & Pathfinding - it was the topic of my degree - but somehow the language in which I understand those academic topics is very different from the language in which I express code. Equally, when I'm writing game code (I've hit this block a few times before) my mind jumps ahead, because I already know the (implementation) answer to the questions I've not quite reached yet; but that means that rather than smoothly grow my code base, I'm stuck trying to work out how to plug together the code I've written, and the code I know I'm going to have to write in the near future.

So I stopped thinking ahead. The emphasis on "just get it working, it doesn't have to be pretty" finally pushed me through the block, as I stopped thinking in terms of a grander design and just focused on the bit of code right in front of me. Of course, just getting it to work means that the pathfinding code I have now is nothing like as efficient as it could be, but it works, and it'll be easier now to go back and polish it up, now that I've tied it in to the rest of the code.

Take better care of yourself

Staying up too late on Friday night (by which I mean 8am Saturday) threw my whole sleep cycle off; I wasn't properly focused on Saturday/Sunday & would estimate I hit a 50% velocity for the whole weekend, when I should really have managed 70-80%. What I got right was my eating habits, for once; plenty of caffeine shots to keep me alert, glucose drinks and "energy" snacks (mmm pistachios) to replenish my blood sugar (continuous mental activity can really drop your blood sugar levels) and only eat "heavy" foods with complex carbs at main meals (I'll confess I cheated and ordered in pizzas, but hey, time saved cooking, right?) because the effort of digestion makes you sluggish.

Prep your materials in advance

Ye gods, the time I lost searching for isometric sprites. I'm not an artist by any stretch of the imagination, so all the art I used is downloaded from very generous people on the 'net; but there's a lot of dross out there, and to make things worse, many of the sites that do publish free sprites are horrific to navigate and search. I was really lucky to find OpenGameArt, from where I got most of the artwork that I used in the game, but ultimately the time I spent searching for and prepping sprites (especially given my lack of skill or experience) was time I couldn't spend building features. So next time, I'm working out a shopping list of the artwork I'm likely to need and getting it together before I start.

Play games before you write games

The first thing I did was to sit down with a notebook and write out a few key phrases and ideas about the game. The second thing I did was to think of all the games that those reminded me of and see if I could have a quick playthough of each; which meant that the first hour or so of Friday evening was spent playing X-COM & The Sims, re-learning the interfaces and "vocab" of an isometric UI. I guess this point (for me) is more about user experience and interface; I don't have anything new to bring to the table in those fields, so I'd rather point to previous games and say "They got it right, so I followed their example." and focus on gameplay, which is what I'm trying to improve.

Of course, having talked about all this I should mention what I actually achieved over the weekend, which is admittedly not a great deal. The game idea I had was quite grand for two-day sprint, so the result is more of a simple tech demo than a playable game, but I've uploaded it for now and will see about turning into something actually fun at the next jam; or maybe at the next CB2 Indies meet (there's one tomorrow, actually, will see about heading along).

Go here to get the current download; when you run it up, you'll see the example map with a single survivor. There is code in place for ambling/hunting/killing zombies (hence the red health bar about the survivor's head) but I need to wire up a button or something to spawn one on the map. A task for tomorrow, I guess.

Anyhow, middle-click somewhere on the map to refocus the camera on that spot, and the scroll wheel moves the camera up and down levels - note the basement under the house.
Left click the survivor to select them, then right-click to tell them to move; sorry about the lack of animations...
With the survivor selected, click the wall button in the top-left to select the Build Wall tool; then right-click on two locations on the map (two clicks, not click & drag) (also, two locations in a straight line) to tell the survivor to start building a new wall. It is a slow action, so don't worry if they seem to move to the site of the new wall and stare blankly at it for a few seconds - they're working, honest!

I'm fairly pleased to have got all that working in a single weekend; it's not exactly fun, but it's broken the back of building a simple Tower Defense vs Zombies game, and from there is just a matter of adding ever more awesome.

Epilogue install

Prereqs

The game is built on v4 of the .NET Framework (Client Profile) & the XNA Framework - that makes it Windows-only, I'm afraid (XP SP3 or later).

Download & Install

You can download the game from here. There's no installer, so just run the Expilogue.exe file in situ.

Core MSIL Assembler

This post contains instructions for downloading/installing the current prototype of the Core MSIL Assembler, and will be updated as new versions are published; explanations of what the assembler is and what it does will appear in later posts.

Prereqs

The assembler is built on version 4 of the .NET Framework. This is automatically installed by Windows Update on Windows 7 & Vista, and is an optional update for Windows XP; you can also download it here.

Download & Install

You can download the assembler here. There's currently no installation step, just unzip and run the CoreAssembler.exe app where it is by dragging a xxx.core file onto it. There's also a CompileAll.bat file that you can run to automatically compile all the example code.

AxumMUD

Just a quick note to say that the code for AxumMUD is online at CodePlex. The network layer is basically complete, though I'll likely go back and refactor when I've got some time, but I'm currently working on the Character/Location design and event communication; hopefully I'll get that checked in later in the week.