Playing Around: Games and scale

In which we consider a talk that shows a different, and perhaps more interesting, approach to ubiquitous computing...[Read More]

In which I emerge from the silence that was caused by the Austin Game Developer's conference, and talk a bit about changes in our mechanism for code reviews.[Read More]

In which we consider what it takes to get a good system design. There are no real answers here, but there is some description of how we in the core Project Darkstar team go about designing. You may find it useful, but your mileage may vary...[Read More]

In which we talk about the development aspect of code written as part of a research project, and argue that such code needs to be more reliable than that contained in a product. Along the way, we also describe what you (and all members of the Project Darkstar team) have to go through to get your code into the trunk...[Read More]

One of the interesting, and sometimes disquieting, things about doing an open source project is that you don't know if the work you are doing is being watched by anyone, or that you are doing the kinds of things that those who might be watching are interested in seeing done. It's sort of like blogging in that way. 

To try to get some sense of what, if anything, the community would like us to do (now, be nice), we (the Darkstar team) have put together a survey. It is short-- just five questions, mostly multiple choice. So if you have a chance, head on over and take the survey. We promise to at least read all of the responses, and it will help us to know what the community thinks of what we are doing... 

In which I discuss the research aspect of Project Darkstar, and why it is that we are not trying to get things out more quickly by reducing the risk in the development. In fact, we are actively trying to increase the risk of what we are doing (in a controlled way). That is how you learn things, which is the real goal behind the project.[Read More]

In which we discuss the fourth miracle that is needed to get to a scaling multi-node version of Project Darkstar: how to move tasks from one node in the server cluster to another. Part of this is easy; the rest is not. As usual, it is the distributed nature of the beast (along with concurrency) that makes this interesting...[Read More]

In which we discuss the third miracle needed for us to get to a scalable multi-node version of Project Darkstar. This miracle involves being able to tell how a node (aka game server) is doing, and then having some place in the system that can do something sensible about those that are not doing so well. This brings us to questions such as continuous vs. discrete measures, and whether or not it is ok for a set of machines to be working just fine but not accept any new players into the game...[Read More]

In which I discuss the second of four miracles that needs to be performed to get the Darkstar server to support multi-node scaling. This miracle has to do with making sure that the tasks that are manipulating data are located on the same game server as the (cached) data. Our approach may not work, but it is really interesting to think about, and it might just work after all...[Read More]

In which I try to explain one of the ways that we are trying to get Project Darkstar to scale well over multiple nodes. This particular miracle has to do with creating a data cache that trades off durability for low latency while still insuring consistency. If it works, we can finish up transactional tasks that alter data without making transfers over the network. The downside is that we might lose some (hopefully small number of) seconds of game play, but even in that case we end up coming back into a world that is consistent, just somewhat behind where we thought we were. [Read More]

Currently the Project Darkstar infrastructure allows spreading a game over multiple threads in a scalable fashion. While the infrastructure also allows spreading a game over multiple servers, the technology does not scale well when that is done. Our current research is centered on enabling scaling over multiple nodes. In this, the first of a series of posts about how we are approaching this problem, I talk a bit about our overall approach, and lay out the four interacting problems that we are trying to solve. I also promise to go into detail on each of the four pieces in the (near, I hope) future.[Read More]

It's been way too long since I last posted, but many of you will realize that times have been, um, rather odd around here. And I've been pretty busy, both with Darkstar and with a bunch of other things.[Read More]

In which I reflect on specifications, wire protocols, and irony, though perhaps not in that order.[Read More]

In which I ponder the meaning of a roadmap in an open source project, find that the production of such a roadmap is far less distasteful than it has been in the past, and announce our new roadmap, asking for others to join in the discussion.[Read More]

I've been spending the past couple of weeks returning to my roots by bringing the Project Darkstar C-language client up to the most recent Darkstar protocol and API (the one that we released as the 0.9.6 release). This became my job for the most part because I was the only member of the team not gainfully employed in doing work on the overall system performance. But it also became my job since I have probably had the most experience writing C code in the group.

In doing this, I've re-discovered what I liked about the C programming language. Even more, I re-discovered why I like the Java language even more.

I've always thought of programming languages as encapsulating models of what the computer is. BASIC makes the computer look like a calculator (historically, of course, the causal connection is the other way around). COBOL models the computer as a filing cabinet. FORTRAN models the computer as an array processor.

The C language models the computer as a PDP-11. The PDP-11, for those not old enough to know, was a great mini-computer made by DEC back in the day. It was cheap enough (by the standards of the day) that there were a lot of them around, reliable enough that you never felt like taking a baseball bat to one, and powerful enough (again, by the standards of the time) to do lots of interesting work. Although C was first written for the PDP-7 (an earlier DEC machine) it got a lot of use in the 11, where it was the system language of choice for Unix hackers.

I wrote a lot of C code early in my career (actually, I started out writing a lot of COBOL code-- a fact that I'm willing to admit, but not a time that I look back on fondly). Brian Kernighan and Dennis Ritchie have wonderful taste, and using the language was an education in itself in how to design something that was small, simple, elegant, and immensely useful. I don't know anyone who did a lot of programming in C that didn't have a lot of affection and respect for the language.

Returning to programming in C after a number of years (and the number has two digits, if we are talking about serious programming) I felt like the Tin Woodman in the movie version of the Wizard of Oz when Dorothy first finds him-- completely encrusted by rust. Every movement was somewhat painful, but once I started flexing a bit it all came back and it was remarkably fun. I found myself once again having the experience I often had, when C was my day-to-day language, of having my first version of a function taking 20 lines, and then cutting that in half, and then in half again, until the final version didn't have a whole lot extra in it. Quite frankly, it felt good and it was fun.

But it also made me appreciate the Java language all the more. I remembered what it was like before we had garbage collection (do I free here, or there-- need to do it at least once, but don't free anything twice). I remembered all of the mechanism that is needed to get the right code working on the right data structures. And while there are times when you have to do some nasty assertive forms of casting (I was reading byte streams from the network and reconstructing real data, after all) I was reminded of all of the errors that can occur when there are no checks.

Still, I did enjoy myself, and will probably do some more C coding in the near future (I've already thought of a couple of lines I can remove from the client). But it has also made me appreciate Java all the more, and wanting to get back to writing some code in that part of the system.