Geek Smithology

February 28, 2009

It’s ALIIIIIVE!!1!

Filed under: Announce,Craft of Dev,Ruby by Nathan @ 11:34 pm

chess Shortly after I left Service Intelligence all those years ago, a small panel convened at a Tim Horton’s to discuss the future. Well, actually, it was me, The Ront and Foo and since we seemed to like working together we figured we’d write a game of some kind. After not a whole lot of thought we figured we’d go ahead and write a chess game in Java. We didn’t really get anywhere with that, though. Then it happened.

I remember it like it was yesterday. But really, it was right after No Fluff Just Stuff 2005. My eyes were opened and it was decided right then and there that the chess game would written in Ruby. Sure it’s not the most “performant” language for chess calculations, but I wanted to learn Ruby and it’s always better to have a real project to work toward.

We decided to call it “pawnzilla”, for all the reasons you can imagine someone might want to call a chess engine pawnzilla. It’s been a bit of a roller coaster – we went through a horrible OO implementation, transition to 64-bit love and bitboards, and a fairly recent switch from sourceforge to GitHub, but after three and a half years (maybe a couple person months total effort – it’s tough to get time to work on this stuff!) I finally realized a dream, and played a game against my own chess engine.

It was pretty anticlimactic. All of the time so far has been spent working on a complete rules engine (this turns out to be quite a difficult proposition!), so the AI is pretty stupid. So stupid that the game lasted only 7 moves. But still, after all this time, it was great to watch pawnzilla make moves and actually “understand” that the game was over.

I think that we’ve implemented some pretty cool ideas in pawnzilla (including our domain specific approach to testing) and now that there is finally something to see, I hope to start blogging about some of those ideas. The next big goal is to develop the AI into something that doesn’t suck and hook it up to the free internet chess server so that it can get rated. So far it’s been a hell of a ride, and even though it took so long, it’s nice to get to such a significant milestone in my own labour of love.

If you’d like to see it, pawnzilla is licensed under the Apache 2.0 license and you can check it out over on github. And if anyone out there wants to play with a pure-ruby chess engine (should be a lot of fun just to work on different AI strategies now that the rules engine is in place) drop me a line and I’ll be happy to talk about it.

So cheers to pawnzilla.

February 1, 2009

This is your project on Git…

Filed under: Craft of Dev,Ruby by Nathan @ 3:36 pm

There’s been a lot of noise ’bout distributed source control the last year or two, especially with regard to Linus’ other baby, Git. Even I’ve bit – moving pawnzilla from sourceforge to GitHub last year, with serious thought to going with gitsvn in my professional life.

Stuart over at Relevence even went so far as to say that If you run a significant open source project that is not on a distributed SCM, it is a clear warning sign that you are a dinosaur.

He points to a pretty brilliant data visualization of commits to the rails project over the years. Take a look around April 2008, when they moved to Git. Make sure to watch the HD version fullscreen, it’s truly something to behold.

As always, our friends the pragmatic programmers have a book about Git to get you started on the road to enlightenment. (Or distributed SCM, as if they aren’t the same thing…)

February 27, 2008

Domain Specific Adventures in Real Life

Filed under: Craft of Dev,Ruby by Nathan @ 6:49 pm

There is a lot of buzz about Domain Specific Languages and about ten thousand pages describing what they are, so I’m just going to demonstrate a Domain Specific language that I am personally using. It’s for testing our chess engine, pawnzilla.

Probably the simplest thing one can do in chess is move a piece. Here is a sample unit test from earlier in the project:

1
2
3
4
5
6
7
8
9
def test_should_move_piece
    state = GameState.new
    state.clear
    state.place_piece(Coord.new(0, 0), Chess::Piece.new(Chess::Colour::WHITE, Chess::Piece::PAWN))
    state.move_piece(Coord.new(0, 0), Coord.new(0, 1))
    assert_nil(state.sq_at(Coord.new(0, 0).piece)
    assert_not_nil(state.sq_at(Coord.new(0, 1).piece)
    assert_equal(Chess::Piece.new(Chess::Colour::WHITE, Chess::Piece::PAWN)
end

Right now you’re probably thinking “DAMN, that’s a lot of code, and I can barely read it!” And you’d be right. Here’s what the exact same test looks like after some massive rethinking:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def test_should_move_piece
    state = GameState.new
    state.place_pieces("
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        p - - - - - - -
    "
)
    state.move_piece(A1, A2)
    assert_position(state, "
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        p - - - - - - -
        - - - - - - - -
    "
)
end

Now, dear reader, assuming you know even the tiniest little bit about chess, is there any way you can’t understand this test? This relies on two primitives, each very important in their own way. First, the chessboard itself. Chess players will be familiar with FEN, or Forsythe Edwards Notation, which is used to describe chess positions. Now, FEN is less verbose than our chess board object, but we take the idea of piece representation – white pieces are lower case, black pieces are upper case. This makes it trivial to present any conceivable chess position in the same amount of space, and anybody looking at a test knows exactly what it’s trying to accomplish. The second important thing to note is that in the move_piece method, we’re using simple constants that represent the coordinates of all 64 squares in the corresponding algebraic notation. While (0, 0) (0, 1) is convenient for the engine, it’s very tough for a chess playing human – you end up spending several seconds mentally transposing the cartesian coordinates into algebraic notation. Why not just remove the barriers?

Tests that may have previously been a nightmare to read and figure out become dead simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def test_should_detect_blockable_back_rank_checkmate
    state = GameState.new
    state.place_pieces("
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - - - - - - - -
        - b - - - - - -
        - - - - - - - -
        - - - - - p p p
        R - - - - - k -
    "
)
    assert(!state.checkmate?(BLACK))
end

This idea can extend to other tests as well. For example, let’s define a square under attack using an asterisk(*). Then testing attack vector calculations also becomes an easy visual test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def test_should_calcluate_queen_attack
    state = GameState.new
    state.place_pieces("
        - - - - - - - -
        - - - - - P - -
        - - - - - - - -
        - - - Q - - - -
        - - - - - - - -
        - - - - - p - -
        - - - - - - - -
        - - - - - - - -
    "
)
    state.calculate_queen_attack(BLACK)
    assert_attack_state(state, "
        * - - * - - - -
        - * - * - P - -
        - - * * * - - -
        * * * Q * * * *
        - - * * * - - -
        - * - * - p - -
        * - - * - - - -
        - - - * - - - -
    "
)
end

Now you can really see the power! Can you imagine trying to make sense of a test that had to check all of that board real estate using coordinates?

This demonstration is the quickest way I know how to define a domain specific language. We take concepts directly from our domain, the game of chess, and create primitives that translate directly to real world concepts. Because of its lovely meta-programming facilities, it’s very easy to make Ruby use our DAL by adding our code to the Unit test module. It is important to note that we are not extending the unit test module, but adding behavior to it, as if our domain concepts are native. It’s quite simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
require "geometry"
module Test::Unit
    class TestCase
        A1 = Coord.new(0, 0)
        # etc.
        H8 = Coord.new(7, 7)

        def assert_position(state, position)
            # code
        end

        # and so on
end

Now all of our tests can call upon an incredibly rich toolset that brings us to a holy grail that I previously thought was unattainable: Code that is actually easier to read than it is to write. That it’s not hard to write either is just icing on the cake. You just create a macro that drops an empty grid into the code, place your pieces, and away you go.

One of the most interesting things about this is that from a technical standpoint, this was a very easy thing to implement – Ron did the translation stuff and converted several test in only a few hours. The important breakthrough here, like when Google unveiled their map website, is NOT that it requires crazy voodoo programming, but that it requires a different way of applying what’s right in front of us.

September 18, 2006

The State of Enterprise Ruby

Filed under: Craft of Dev,Ruby by Nathan @ 8:48 pm

I have been to the mountain! Having spent three days in the cloister, intensely meditating the Zen at the core of the Enterprise Ruby scene, I have clearer insight into the current state of affairs. Using the simplest terms, in the most convenient definitions, here are the enterprise concerns we attempted to allay…

Scalability

When contemplating the appropriate context for scalability, let me remind you that the word “big” is both too subjective and too narrow. Think arbitrarily sizable without significant development overhead. Ruby is good at this. So is Java. So is Perl. So is any system with the appropriate architecture. I worked on a system with tens of thousands of users (several hundred simultaneous) that required no client server stickiness whatsoever – merely a round-robin load-balanced cluster of JBoss dragoons. If the load got heavy, we’d just add more heat, and the system would chew through simultaneous connection like Sauron’s hammer. Ignoring all other aspects of this system, Ruby could certainly have handled that same load. ‘nuff said.

Flexibility

Ruby is a smorgasbord of tools that increase your flexibility: dynamic typing, blocks, closures, mixins, and that’s just the list of things that are easy to understand. You simply cannot beat Ruby for flexibility, and the maxim “Thou shalt trade CPU cycles for developer cycles” makes its most sense in this area. Not quite a year ago, during dinner with Dave Thomas, he mentioned that people were doing enterprise development with Ruby, using it as glue between big iron systems. This I get. You want a quick cron to check a property in LDAP and then use a Web Service to drop a message into a queue? Save yourself some hell and use Ruby. Have a lot of legacy code in another scripting language (like, say, Perl…not that I have any experience with that or anything) and you need to start talking to MOM components, the only thing sillier than not using Ruby would be to put nipples on the Batman suit. And you know that worked out.

Reliability

Here, we’re thinking enterprise reliability (like, 1 + 1 always equals 2, and if it doesn’t, your software is the last thing you’ll care about), not script reliability (like “M. Night Shyamalan will always make good movies, and if he makes the Village, well let’s just not bother with Lady in the Water.”) A good example is Rinda. You can share distributed tuplespaces until the cows come home[1], or until the machine crashes, whichever comes first. I for one don’t want to restrict the lifetime of my messaging infrastructure to the MTBF of that Dell server my client bought because it was “cost effective.” You want guaranteed delivery? You want durable messaging? Then consult your friendly neighborhood JMS implementation (there are some good open source MQ systems) and let Ruby talk to it.

If your server infrastructure is running Ruby, be aware of the performance (or lack thereof.) It doesn’t have native threads and isn’t compiled. I know, I know, 37 signals is running a gazillion hits through Rails, but if you’re doing heavy back end processing (say, image processing or time critical data analysis for reporting) you’ll want to trust something a little more specialized, compiled, and up to the task. However, keep in mind the words of Dave Thomas:

I wouldn’t do stuff that required high performance in Ruby, but I wouldn’t do it in Java either.

There was another reliability hit: it was harder to get some piece of the puzzle working than getting Paris Hilton and Nicole Richie speaking again. LDAP took a Conan-at-the-wheel-of-pain level of effort to get running under Windows (and we gave up on the pipe dream of actually building it and just Googled some pre-built dlls), and we never did get the JRuby web server to work.

Finally, damning as all the rest, there are no distributed transactions. You need to make sure you grab something from the queue, write to the database, and drop a new message in a single transaction? Ruby won’t do that. So while this stuff is coming along for internal tools and admin scripting, you can’t trust mission critical back ends to Ruby.

IMPORTANT NOTE:

One thing I have to make crystal clear is that I am writing this for people who are wondering about Ruby’s suitability as an enterprise platform. This is NOT a rebuttal to the Ruby community, because they are so damn pragmatic, they wouldn’t endorse Ruby as a back end platform. I cannot stress this enough: Stuart and Justin never sold Ruby as anything other than a very convenient tool. Nobody is saying it’s a replacement for Java, just a good companion. Justin has a very good blog post about this exact subject.

Security

There was not much said about security, because Ruby doesn’t have a lot of support that you don’t code yourself. Concepts from the .NET and Java worlds like Declarative Role Based security or Authenticated Principle objects just don’t exist. I’m not saying that those languages have the problem licked, but I am saying that they make it easier to track authenticated users (at a thread level in the case of .NET). That’s a big deal when we’re having the debate about how best to spend developer cycles.

Internationalization

Ruby ostensibly supports Unicode, but most of its libraries do not. And unlike Java and .NET, there is no support for locales (cultures, for the .NET folk) and bundled resources. Of course, in Java and .NET, most developers don’t pay any attention to character encoding and end up painting themselves into the exact same corners. But as someone who was attempting to run a dynamically multilingual website, this is a tough pill to swallow. Oh, and I have to say this: extended ASCII is NOT internationalization.

In Closing…

So there it is. I know I’ve painted a grim picture, but that is only because I was earnestly hoping that Ruby was more mature. I am a staunch support of Ruby and genuinely love programming in the language. I’m buoyed by the progress made so far, especially compared to other scripting languages. And one can’t help but be optimistic that the rich communities of great minds doing significant work in Ruby will eventually elevate it to a higher plane.

The real takeaway is that if you truly believe in using the best tool for the job, then you will be using Ruby at some point in the future.

[1] As a resident of Calgary, dear reader, there are two things I can promise: The postman always rings twice, and the cows always come home

September 12, 2006

Enterprise Ruby Pragmatic Studio, Day 2

Filed under: Ruby by Nathan @ 9:45 pm

(Go check out Day 1)

I know what you’re thinking – why wasn’t the day 2 report up on day 2 (for those of you reading this in the future, I used my 1337 h4X0r sK1LLz to make this post appear to have been posted on the 12th, but that wasn’t the case.) Well, it turns out that even though I thought 2 hours was no big deal in the vast sea of jet lag, I was wrong in ways that only a 32-year-old with a severe case of temporal hubris can be. After typing out the Day 1 missive, I proceeded to not sleep. So after day 2, I didn’t do much else. ’nuff said.

Day 2 expanded on Rinda, our end of Day 1 topic. We explored the world of the Rinda ring server, which let’s you forage into the ether looking for tuple sockets to which you may attach (while that sounds vaguely lewd, I assure you that there is no fetish of which I am aware that could make Rinda erotic.) I’ll avoid the obvious Tolkien and less obvious Niven puns here – goodness knows we didn’t at the studio. Far from making you all powerful (and invisible, like Jessica Alba) these rings function like nameservers, letting you find shared tuple spaces by name. I am so attracted to this in theory, but I just can’t think of a practical use (where I wouldn’t desire a more robust technology, that is.)

Next we played in the wonderful world of HTTP. We wrote a web server in a few lines of code (which blew my mind when I first saw it done in Perl several years ago, but appears to have become de rigeur if you want anyone to take your scripting language seriously) using WEBrick. There were a few jabs at Tomcat, but to suggest[1] that WEBrick has even a small fraction of the utility of Tomcat is beyond even the most devout Rubyist (I wouldn’t put it past Perl Hackers, tho’) Then we played with HTTP::Client, which let us talk over the web using Ruby. More of an “of course” than a “ooooh, sweet!”

LDAP. The technology we all love to hate. Well, I don’t hate the technology, I just hate the specification. cn? dn? oid? OY! The good: there are some really cool tools for helping you work with LDAP, along the same lines as our old friend ActiveRecord from the rails world. The bad: You need a rather specifically designed schema for it to work, along the same lines as our old friend ActiveRecord from the rails world. The ugly: If you have a legacy system, you probably do not have the right schema, along the same lines as our old friend ActiveRecord from the rails world. Getting this lab to run under windows was quite a pain, which is disturbing – if Ruby wants to be a serious contender to Java, its tools gotta work somewhere other than *nix.

SOA. Love or hate the buzzword, we’ve all gotta deal with services. And I’ll bet you a tidy sum that at least some of those services will be on the web. And in a world that created the specification for LDAP, I’m shocked to learn that these services running over the web are called web services. Support for producing and consuming web services between Ruby clients is dead simple, very clean, and it just works. In a world of ever mutating WS-* standards and the wonderful world of SOAP, Stuart and Justin admitted that there would be a lot of manual work involved to keep up. The bright side is that it will only be a couple lines of code. That said, Ruby does the things it’s supposed to, like auto generate your WSDL and make it incredibly easy to play with the headers. If you have some scripts that need to make use of web services, this would definitely be quicker with SOAP4r than equivalent apis in Java or .NET, but I wouldn’t do a back end or mission critical pipeline service with it yet (and that’s not even worrying about transactions, but that’s for later)

And that was Day 2. One of these things I love about these conferences is getting to talk to other developers (and developers “in the know” – the average calibre of people doing Ruby, finding out about this conference, and getting out to Boston is pretty high). In addition to Stuart and Justin, Mike Clark is here and I had lunch with him today. Good talks about Rails and hearing what other people out there are doing with Ruby in general.

[1] At this moment, the power went out. Thank goodness for laptops and their battery power. Boston (well, the wee part of Boston where I am) is dark.

September 11, 2006

Enterprise Ruby Pragmatic Studio, Day 1

Filed under: Ruby by Nathan @ 9:46 pm

In a crusade to sniff some of the glue that never sets, I’ve come to Boston for the latest Pragmatic Studio, the one playfully and provocatively named Enterprise Ruby. As I’ve mentioned in the past, there’s meaningful enterprise[1] application development, and then there’s vendor bullshit. Our hosts, Justin Gehtland and Stuart Halloway, are poised to take us on a three day roller coaster ride that (for the love of $DEITY) does not involve Rails.

I got to the room where the studio is being held, plugged in my stuff, and realized I am the lone Dell in a sea of Apples and “PC Laptops that are not Dells“. People instinctively moved away, scared that it might explode at any second. The class got off to the traditional inquiry for a count of hands from folks currently being paid to sling Ruby. I’m fairly certain at least a few folks were plotting the death of the one man who almost apologetically held up his hand.

I decided to partake of the “continental” breakfast. Apparently “continental” is short for “quaffable but unremarkable coffee, cantaloupe, and an array of pastries.” You shouldn’t be allowed to use the c-word to describe your breakfast if there are no croissants. How can I save up enough buttery French heart attack bombs to build a Chinese Throwing Croissant if I can’t even find one?

The session started out usefully, but slowly, introducing us all to the Tiddly Wiki. To be fair, it’s an extremely cool little product – an entire wiki in a single page (a single page with a crazy amount of javascript). That’s where all of the course material is held. I will be using it in the future, it’s that cool.

Next we went onto rake, which was depressing. Depressing like a fox. You know all those times you’ve gotten mad at make, nmake, ant, nant, or because they were missing just one little thing? Well, you can’t get mad at rake, because it’s not text or xml or happy-fun-dsl, but ruby. I remember having to sign a whole boatload (well, it would’ve been a small boatload, but about 14 wouldn’t have made much of an impact, no?) of jars for a java applet and thinking to myself Self, wouldn’t it be nice if we could write a loop here?

Rake was followed by some quality time with flexmock. It reminded me somewhat of Rhino Mocks for .NET (I can’t think of anything remotely similar in Java, but I’ve been away for a half year), but quintessentially Rubenesque. Er, I mean Rubyesque. Check this out:

1
2
mock = flexmock("foo")
mock.should_receive(:bar).and_return("baz").at_least.once

You don’t need to know anything about this to figure that out. That said, C# isn’t too far behind…

1
2
IFoo mock = mockRepository.CreateMock<ifoo>();
Expect.Call(mock.bar()).Return("baz").Repeat.Times(1, int.MaxValue);

Anyway…we put all this into a nice little directory so that we can use it as we go forth (goal is to unit test everything.)

Next was DRb – distributed Ruby. If you’re thinking about CORBA or RMI, you’re on the right path, except it’s Ruby, so it’s easy. Not entirely sure where I’d use this in the real world, but it’s there in the core of Ruby. It’s got some sophisticated methods for creating observers and callbacks, and there are ways to synchronize several machines to do a task, so who knows. During the lab, Justin made the comment We just had an entire room of people, some of who don’t even know Ruby, write a client-server in 10 minutes. Okay, this was pretty cool.

The day ended with an introduction to Rinda. To put this simply, it allows for a distributed Ruby system to share a collection of Tuples. To put it anything other than simply would make your head explode. Well, my head at least. I’m familiar with distribution, and I’m familiar with Tuples, but this was a very interesing look at things. Rinda is based on Linda, a coordination language. Once again, I’m not sure of the immediate implications, but in terms of a toolbox, Ruby seems to have a lot more going on than meets the eye.

I’m still annoyed that Ruby doesn’t have native threads (but they’re coming…), but both DRb and Rinda come with the core Ruby distribution, so there are some impressive tools coming out. All in all, it was a fun day and I’m looking forward to tomorrow.

[1] That’s right, enterprise doesn’t need to be capitalized when Captain Kirk isn’t in the picture.

Powered by WordPress