Recently in Programming Category

Scenes From an Interview

|

"Obviously the job requires some pretty serious math skills. As the first-line screener, my job is basically to do a sanity check and make sure that you meet the baseline we're looking for, so that we don't waste our time and your time with further interviews that aren't going anywhere. I see you've got an impressive resume and a lengthy career, so please don't be offended if my questions seem simple; it's not ajudgementon you, but a reflection of the goal here. Any questions? No? Good."

"So here's my first questions: What's one plus one?"

Loser answer number 1:

"Geez… I don't think anybody's asked me to add one and one since elementary school. Is that really something you do? I've been working in this field for twenty years now, and I don't think I've had to know one plus one even once… any competent professional has Excel, or a calculator, or somethingthat will take care of details like that for them. I'd go so far as to say that anybody who's manually adding single digit integers like that probably should be fired."

Loser answer number 2:

"Oh, I remember this one! I haven't actually done one plus one since kindergarten, but I studied up on the basics before this interview, and I remember that the answer is two. Here, let me write a two for you… It's kind of curvy around the top, I think, like this… no, more like… here we are, that's a two."

Loser answer number 3:

"Eleven. Yes, I'm sure. No, there are no problems with my answer. You put the two ones next to each other, and it's eleven. What the hell is wrong with you?"


The humor here's pretty obvious, right? No reasonable person misses the fundamental problem with these answers, I hope. Any yet if you shift the field to computer science, suddenly people think these sorts of shenanigans are reasonable.

Of course nobody would implement ELEMENTARY_DATA_STRUCTURE_1 by hand in this day and age; it's in the standard library on any platform worth dealing with (well… maybe not Cocoa…) and the folks writing the library probably spent a lot more time debugging and optimizing than you'll ever be able to devote. You don't get asked to implement these things because the interviewer's going to steal your code and check it in to source control.

And you also don't get asked because, gosh darn it, we totally need an idiot with an eidetic memory who can vomit code snippets he doesn't understand. We have Google for that! We're not testing your memory, but your ability to understand basic computer science principles and - gasp - write code! If you don't have the code paged into memory right now, from our perspective that's a HUGE WIN. We want to watch you re-derive things from fundamentals, not regurgitate all over Collabedit.

In the course of a recent quick JavaScript side-project — post forthcoming — I needed a simple cache. Then I needed another cache within the cached objects. Then I needed another cache. It was at that point that I realized I should probably factor things out into a reusable component, and built JSLRU.

Of course then I refactored things dramatically and it's barely used at all. But if you need a JavaScript least-recently-used cache, check it out.

Generic Accumulators in C#, Redux

|

It turns out one of the most-trafficked pages on this site is my discussion of generic accumulators in C#. It occurs to me that it could use a bit of an update, as some newer features like lambdas and the predefined Func<> family simplifies things quite a bit:

class Program
{
    public static Func<T, T> MakeAccumulator<T>(T start, Func<T, T, T> addFunction)
    {
        return inc => start = addFunction(start, inc);
    }

    static void Main(string[] args)
    {
        var intAccumulator = MakeAccumulator(0, (i, j) => i + j);

        Debug.Assert(0 == intAccumulator(0));
        Debug.Assert(1 == intAccumulator(1));
        Debug.Assert(11 == intAccumulator(10));
        Debug.Assert(55 == intAccumulator(44));

        var floatAccumulator = MakeAccumulator(0.0, (i, j) => i + j);

        Debug.Assert(0 == floatAccumulator(0.0));
        Debug.Assert(0.1 == floatAccumulator(0.1));
        Debug.Assert(1.1 == floatAccumulator(1.0));
        Debug.Assert(5.5 == floatAccumulator(4.4));

        var stringAccumulator = MakeAccumulator("", (i, j) => i + j);

        Debug.Assert("" == stringAccumulator(""));
        Debug.Assert("ZYZ" == stringAccumulator("ZYZ"));
        Debug.Assert("ZYZZY" == stringAccumulator("ZY"));
        Debug.Assert("ZYZZYVA" == stringAccumulator("VA"));

        Console.WriteLine("Success!");
        Console.ReadLine();
    }
}

So there's that. Still not terribly useful, but I do like shortening code.

Should've Bought an Xbox

|

I bought a Playstation 3 a few years back (when I got my HDTV, as I recall) primarily to use as a blu-ray player (at the time it was actually cheaper than a standalone player). As Sony doubtless expected, it entered my home as a shiny black Trojan horse, and I've bought a handful of games since then.

Including, fatefully one from the Playstation Network (link goes to a Flash-only site, because Sony's just in full-on pissing-me-off mode at the moment).

If you're reading this there's roughly a 99% probability that you already know that the PSN has been 'out' for nearly a week, and the latest news from Sony is that it ain't comin' back anytime soon, and oh, by the way, some hackers probably have your credit card numbers.

I know this the same way you do, through the magical collaborative power of the Internet. Not, say, from an email sent to me by Sony warning me that my credit card number had been stolen. Strike one, Sony. You assholes.

But as annoying — and probably actionable — as their failure to proactively warn their customers may be, that's not what really pisses me off.

No, what's really bugging me is that I can't use Netflix on my PS3.

Why? Because Netflix on the PS3 — despite being tied to my totally independent non-Sony Netflix account — requires a PSN login to function. For no reason at all except that Sony are a bunch of goddamned morons.

Thank God for the Apple TV.

Potemkin Unit Testing

|

After the Russian Empire annexed the Crimean peninsula in 1783, Tsarina Catherine II (motto: “I'm greeeeat!”) took a tour of the area, guided by her trusted advisor and alleged-favored-non-equine-paramour Gregory Potempkin (or if you're not into the whole Anglicization thing, Grigory Potyomkin... or if you're really not into Anglicization, Григо́рий Алекса́ндрович Потёмкин).

As the story goes, Catherine's inspection took the form of a leisurely cruise down the Dniper River. Potemkin, being aware that the Crimea was kind of a crummy area and fearful that the Tsarina would mislike the realization that her new territory kinda sucked, ordered the hasty construction of numerous fake village façades along the river bank. The illusion held; Catherine went home thinking the Crimea was a bustling hub of non-squalid not-mud, and our modern world gained a vivid new idiom: 'Potempkin Villages'.

Of course all the actual evidence is that the story is bunk — Potemkin may have instructed the peasants to clean up a bit and maybe paint some fences, but the idea that the whole area was an elaborate set was an extreme exaggeration spread by his political rivals. But it's one of those stories that's just too good to not tell, isn't it? It's 'truthy'.

Changing direction entirely: unit testing is the new hotness in software development. The basic idea is that you decompose your software into small, manageable units and create dedicated automated tests that verify the functionality of those units. And it is, in general, a really good idea.

The problem that crops up with unit tests is that some things are just really hard to test, particularly when your code interacts with something outside your control — such as a human user, a piece of hardware, or a third-party web service. In those cases you need to get tricky; you create new layers of abstraction that separate the interaction from the main work of your module, and then you create 'mock' version of those abstraction layers and substitute in a 'simulated user' for a real user, or a 'simulated serial port' for your hardware, or a 'simulated web service' for the third-party service.

In some cases that approach may make sense. For some pieces of hardware the range of possible states is limited; a button is up or down, and it's never both. If a user is given an extremely limited set of buttons to click, you can reasonably simulate every possible interaction model. But for anything more complex, you're kind of creating a Potemkin unit test. So you've extracted out an ITwitterApiProxy that your TwitterService talks to, and you can plug in your MockTwitterApiProxy and write a whole host of tests for your TwitterService. Every failure mode you can think of can be simulated by the MockTwitterApiProxy and you've coded the TwitterService to deal with all of them reasonably — and hurray, the unit tests mean you're safe!

Now you go to production, with the RealTwitterApiProxy, and stuff goes wrong. Twitter's failing in ways you never anticipated! So you add more switches and dials to your MockTwitterApiProxy, and add more code to your TwitterService to deal with them, and write tests to exercise them... and hurray, the unit tests mean you're safe! ...at least until Twitter fails in another new way you haven't seen.

Compare that to how you would have handled things without unit tests: You code up TwitterService, handle the error cases you anticipate, go to production and discover some more error cases, update TwitterService to handle them... lather, rinse, repeat. It's essentially the same workflow.

So what does the extra effort buy you? What's the benefit of building your API abstraction and unit tests?

Well the obvious answer is that it buys you protection against regression. You have an explicit test in place for handling error type A, so if in the course of implementing support for type G errors you accidentally break type A, the unit test will fail and you'll find out. There's that, certainly. Without unit tests it's basically down to trust: I trust myself (or my fellow developers) not to break this part of the code while working on that part of the code.

But on the other hand, your test for handling error type A isn't really testing your handling of error type A, it's testing your handling of your mock simulation of error type A. How can you be sure that in the course of adding code to simulate error G to your mock you didn't accidentally break the code that simulates error type A? Isn't that the same sort of "trust yourself" approach that the non-unit-testing approach is left to? Should you add unit tests for your mocks to ensure that they're properly simulating errors?

Ultimately a lot of this is down to personal judgement, like most of the interesting questions in software development. And I'm certainly not going to argue that unit tests aren't a good idea a lot of the time. But I do believe that there are certain situations where unit tests create a lot of extra code to maintain without really adding much in the way of protection, and that committing to building such Potemkin unit tests is a foolish consistency of precisely the sort that will prove a poor substitute for Norman Osborn.

Damned Whippersnappers

|

Remember being a teenager*?

Teenagerititude is the state of believing that everyone else in the world is phenomenally stupid — that the solution to every problem is blatantly obvious, and that everyone would be much better off if they'd just shut up and do as you say.

The interesting thing about being a teenager is that you don't actually outgrow it.

At some point a light goes on in your mind, the scales drop from your eyes, the metaphor similes upon you, whatever, and you realize you've been a teenager and you stop. You congratulate yourself on being so adult and on owning up to your past bad acts and you move on with your life.

And then a few years later you realize that you were mistaken, that you were actually still a very-slightly-more-evolved form of teenager despite that revelation, and the scales drop from your lights and you eye what you metaphor and you move on with your life.

Until it happens again.

Eventually you reach a degree of meta-awareness — you recognize that Socrates kinda had a point about the whole 'knowing you know nothing' schtick. That's when you ascend to a higher plane of existence! Then you help Teal'c and MacGuyver out a few times, and eventually you return to the show with your tail between your legs because it turns out your landlord won't accept 'art' in lieu of money. But hey, higher power. You got that going for you.

Until, damn it, it happens again.

Meta ain't enough, nor is meta-meta. Maybe there's some omega-meta state where you stop realizing that you're an idiot, and you get to draw a Batman logo on Anthro's cave wall just to reassure the idiots who didn't figure out the pattern based on Kal and Hal and whatnot. But I can't be sure of that. I think any form of personal growth boils down to suddenly recognizing what a jackass you've been and thus becoming an exciting new form of jackass.

...

So what's the moral of the story? The moral is that I'm stupid, you're stupid, he's stupid, she's stupid, and the primary differentiating point between us is our awareness of our own incompetence. So when something looks dumb to you, remember that it either is dumb, or it's smart in a way you haven't thought yet deciphered — and prudence dictates that you assume the latter until you've assembled reasonable support for the former. And even if something turns out to be legitimately dumb, don't draw conclusions from that, because oftentimes there's more to the story. It's not uncommon to find clever implementations of a piss-poor design; that may be a sign of abstractional schizophrenia, or it may be a sign of the blind leading the brilliant.


*If you currently are a teenager, shut up and get the hell off my lawn. Damned kids.

Ask each of your developers who the three best developers in the company are. Only ask the developers — managers can't answer because they're in no good position to know. Average out the votes and produce a consensus 'top three' list. Hold on to it, and repeat the survey every six month.

If over a year has passed and two of those three people are still on the list, your company is probably screwed. You might not actually have realized it yet, but that doesn't make it any less true. Your management or business metrics may actually be positive, but if your development team isn't improving, the technology underlying everything is stagnating and sooner or later the smell will reach your customers.

Unless you already employ the best programers in the world, of course.

And if you really and truly believe that you employ the best programmers in the world, then unless your company name rhymes with 'Foogle', 'Schnapple', or 'Sicromoft', you're probably really screwed.

Normal function calls are easy to write; you call DoSomething( ), it executes and returns, and you continue on your merry way.

Asynchronous function calls seem, at first blush, only a little more difficult — instead of calling DoSomething() and executing your follow-up code after it returns, you pass in a callback: BeginDoSomething( Action callback ).

So, problem solved, let's go home. Unless you need a return value, that is. But even then it seems simple; to turn a synchronous method like int CalculateSomething( ) into an asynchronous method, you just pass it a delegate that takes a parameter: void BeginCalculateSomething( Action<int> callback ).

So is that it? Nope. Because all of that is wrong.

Even though the original DoSomething( ) method had no return type, it still had a return path — it could throw an exception. Let's imagine that BeginDoSomething looks something like this:


public void BeginDoSomething(Action callback)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                LongRunningOperation();

                callback();
            });
}

A handy way to think about this sort of thing is to figure out where a thrown exception would emerge.

If something goes wrong within the call to PrepareForLongRunningOperation, that happens in the same context as the calling code — any exceptions will throw up to the calling code and come out of its call to BeginDoSomething. The same applies to the call to ThreadPool.QueueUserWorkItem — no problem there.

But what if LongRunningOperation throws?

LongRunningOperation would throw up into whatever internal part of the ThreadPool implementation actually launched it. That exception can't come out of ThreadPool.QueueUserWorkItem, because by the time the asynchronous anonymous delegate is running ThreadPool.QueueUserWorkItem has already returned. And since the exception can't come out of ThreadPool.QueueUserWorkItem, it also can't come out of BeginDoSomething — which means there's no way for the calling code to get the exception.

There are two main approaches to this problem — error handlers and completion calls.

Error Handlers

Instead of passing your begin method one callback, pass two: a callback to be invoked if everything goes to plan, and an exception-accepting callback to which errors will be passed.


public void BeginDoSomething(Action callback, Action<Exception> errorHandler)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                try
                {
                    LongRunningOperation();
                    callback();
                }
                catch(Exception ex)
                {
                    errorHandler(ex);
                }
            });
}

// Sample usage:

BeginDoSomething(
    delegate
    {
		// Do something now that we're done
    },
    delegate(Exception ex)
    {
		// Do something with the error
    });

There are strengths and weaknesses to this approach. The biggest strength of this model is that it forces the calling code to think about error handling — the prompt for it is right there in the method signature. Error handling tends to fall through the cracks in any sort of code, but it's especially easy to overlook in an asynchronous context (It's also a lot more dangerous in an asynchronous context, because often dropping a callback invocation will cause a process to spin forever, sucking down resources and accomplishing nothing).

Separating the success case from the failure case may be either a strength or a weakness, depending on the particular task. Sometimes it makes your code much cleaner, but it often happens that your success and error handler need to share context and implementation, which can make for some very ugly code.

Completion Calls

Instead of just invoking a parameter-less Action callback or a single-parameter Action<TReturn> callback, your code calls a single-parameter callback and passes it an Action or Func<TReturn> that the callback in turn invokes.


public void BeginDoSomething(Action<Action> callback)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                try
                {
                    LongRunningOperation();
                    callback(delegate { });
                }
                catch(Exception ex)
                {
                    callback(delegate { throw ex; });
                }
            });
}

// Sample Usage:

BeginDoSomething(
    delegate(Action complete)
    {
        try
        {
            complete();
        }
        catch (Exception ex)
        {
            // Do something with the error
            return;
        }
        // Do something now that we're done
	})

At first blush, this seems like a much clumsier solution; you're essentially trusting the calling code to call your completion method. That's true, at least in this case.

Where completion calls really shine are for asynchronous calls returning values; instead of calling their callback and handing in an Action, you call their callback and give it a Func<TReturn>, which they then must invoke to get their result. That gives you an opportunity to throw exceptions that they can't cleverly bypass:


public void BeginCalculateSomething(Action<Func<int>> callback)
{
    PrepareForLongRunningCalculation();

    ThreadPool.QueueUserWorkItem(
        delegate
        {
            try
            {
                int result = LongRunningCalculation();
                callback(() => result);
            }
            catch (Exception ex)
            {
                callback(delegate { throw ex; });
            }
        });
}

// Sample Usage:

BeginCalculateSomething(
    delegate(Func<int> complete)
    {
        int result;
        try
        {
            result = complete();
        }
        catch (Exception ex)
        {
            // Do something with the error
            return;
        }
        // Do something with the content of result;
	})

Personally I prefer completion calls, mainly because the the pattern works so well for return values. In practice, any time you need this sort of a pattern it's because you care about return values; if you need to ensure that X happens after Y, it's generally because X depends on the result of Y. If X doesn't depend on Y, that's often a sign that you're being too linear in your analysis and that the tasks should be happening in parallel.

Microsoft seems to have collectively reached the same conclusion; IHttpAsyncHandler, the asynchronous methods off of SqlCommand, and the asynchronous forms of WebRequest all use the completion call approach.

Functional Programming

|
public void BeginGetSingle(TIdentityCriteria identityCriteria, CompletionCallback<TItem> callback)
{
    TFilterCriteria filterCriteria =
        CriteriaUtilities.UpgradeCriteria<TIdentityCriteria, TFilterCriteria>(identityCriteria);

    RestClient.BeginGet<TItem>(
        CriteriaUtilities.CriteriaToUrl(
            filterCriteria,
            m_Map,
            m_ServiceUrlBase,
            r => typeof (TItem).FullName.Equals(r.OutputPayloadClass) && r.AllowedVerbs.Contains(Verb.Get)),
        completionFunction => callback(() => completionFunction().Payload));        
}

My function takes a function and when done it calls that function, passing a function that the calling function calls for the result.

Now in fact my function calls another function taking a function accepting a function to call for its result, and to that function it passes a function which when called calls the first function passing a new function that when called calls the function that was passed to the function that my function passed to the other function, thereby returning the result of that function to the function that called my function.

And they say you can't write Lisp code in C#.

Xclan Night

|

Still a work in progress, but pretty cool looking already in my rarely-humble opinion. The trick is to dim the red and green components while leaving the blue intact.

Xclan screenshot 1
Xclan screenshot 2

I'm trying to decide the best way to show text bubbles. Everything's pretty scrupulously iPhone-friendly this time around (that's actually what got me started on Xclan again, figuring out how to work with UDP CFSockets in a run loop), but as I understand it uploading new text textures every few frames will bog an iPhone down pretty quickly. I'm thinking of switching my NSOpenGLView to a CAOpenGLLayer and putting a bunch of CATextLayers on top for the bubbles.

There's actually an NSTextField in a layer in the game window; on Leopard it floats above the NSOpenGLView, but in Snow Leopard (where the screen shots were taken) it's invisible. I haven't pursued it because I was planning on replacing the OGL view with a layer anyway, but it's a strange difference.

Pages