The discourses of Master Foo

Unix koans, via Eric S Raymond.

Best emacs blog post ever

Astounding! C-u 999 M-x all-hail steveyegge. I've used the latest emacs nearly every day for 13 years and never picked up on this \# and \, business in replace-regexp. Why wasn't this hailed in the newspapers and plastered on billboards along the interstate?!

Teach yourself programming in ten years

Here's Peter Norvig's good old essay about the time and effort it takes to learn a craft. One of his points is to learn a lot of languages; here's a list of the ones I've dabbled in since 1977:

• BASIC
• FORTRAN
• C
• TeX/LaTeX
• TCL
• PostScript
• emacs lisp
• perl
• Mathematica

and I'm probably leaving some out. I'm not an expert in any of them, though I did write some very cool stuff in TeX and PostScript lo! these many years ago. I've found it takes some effort to switch from my current language to one of the older ones - like traveling to France with only your high-school French to get by on for a few days.

On functional programming

Looks like functional programming is the next big hype in programming - expect to see lots of bullshit-bingo corporate-speak about it in the coming years. Meanwhile, here's a decent overview in Dr Dobb's Journal of a few languages in which functional programming is possible, including a peek at the Mathematica language, which I write in all day every day.

Mathematica version 7 released

And there was much rejoicing! Here's an overview from the boss, Stephen Wolfram.

Rule of thumb

When your daily tests start taking longer than twenty-four hours to finish, it's time to optimize your code for pure speed (or punt and move to a faster machine).

Surprised at the Sands

Frank Sinatra's first live album, his 1966 Sinatra at the Sands, is surprisingly good music to write computer programs by, and it's just plain good music at the Copa Room with Count Basie and his Orchestra, with arrangements and conducting by a young Quincy Jones. There are some good reviews at amazon.

With Sinatra's monologues and asides, the album also manages to capture the classic mid-60's Las Vegas full of boozers and two-bit hustlers busy being entertained by the Rat Pack.

Good coding music

A list of good stuff to listen to while writing code:

• Aimee Mann's 2002 album Lost in Space, disk 1.
• Dire Straits' first album, the eponymous 1978 Dire Straits.
• Mazzy Star's 1993 album So Tonight That I Might See.
• Pinkback's 2004 album Summer in Abaddon.
• Sufjan Stevens' 2005 album Illinois.
• JS Bach's Six Suites for Unaccompanied Cello.

A new interview with Donald Knuth

I've already had to look it up twice, so I'll put it here for future reference: Donald Knuth's interview with Andrew Binstock. Abstract:

Andrew Binstock and Donald Knuth converse on the success of open source, the problem with multicore architecture, the disappointing lack of interest in literate programming, the menace of reusable code, and that urban legend about winning a programming contest with a single compilation.

So that's what Joe was up to

Sometimes it takes me a decade or so to realize what's going on. Reading this blog entry tonight, I finally realized 14 years later what Joe Kaiping and Joe Grohens were up to.

It was 1994; Wolfram Research's Joe Grohens had sent one of his TeX gurus, Joe Kaiping, to the TeX Users Group conference in Santa Barbara, California, while my employer sent me, Rich Rogers and Mike Hockaday. We met Kaiping and he joined our little gang of young geeks. It happened that his problem and my interests overlapped, and I wound up months later porting the Wolfram flavor of LaTeX to some Mac version of TeX. This was the early 1990s, so when I finished the project I FedEx'd the code to him on a pile of floppies. (!)

One day in 1996 he dropped me an email and suggested I apply for his position at Wolfram - he was heading off to a private consulting gig and they needed another guy who knew TeX. My interview with his boss, Joe Grohens, was rather perfunctory and involved no coding at all. And now I realize, after all these years, that they had already conducted the code-writing part of the interview by having me do the porting job months earlier.

Fun with Mathematica

Once you get used to writing things in the Mathematica language, all sorts of fun things become possible. This morning I printed a set of small two-sided Mexican flags that we'll wrap around toothpicks, glue together and stick into Krispy Kreme doughnuts tonight:

TableForm[
  Partition[
    Table[
      Framed[
        CountryData["Mexico", "Flag"]
      ],
      {24}
    ],
    2
  ],
  TableSpacing -> {Automatic, 2}
]

and I just solved a little regular expression problem posed by my boss:

> What's the regex for matching a three digit version number?
>
> Valid examples (where 'x' is an integer > 0)
> x
> x.
> x.x
> x.x.
> x.x.x

Howzabout -

In[19]:= strings = {"6", "6.", "6.0", "6.0.", "6.0.1"}
Out[19]= {"6", "6.", "6.0", "6.0.", "6.0.1"}

In[30]:= StringMatchQ[#, RegularExpression["[0-9](\.([0-9](\.([0-9])?)?)?)?"]] & /@ strings
Out[30]= {True, True, True, True, True}

Something like Subsets

This is the first entry in what I'm calling my "list cookbook". When you deal with sets of things, you sometimes need to transform the sets to do something nifty with them. Here's the example that started me off on the cookbook idea.

Summary

• {a, b, c} => {{a}, {a, b}, {a, b, c}
• {a, b, c} => {{a, b, c}, {b, c}, {c}}
• {a, b, c} => {{c, b, a}, {c, b}, {c}}
• {a, b, c} => {{c, b, a}, {b, a}, {a}}

Problem

Given a list {a, b, c, d, e}, what Mathematica commands will turn that into {{a}, {a, b}, {a, b, c}, {a, b, c, d}, {a, b, c, d, e}}?

Solution

Thanks to Mike Pilat of Wolfram Research:

In[1]:= list = {a, b, c, d, e}
Out[1]= {a, b, c, d, e}

In[2]:= Reverse[NestList[Most, list, Length[list] - 1]]
Out[2]= {{a}, {a, b}, {a, b, c}, {a, b, c, d}, {a, b, c, d, e}}

Variations

The beauty of Mike's solution is that by tweaking bits of it you can arrive at other possibly useful list transformations. Change Most to Rest and don't Reverse the list and you get:

In[3]:= NestList[Rest, list, Length@list - 1]
Out[3]= {{a, b, c, d, e}, {b, c, d, e}, {c, d, e}, {d, e}, {e}}

Similarly, but Reverse-ing the list before processing it:

In[4]:= NestList[Most, Reverse@list, Length@list - 1]
Out[4]= {{e, d, c, b, a}, {e, d, c, b}, {e, d, c}, {e, d}, {e}}

which is equivalent to swapping Rest for Most and Reverse-ing each element of the resulting list:

In[5]:= Reverse /@ NestList[Rest, list, Length@list - 1]
Out[5]= {{e, d, c, b, a}, {e, d, c, b}, {e, d, c}, {e, d}, {e}}

One more variation:

In[6]:= NestList[Rest, Reverse@list, Length@list - 1]
Out[6]= {{e, d, c, b, a}, {d, c, b, a}, {c, b, a}, {b, a}, {a}}

which is equivalent to:

In[7]:= Reverse /@ NestList[Most, list, Length@list - 1]
Out[7]= {{e, d, c, b, a}, {d, c, b, a}, {c, b, a}, {b, a}, {a}}

Fix vs IntegerPart

I occasionally play around with computational astronomy, and the book on the subject is Paul J. Heafner's Fundamental Ephemeris Computations, which gives routines for working with JPL's high-accuracy planetary ephemerides.

Since Dr Heafner wrote his functions in PowerBASIC, I needed to translate PowerBASIC's FIX function to some Mathematica equivalent this evening. I read over the scanty docs at powerbasic.com and thought it sounded like Mathematica's IntegerPart; checking the docs for IntegerPart I was amazed at the amount of information provided - from basic definitions and explanations to arcane unexpected applications of the function, plus relevant links to related functions and, of course, the magisterial entry at MathWorld - check out that table of fractional-part/integer-part functions! I'm mighty proud of our documentation.

Not just an employee, but also a satisfied customer.

Beware the complete rewrite

Very sound advice from Joel Spolsky on the temptation to chuck all that code and rewrite the whole danged thing. In a word: don't.

Moving mountains

This week I'm revisiting the very first thing I wrote in Mathematica, a documentation testing package. A quick initial survey revealed a vast panorama of tottering kluge towers built high atop mountains of cruft. It's been very pleasing to retain the odd little working pieces and utterly trash the rest of the infrastructure in favor of a tidy little routine that handles more in a dozen lines of code than what I initially squeezed into 600 lines of bizarre rigamarole.

Here it is, all boiled down to the uttermost simplicity:

RunTests[dir_String, tests_List] :=
  Module[{manifest, res},
    manifest = FileNames[{"*.nb"}, dir, Infinity];
    res = Reap[Module[{nb, nbExpr},
      nb = #;
      nbExpr = Quiet@Get@nb;
      Module[{test},
        test = #;
        If[DocumentationCheckQ[nbExpr, test] === False, Sow[nb, test]];
      ]& /@ tests;
    ]& /@ manifest, _, Rule][[2]];
    (* return results sorted from most failures to least *)
    Sort[res, Length[Last[#1]] > Length[Last[#2]] &]]

Adding pairs of numbers: programming styles in Mathematica

Via Lunchtime Playground: given a list of pairs of numbers, return a list consisting of the sum of each pair.

pairs = {{58, 96}, {85, 22}, {100, 69}, {5, 37}, {32, 64}, {41, 86}, {14, 0}, {79, 22}, {55, 36}, {86, 39}, {11, 15}};

Six ways to add the numbers in Mathematica:

• Style 1:

result = Table[Null, {Length[pairs]}];
Do[result[[k]] = pairs[[k, 1]] + pairs[[k, 2]], {k, 1, Length[pairs]}]
result

• Style 2:

Table[pairs[[k, 1]] + pairs[[k, 2]], {k, Length[pairs]}]

• Style 3:

Apply[Plus, pairs, {1}]

• Style 4:

Map[Total, pairs]

• Style 5:

pairs /. {p_, q_} -> p + q

• Style 6:

pairs[[All, 1]] + pairs[[All, 2]]

Let's play "Programmer or Hell's Angel?"

A great series of photos and comments here (part 1) and here (part 2). Maybe that's why I've let my beard go the last few months...

Alert! Alert! Did you know?!

Don't stop my flow with some stupid dialog box:

Come to think of it, that's a lot like working from home with a kid with Aperger's Syndrome.

Never throw away information

That was the first bit of advice Joe Grohens gave me when I started working at Wolfram Research in 1996, and it's still the best programming maxim I know. I just rediscovered its value.

Five years ago Dave and I were editing some documentation source files when we found some old TeX macros that weren't used anymore. He left them in the code and defined them as no-ops, while I was leaning towards getting rid of them all. And there those macros sat doing nothing for half a decade til tonight when I suddenly needed the information they preserved.

Never throw away information.

Word counts

My current pleasant little project in between the cracks of my other larger project: word counts for translations. First, there's lots of good old-fashioned electronic archaeology to find the specific English Mathematica notebooks used for a partial translation way back when, followed by customization of my notebook word count routines to account for archaic data formats in the old files.

You don't have to be a math whiz to work at Wolfram Research.

Untangling

When I fell into the "documentation testing" gig couple of years ago I began by modifying some of my boss's existing tests to extract a bit more information here and there. That file grew and sent out tentacles into other files and nowadays it's grown to a tangled web of functions across half a dozen files. Now my job is to reunite everything under one clean standalone Mathematica package. Finally a chance to sit back and think about all this stuff and get it organized. It will also be my third rethinking of the whole thing - I've found I should plan to throw away my first couple of implementations of anything as I learn what works in the long run and what doesn't. Frederick Brooks said you should plan to throw one away; it takes me two, at least.

Looking for a programmer?

Michael Olson is graduating from Purdue in December, so grab him now before some other company does. I know him as the maintainer and developer of the emacs wiki & publishing package called muse, which I use to publish this here blog. He does lots of other stuff, too. Here are his resume and curriculum vitae.

How estimates go awry

A must read for programmers: Building a Fort: Lessons in Software Estimation by Steve McConnell, whose blog I'm putting in the sidebar now.