Sep 27

Twenty years after

I just shipped what was probably the silliest and most pointless software release of my career. But hey, it’s the reference implementation of a language and I’m funny that way.

Because I write compilers for fun, I have a standing offer out to reimplement any weird old language for which I am sent a sufficiently detailed softcopy spec. (I had to specify softcopy because scanning and typo-correcting hardcopy is too much work.)

In the quarter-century this offer has been active, I have (re) implemented at least the following: INTERCAL, Michigan Algorithmic Decoder, and a pair of obscure 1960s teaching languages called CORC and CUPL, and an obscure computer-aided-instruction language called Pilot.

Pilot…that one was special. Not in a good way, alas. I don’t know where I bumped into a friend of the language’s implementor, but it was in 1991 when he had just succeeded in getting IEEE to issue a standard for it – IEEE Std 1154-1991. He gave me a copy of the standard.

I should have been clued in by the fact that he also gave me an errata sheet not much shorter than the standard. But the full horror did not come home to me until I sat down and had a good look at both documents – and, friends, PILOT’s design was exceeded in awfulness only by the sloppiness and vagueness of its standard. Even after the corrections.

Continue reading

Sep 24

Dilemmatizing the NRA

So, the Washington Post publishes yet another bullshit article on gun policy.

In this one, the NRA is charged with racism because it doesn’t leap to defend the right of black men to bear arms without incurring a lethal level of police suspicion.

In a previous blog post, I considered some relevant numbers. At 12% of the population blacks commit 50% of violent index crimes. If you restrict to males in the age range that concentrates criminal behavior, the numbers work out to a black suspect being a a more likely homicidal threat to cops and public safety by at least 26:1.

Continue reading

Sep 18

Thinking like a master programmer, redux

Yes, there was a bug in my vint64 encapsulation commit. I will neither confirm nor deny any conjecture that I left it in there deliberately to see who would be sharp enough to spot it. I will however note that it is a perfect tutorial example for how you should spot bugs, and why revisions with a simple and provable relationship to their ancestors are best

Continue reading

Sep 17

Thinking like a master programmer

To do large code changes correctly, factor them into a series of smaller steps such that each revision has a well-defined and provable relationship to the last.

(This is the closest I’ve ever come to a 1-sentence answer to the question “How the fsck do you manage to code with such ridiculously high speed and low defect frequency? I was asked this yet again recently, and trying to translate the general principle into actionable advice has been on my mind. I have two particular NTPsec contributors in mind…)

So here’s a case study, and maybe your chance to catch me in a mistake.

Continue reading

Sep 13

Trials of the Beast

This last week has not been kind to the Great Beast of Malvern. Serenity is restored now, but there was drama and (at the last) some rather explosive humor.

For some time the Beast had been having occasional random flakeouts apparently related to the graphics card. My monitors would go black – machine still running but no video. Some consultation with my Beastly brains trust (Wendell Wilson, Phil Salkie, and John D. Bell) turned up a suitable replacement, a Radeon R360 R7 that was interesting because it can drive three displays (I presently drive two and aim to upgrade).

Last Friday I tried to upgrade to the new card. To say it went badly would be to wallow in understatement. While I was first physically plugging it in, I lost one of the big knurled screws that the Beast’s case uses for securing both cards and case, down behind the power supply. Couldn’t get it to come out of there.

Then I realized that the card needed a PCI-Express power tap and oh shit the card vendor hadn’t provided one.

Much frantic running around to local computer stores ensued, because I did not yet know that Wendell had thoughtfully tucked several spares of the right kind of cable behind the disk drive bays when he built the Beast. Which turns out to matter because though the PCI-E end is standardized, the power supply end is not and they have vendor-idiosyncratic plugs.

Eventually I gave up and tried to put the old card back in. And that’s when the real fun began. I broke the retaining toggle on the graphics card’s slot while trying to haggle the new card out. When I tried to boot the machine with the old card plugged back in, my external UPS squealed – and then nothing. No on-board lights, no post beep, no sign of life at all. I knew what that meant; evidently either the internal PSU or the mobo was roached.

Continue reading

Aug 18

Some aphorisms on software development methodology

The net benefit of having anything that can be called a software development methodology is in inverse proportion to the quality of your developers – and possibly inverse-squared.

Any software-development methodology works well on sufficiently small projects, but all scale very badly to large ones. The good ones scale slightly less badly.

One thing all methodologies tagged “agile” have in common is that they push developers away from the median. Competent developers work better; mediocre developers don’t change; incompetent ones get worse.

Software metrics have their uses. Unfortunately, their principal use is to create the illusion that you know what’s going on.

Structured development methodologies have their uses, too. Unfortunately, their principal use is to create an illusion of control.

Trust simple, crude metrics over complex ones because the simple ones are less brittle. KLOC is best, though a poor best.

Agile development is efficient only in environments where the cost of flag days is low. Otherwise, slow down, and take time to think and write about your architecture.

Good programmers are difficult to control; great ones are nearly impossible to control. Different methodologies require different kinds and degrees of control; match them to your developers wisely.

Process is not a good substitute for judgment; when you have to use it as one, your project is probably too large. Sometimes this is unavoidable, but don’t fool yourself about what that will cost you.

The difference between O(n**2) and O(n log n) really matters. It’s why lots of small teams working coordinated small projects works better than one big team building a monolith.

A million dollars is roughly a 55-gallon oil drum full of five-dollar bills. Large-scale software development is such a difficult and lossy process that it’s like setting fire to several of these oil drums and hoping a usable product flutters out of the smoke. If this drives you to despair, find a different line of work.

Jul 22

Big fish, small pond?

As of tonight, I have a new challenge in my life.

Sifu Dale announced tonight that next year he plans to send a team to next year’s Quoshu, a national-level martial-arts competition held every year in northern Maryland in late summer.

I told Sifu I want to go and compete in weapons, at least. He was like, “Well, of course,” as though he’d been expecting it. Which is maybe a little surprising and flattering considering I’ll be pushing 60 then.

It’ll mean serious training for the next year, and maybe a pretty humiliating experience if it turns out I’m too old and slow. But I want to try, because it’s national-level competition against fighters from dozens of different styles, and…frankly, I’m tired of not having any clear idea how good I am. Winning would be nice, but what I really want is to measure myself against a bigger talent pool.children tracker

Continue reading

Jul 03

Count the SKUs

The Washington Post is running a story alleging that surveys show gun ownership in the U.S,. is at a 40-year low. I won’t link to it.

This is at the same time gun sales are at record highs.

The WaPo’s explanation, is, basically, that all these guns are being bought by the same fourteen survivalists in Idaho.

Mine is that the number of gun owners with a justified fear that “surveys” are a data-gathering tool for confiscations is also at a record high, and therefore that the number lying to nosy strangers about having no guns is at a record high. videochat

I think there’s a way to discriminate between these cases on the evidence.

Continue reading

Jun 25

More scenes from the life of a system architect

Haven’t been blogging for a while because I’ve been deep in coding and HOWTO-writing. Follows the (slightly edited) text of an email I wrote to the NTPsec devel list that I I think might be of interest to a lot of my audience.

One of the questions I get a lot is: How do you do it? And what is “it”, anyway? The question seems like an inquiry into the mental stance that a systems architect has to have to do his job.

So, um, this is it. If you read carefully, I think you’ll learn a fair bit even if you haven’t a clue about NTP itself.

Continue reading

May 04

NTPsec dodges 8 of 11 CVEs because we’d pre-hardened the code

While most of the NTPsec team was off at Penguicon, the NTP Classic people shipped a release patched for eleven security vulnerabilities in their code. Which might have been pretty embarrassing, if those vulnerabilities were in our code, too. People would be right to wonder, given NTPsec’s security focus, why we didn’t catch all these sooner.

In fact, we actually did pre-empt most of them. The attack surface that eight of these eleven security bugs penetrate isn’t present at all in NTPsec. The vulnerabilities were in bloat and obsolete features we’ve long since removed, like the Mode 7 control channel.

I’m making a big deal about this because it illustrates a general point. One of the most effective ways to harden your code against attack – perhaps the most effective – is to reduce its attack surface.

Thus, NTPsec’s strategy all along has centered on aggressive cruft removal. This strategy has been working extremely well. Back in January our 0.1 release dodged two CVEs because of code we had already removed. This time it was eight foreclosed – and I’m pretty sure it won’t be the last time, either. If only because I ripped out Autokey on Sunday, a notorious nest of bugs.

Simplify, cut, discard. It’s often better hardening than anything else you can do. The percentage of NTP Classic code removed from NTPsec is up to 58% now, and could easily hit 2/3rds before we’re done,

May 03

With a little help from my friends

I had a great time at Penguicon 2016, including face time with a lot of the people who help out on my various projects. There are a couple of thoughts that kept coming back to me during these conversations. One is “It is good, having so many impressively competent friends.”

The other is that without me consciously working at it, an amazing support network has sort of materialized around me – people who believe in the various things I’m trying to do and encourage them by throwing hardware and money and the occasional supportive cheer at me.

Because I didn’t consciously try to recruit these people, it’s easy for me to miss how collectively remarkable they are and how much they contribute until several of them concentrate in one place as happened at Penguicon.

Where I thought: “I’ve been taking these people a bit for granted. I should do better.”

So here, in no particular order, is a (partial) list of people who are really helping. It focuses on those who were at Penguicon and are A&D regulars, so I may have left off some people that would belong on a more complete list.

Continue reading

Apr 27

Where your donations go (#1)

Because people do in fact drop money in my PayPal and Patreon accounts, I think a a decent respect to the opinions of mankind requires that I occasionally update everyone on where the money goes. First in an occasional series,

Recently I’ve been buying Raspberry Pi GPS HATs (daughterboards with a GPS and real-time clock) to go with the Raspberry PI 3 Dave Taht dropped on me. Yesterday morning a thing called an Uputronics GPS Extension Board arrived from England. A few hours ago I ordered a cheap Chinese thing obviously intended to compete with the Adafruit GPS HAT I bought last week.

The reason is that I’m working up a very comprehensive HOWTO on how to build a Stratum 1 timeserver in a box. Not content to merely build one, I’m writing a sheaf of recipes that includes all three HATs I’ve found and (at least) two revisions of the Pi.

What makes this HOWTO different from various build pages on this topic scattered around the Web? In general, the ones I’ve found are well-intended but poorly written. They make too many assumptions, they’re tied to very specific hardware types, they skip “obvious” steps, they leave out diagnostic details about how to tell things are going right and what to do when things go wrong.

My goal is to write a HOWTO that can be used by people who are not Linux and NTP experts – basically, my audience is anyone who could walk into a hackerspace and not feel utterly lost.

Also, my hope is that by not being tightly tied to one parts list this HOWTO will help people develop more of a generative understanding of how you compose a build recipe, and develop their own variations.

I cover everything, clear down to how to buy a case that will fit a HAT. And this work has already had some functional improvements to GPSD as a side effect.

I expect it might produce some improvements in NTPsec as well – our program manager, A&D regular Mark Atwood, has been smiling benignly on this project. Mark’s plan is to broadcast this thing to a hundred hackerspaces and recruit the next generation of time-service experts that way.

Three drafts have already circulated to topic experts. Progress will be interrupted for a bit while I’m off at Penguicon, but 1.0 is likely to ship within two weeks or so.

And it will ship with the recipe variations tested. Because that’s what I do with your donations. If this post stimulates a few more, I’ll add an Odroid C2 (Raspberry Pi workalike with beefier hardware) to the coverage; call it a stretch goal.

Apr 17

Friends of Armed & Dangerous party 2016!

This year’s meatspace party for blog regulars and friends will be held at Penguicon 2016 On Friday, April 29 beginning at 9PM 10PM.

UPDATE: Pushed back an hour because the original start time conflicted with the time slot assigned for my “Ask Me Anything” event.

The venue is the Southfield Westin hotel in Southfield, Michigan. It’s booked solid already; we were only able to get a room there Friday night, and will be decamping to the Holiday In Express across the parking lot on Saturday. They still have rooms, but I suggest making reservations now.

The usual assortment of hackers, anarchists, mutants, mad scientists, and for all I know covert extraterrestrials will be attending the A&D party. The surrounding event is worth attending in itself and will be running Friday to Sunday.

Southfield is near the northwestern edge of the Detroit metro area and is served by the Detroit Metropolitan Airport (code DTW).

Penguicon is a crossover event: half science-fiction convention, half open-source technical conference. Terry Pratchett and I were the co-guests-of-honor at Penguicon I back in 2003 and I’ve been back evey year since.

If you’ve never been to an SF con, you have no idea how much fun this can be. A couple thousand unusually intelligent people well equipped with geek toys and costumes and an inclination to party can generate a lot of happy chaos, and Penguicon reliably does. If you leave Monday without having made new friends, you weren’t trying.

Things I have done at Penguicon: Singing. Shooting pistols. Tasting showcased exotic foods. Getting surprise-smooched by attractive persons. Swordfighting. Playing strategy games. Junkyard Wars. Participating in a Viking raid (OK, it turned into a dance-off). Punning contests. And trust me, you have never been to parties with better conversation than the ones we throw.

Fly in Thursday night (the 28th) if you can because Geeks With Guns (the annual pistol-shooting class founded by yours truly and now organized by John D. Bell) is early Friday afternoon and too much fun to miss.

Apr 15

The midrange computer dies

About five years ago I reacted to a lot of hype about the impending death of the personal computer with an observation and a prediction. The observation was that some components of a computer have to be the size they are because they’re scaled to human dimensions – notably screens, keyboards, and pointing devices. Wander outside certain size extrema and you get things like smartphone keyboards that are only good for limited use.

However, what we normally think of as the heart of a computer – the processing and storage – isn’t like this. It can get arbitrarily small without impacting usability at all. Consequently, I predicted a future in which people would carry around powerful computing nodes descended from smartphones and walk them to docking stations bundling a screen, a pointing device, and a real keyboard when they need to get real work done.

We’ve now reached an interesting midway point on that road. The (stationary) computers I use are in the process of bifurcating into two classes: one quite large, one very small. I qualify that with “stationary” because laptops are an exception for reasons which, if not yet obvious, will be in a few paragraphs.

Continue reading

Apr 12

TPP and the Law of Unintended Consequences

Once upon a time, free-trade agreements were about just that: free trade. You abolish your tariffs and import restrictions, I’ll abolish mine. Trade increases, countries specialize in what they’re best equipped to do, efficiency increases, price levels drop, everybody wins.

Then environmentalists began honking about exporting pollution and demanded what amounted to imposing First World regulation on Third World countries who – in general – wanted the jobs and the economic stimulus from trade more than they wanted to make environmentalists happy. But the priorities of poor brown people didn’t matter to rich white environmentalists who already had theirs, and the environmentalists had political clout in the First World, so they won. Free-trade agreements started to include “environmental safeguards”.

Next, the labor unions, frightened because foreign workers might compete down domestic wages, began honking about abusive Third World labor conditions about which they didn’t really give a damn. They won, and “free trade” agreements began to include yet more impositions of First World pet causes on Third World countries. The precedent firmed up: free trade agreements were no longer to be about “free” trade, but rather about managing trade in the interests of wealthy First Worlders.

Today there’s a great deal of angst going on in the tech community about the Trans-Pacific Partnership. Its detractors charge that a “free-trade” agreement has been hijacked by big-business interests that are using it to impose draconian intellectual-property rules on the entire world, criminalize fair use, obstruct open-source software, and rent-seek at the expense of developing countries.

These charges are, of course, entirely correct. So here’s my question: What the hell else did you expect to happen? Where were you idiots when the environmentalists and the unions were corrupting the process and the entire concept of “free trade”?

Continue reading

Apr 07

Too clever by half

The British have a phrase “Too clever by half”, It needs to go global, especially among hackers. It can have any of several closely related meanings: the one I mean to focus on here has to do with overconfidence in one’s intelligence or skill, and the particular bad consequences that can have. It’s related to Nassim Taleb’s concept of a “fragilista”.

Continue reading

Apr 03

This may be the week the SJWs lost it all

This may be the week the SJWs lost it all…or, at least, their power to bully people in the hacker culture and the wider tech community.

Many of you probably already know about the LambdaConf flap. In brief: LambdaConf, a technical conference on functional programming, accepted a presentation proposal about a language called Urbit, from a guy named Curtis Yarvin. I’ve looked at Urbit: it is very weird, but rather interesting, and certainly a worthy topic for a functional programming conference.

And then all hell broke loose. For Curtis Yarvin is better known as Mencius Moldbug, author of eccentric and erudite political rants and a focus of intense hatred by humorless leftists. Me, I’ve never been able to figure out how much of what Moldbug writes he actually believes; his writing seems designed to leave a reader guessing as to whether he’s really serious or executing the most brilliantly satirical long-term troll-job in the history of the Internet.

A mob of SJWs, spearheaded by a no-shit self-described Communist named Jon Sterling, descended on LambdaConf demanding that they cancel Yarvin’s talk, pretending that he (rather than, say, the Communist) posed a safety threat to other conference-goers. The conference’s principal organizers, headed up one John de Goes, quite properly refused to cancel the talk, observing that Yarvin was there to talk about his code and not his politics.

Continue reading

Apr 03

Sometimes I should give in to my impulses

For at least five years now I’ve been telling myself that, as nifty as it would be to play with the hardware, I really shouldn’t spend money on a small-form-factor PC.

This was not an easy temptation to resist, because I found little systems like the Intel NUC fascinating. I’d look over the specs for things like that in on-line stores and drool. Replacing a big noisy PC seemed so attractive…but I always drew back my hand, because that hardware came with a premium pricetag and I already have working kit.

Then, tonight, I’m over at my friend Phil Salkie’s place. Phil is a hardware and embedded-programming guy par excellence; I know he builds small-form-factor systems for industrial applications. And tonight he’s got a new toy to show off, a Taiwanese mini-ITX box called a Jetway.

He says “$79 on Amazon”, and I say “I’ve thought about replacing my mailserver with something like that, but could never cost-justify it.” Phil looks at me and says “You should. These things lower your electric bills – it’ll pay itself off inside of a year.”

Oh. My. Goddess. Why didn’t I think of that?

Continue reading