Sep 25

Last phase of the desktop wars?

The two most intriguing developments in the recent evolution of the Microsoft Windows operating system are Windows System for Linux (WSL) and the porting of their Microsoft Edge browser to Ubuntu.

For those of you not keeping up, WSL allows unmodified Linux binaries to run under Windows 10. No emulation, no shim layer, they just load and go.

Microsoft developers are now landing features in the Linux kernel to improve WSL. And that points in a fascinating technical direction. To understand why, we need to notice how Microsoft’s revenue stream has changed since the launch of its cloud service in 2010.

Ten years later, Azure makes Microsoft most of its money. The Windows monopoly has become a sideshow, with sales of conventional desktop PCs (the only market it dominates) declining. Accordingly, the return on investment of spending on Windows development is falling. As PC volume sales continue to fall off , it’s inevitably going to stop being a profit center and turn into a drag on the business.

Looked at from the point of view of cold-blooded profit maximization, this means continuing Windows development is a thing Microsoft would prefer not to be doing. Instead, they’d do better putting more capital investment into Azure – which is widely rumored to be running more Linux instances than Windows these days.

Our third ingredient is Proton. Proton is the emulation layer that allows Windows games distributed on Steam to run over Linux. It’s not perfect yet, but it’s getting close. I myself use it to play World of Warships on the Great Beast.

The thing about games is that they are the most demanding possible stress test for a Windows emulation layer, much more so than business software. We may already be at the point where Proton-like technology is entirely good enough to run Windows business software over Linux. If not, we will be soon.

So, you’re a Microsoft corporate strategist. What’s the profit-maximizing path forward given all these factors?

It’s this: Microsoft Windows becomes a Proton-like emulation layer over a Linux kernel, with the layer getting thinner over time as more of the support lands in the mainline kernel sources. The economic motive is that Microsoft sheds an ever-larger fraction of its development costs as less and less has to be done in-house.

If you think this is fantasy, think again. The best evidence that it’s already the plan is that Microsoft has already ported Edge to run under Linux. There is only one way that makes any sense, and that is as a trial run for freeing the rest of the Windows utility suite from depending on any emulation layer.

So, the end state this all points at is: New Windows is mostly a Linux kernel, there’s an old-Windows emulation over it, but Edge and the rest of the Windows user-land utilities don’t use the emulation. The emulation layer is there for games and other legacy third-party software.

Economic pressure will be on Microsoft to deprecate the emulation layer. Partly because it’s entirely a cost center. Partly because they want to reduce the complexity cost of running Azure. Every increment of Windows/Linux convergence helps with that – reduces administration and the expected volume of support traffic.

Eventually, Microsoft announces upcoming end-of-life on the Windows emulation. The OS itself , and its userland tools, has for some time already been Linux underneath a carefully preserved old-Windows UI. Third-party software providers stop shipping Windows binaries in favor of ELF binaries with a pure Linux API…

…and Linux finally wins the desktop wars, not by displacing Windows but by co-opting it. Perhaps this is always how it had to be.

Sep 17

My SubscribeStar account is live

I’ve finally gotten validated by SubscribeStar, which means I can get payouts through it, which means those of you who want nothing to do with Patreon can contribute through it: https://www.subscribestar.com/esr

If you’re not contributing, and you’re a regular here, please chip in. While I’ve had some research grants in the past, right now nobody is subsidizing the infrastructure work I do and I’m burning my savings.

There’s NTPsec, of course – secure time synchronization is critical to the Internet. There are the 40 or so other projects I maintain. Recently I’m working on improving the tools ecosystem for the Go language, because to reduce defect rates we have got to lift infrastructure like NTPsec out of C and Go looks like the most plausible path. Right now I’m modifying Flex so it can be be retargetable to emit Go (and other languages); I expect to do Go support for Bison next.

I’m not the only person deserving of your support – I’ve founded the Loadsharers network for people who want to help with the more general infrastructure-support problem – but if you’re a regular on this blog I hope I’m personally relatively high on your priority list. I’m not utterly broke yet, but the prospect is looming over me and my house needs a new roof.

Even $10 a month is helpful if enough of you match it. For $20 a month you get to be credited as a supporter when I ship a release of one of my personal projects. From $50 a month I can buy my long-suffering wife a nice dinner and afford an occasional trip to the shooting range.

I live a simple life, trying to be of service to my civilization. Please help that continue.

Sep 04

Kyle Rittenhouse and the militia obligation

There’s an angle on the case of Kyle Rittenhouse that I haven’t seen even hinted at in the media.

Section 246 of US Code Title 10, entitled, “Militia: composition and classes” reads:

“(a) The militia of the United States consists of all able-bodied males at least 17 years of age and, except as provided in section 313 of title 32, under 45 years of age who are, or who have made a declaration of intention to become, citizens of the United States and of female citizens of the United States who are members of the National Guard.

(b) The classes of the militia are:

(1) the organized militia, which consists of the National Guard and the Naval Militia; and

(2) the unorganized militia, which consists of the members of the militia who are not members of the National Guard or the Naval Militia.”

That is, all males of military age who are or intend to become citizens of the United States are under federal statute the “unorganized militia”, and have the duty of the militia to defend the Constitution of the United States against enemies foreign and domestic (as both naturalizing citizens and members of the armed forces swear to do).

Kyle Rittenhouse is a 17-year-old male and thus a member of the unorganized militia under black-letter Federal law. When he armed up to defend a friend’s business during a breakdown in civil order, he was acting precisely as all members of the unorganized militia have a legal and Constitutional duty to act in like circumstances.

And yes, shooting violent criminal insurrectionists is included in that duty. Rittenhouse would have been justified in doing so even if he had not been acting in self-defense against lethal threats to his person.

So let’s not hear any more nonsense about a teenager having no business being in that situation. Kyle Rittenhouse recognized one of his core duties as an American citizen and performed it with exceptional skill and courage.

Jul 14

Documentation as knowledge capture

Maybe you’re one of the tiny minority of programmers that, like me, already enjoys writing documentation and works hard at doing it right. If so,the rest of this essay is not for you and you can skip it.

Otherwise, you might want to re-read (or at least re-skim) Ground-Truth Documents before continuing. Because ground-truth documents are a special case of a more general reason why you might want to try to change your mindset about documentation.

In that earlier essay I used the term “knowledge capture” in passing. This is a term of art from AI; it refers to the process of extracting domain knowledge from the heads of human experts into a form that can be expressed as an algorithm executable by the literalistic logic of a computer.

What I invite you to think about now is how writing documentation for software you are working on can save you pain and effort by (a) capturing knowledge you have but don’t know you have, and (b) eliciting knowledge that you have not yet developed.

Humans, including me and you, are sloppy and analogical thinkers who tend to solve problems by pattern-matching against noisy data first and checking our intuitions with logic after the fact (if we actually get that far). There’s no point in protesting that it shouldn’t be that way, that we should use rigorous logic all the way down, because our brains simply aren’t wired for that. Evolved cognition is a kludge – more properly, multiple stacks of kludges – developed under selection to be just barely adequate at coping.

This kludginess is revealed by, for example, optical illusions. And by the famous 7±2 result about the very limited sized of the human working set. And the various well-documented ways that human beings are extremely bad at statistical reasoning. And in many other ways…

When you do work that is as demanding of rigor as software engineering, one of your central challenges is hacking around the limitations of your own brain. Sometimes this develops in very obvious ways; the increasing systematization of testing during development during the last couple of decades, for example.

Other brain hacks are more subtle. Which is why I am here to suggest that you try to stop thinking of documentation as a chore you do for others, and instead think of it as a way to explore your problem space. and the space in your head around your intuitions about the problem, so you can shine light into the murkier corners of both. Writing documentation can function as valuable knowledge capture about your problem domain even when you are the only expert about what you are trying to do.

This is why my projects often have a file called “designer’s notes” or “hacking guide”. Early in a project these may just be random jottings that are an aid to my own memory about why things are the way they are. They tend to develop into a technical briefing about the code internals for future contributors. This is a good habit to form if you want to have future contributors!

But even though the developed version of a “designer’s notes” looks other-directed, it’s really a thing I do to reduce my own friction costs. And not just in the communication-to-my-future-self way either. Yes, it’s tremendously valuable to have a document that, months or years after I wrote it, reminds me of my assumptions when I have half-forgotten them. And yes, a “designer’s notes” file is good practice for that reason alone. But its utility does not even start there, let alone end there.

Earlier, I wrote of (a) capturing knowledge you have but don’t know you have, and (b) eliciting knowledge that you have not yet developed. The process of writing your designer’s notes can be powerful and catalytic that way even if they’re never communicated. The thing you have to do in your brain to narratize your thoughts so they can be written down is itself an exploratory tool.

As with “designer’s notes” so with every other form of documentation from the one-line code comment to a user-oriented HOWTO. When you achieve right mindset about these they are no longer burdens; instead they become an integral part of your creative process, enabling you to design better and write better code with less total effort.

I understand that to a lot of programmers who now experience writing prose as difficult work this might seem like impossible advice. But I think there is a way from where you are to right mindset. That way is to let go of the desire for perfection in your prose, at least early on. Sentence fragments are OK. Misspellings are OK. Anything you write that explores the space is OK, no matter how barbarous it would look to your third-grade grammar teacher or the language pedants out there (including me).

It is more important to do the discovery process implied by writing down your ideas than it is for the result to look polished. If you hold on to that thought, get in the habit of this kind of knowledge capture, and start benefiting from it, then you might find that over time your standards rise and it gets easier to put more effort into polishing.

If that happens, sure; let it happen – but it’s not strictly necessary. The only thing that is necessary is that you occasionally police what you’ve recorded so it doesn’t drift into reporting something the software no longer does. That sort of thing is a land-mine for anyone else who might read your notes and very bad form.

Other than that, though, the way to get to where you do the documentation-as-knowledge-capture thing well is by starting small; allow a low bar for polish and completeness and grow the capability organically. You will know you have won when it starts being fun.

Jul 06

How not to treat a customer

First, my complaint to Simply NUC about the recent comedy of errors around my attempt to order a replacement fan for Cathy’s NUC. Sorry, I was not able to beat WordPress’s new editor into displaying URLs literally, and I have no idea why the last one turns into a Kindle link.

——————————————————-

Subject: An unfortunate series of events

Simply NUC claims to be a one-stop shop for NUC needs and a customer-centric company. I would very much like to do business with an outfit that lives up to Simply NUC’s claims for itself. This email is how about I observed it to fail on both levels.

A little over a week ago my wife’s NUC – which is her desktop machine, having replaced a conventional tower system in 2018 – developed a serious case of bearing whine. Since 1981 I have built, tinkered with, and deployed more PCs than I can remember, so I knew this probably meant the NUC’s fan bearings were becoming worn and could pack up at any moment.

Shipping the machine out for service was unappealing, partly for cost reasons but mostly because my wife does paying work on it and can’t afford to have it out of service for an unpredictable amount of time. So I went shopping for a replacement fan.

The search “NUC fan replacement” took me here:

NUC Replacement Fans

There was a sentence that said “As of right now SimplyNUC offers
replacement fans for all NUC models.” Chasing the embedded link
landed me on the Simply NUC site here:

Nuc Accessories

Now bear in mind that I had not disassembled my wife’s NUC yet, that I had landed from a link that said “replacement fans for all NUC models”, and that I didn’t know different NUCs used different fan sizes.

The first problem I had was that this page did nothing to even hint that the one fan pictured might not be a universal fit. Dominic has told me over the phone that “Dawson” is a NUC type, and if I had known that I might have interpreted the caption as “fit only for Dawsons”. But I didn’t, and the caption “Dawson BAPA0508R5U fan” looks exactly as though “Dawson” is the *fan vendor*.

So I placed the order, muttering to myself because there aren’t any shipping options less expensive than FedEx.

A properly informative page would have labeled the fan with its product code and had text below that said “Compatible with Dawson Canyon NUCs.” That way, customers landing there could get a clue that the BAPA0508R5U is not a universal replacement for all NUC fans.

A page in conformance with Simply NUC’s stated mission to be a one-stop NUC shop would also carry purchase links to other fans fitted for different model ranges, like the Delta BSC0805HA I found out later is required for my wife’s NUC8i3BEH1.

The uninformative website page was strike one.

In the event, when the fan arrived, I disassembled my wife’s NUC and instantly discovered that (a) it wasn’t even remotely the right size, and (b) it didn’t even match the fan in the website picture! What I was shipped was not a BAPA0508R5U, it’s a BAAA0508RSH.

Not getting the product I ordered was strike two.

I got on the Simply NUC website’s Zendesk chat and talked with a person named Bobbie who seemed to want to be helpful (I point this out because, until I spoke with Dominic, this was the one single occasion on which Simply NUC behaved like it might be run by competent people). I ended up emailing her a side-by-side photo of the two fans. It’s attached.

Bobbie handed me off to one Sean McClure, and that is when my experience turned from bad to terrible. If I were a small-minded person I would be suggesting that you fire Mr. McClure. But I’m not; I think the actual fault here is that nobody has ever explained to this man what his actual job is, nor trained him properly in how to do it.

And that is his *management’s* fault. Somebody – possibly one of the addressees of this note – failed him.

Back during the dot-com boom I was on the board of directors of a Silly Valley startup that sold PCs to run Linux, competing directly with Sun Microsystems. So I *do* in fact know what Sean McClure’s job is. It’s to *retain customers*. It’s to not alienate possible future revenue streams.

When a properly trained support representative reads a story like mine, the first words he types ought to be something equivalent to “I’m terribly sorry, we clearly screwed up, let me set up an RMA for that.” Then we could discuss how Simply NUC can serve my actual requirements.

That is how you recruit a loyal customer who will do repeat business and recommend you to his peers. That is how you live up to the language on the “About” page of your website.

Here’s what happened instead:

Unfortunately we don’t keep those fans in stock. You can try reaching out to Intel directly to see if they have a replacement or if they will need to RMA your device. You can submit warranty requests to: supporttickets.intel.com, a login will need to be created in order to submit the warranty request. Fans can also be sourced online but will require personal research.

This is not an answer, it’s a defensive crouch that says “We don’t care, and we don’t want your future business”. Let me enumerate the ways it is wrong, in case you two are so close to the problem that you don’t see it.

1. 99% odds that a customer with a specific requirement for a replacement part is calling you because he does *not* want to RMA the entire device and have it out of service for an unpredictable amount of time. A support tech that doesn’t understand this has not been taught to identify with a customer in distress.

2. A support tech that understands his real job – customer retention – will move heaven and earth rather than refer the customer to a competing vendor. Even if the order was only for a $15 fan, because the customer might be experimenting to see if the company is a competent outfit to handle bigger orders. As I was; you were never going to get 1,000 orders for whole NUCs from me but more than one was certainly possible. And I have a lot of friends.

3. “Personal research”? That’s the phrase that really made me angry. If it’s not Simply NUC’s job to know how to source parts for NUCs, so that I the customer don’t have to know that, what *is* the company’s value proposition?

Matters were not improved when I discovered that typing BAPA0508R5U into a search engine instanntly turned up several sources for the fan I need, including this Amazon page:

A support tech who understood his actual job would have done that search the instant he had IDed the fan from the image I sent him, and replied approximately like this: “We don’t currently stock that fan; I’ll ask our product guys to fix this and it should show on our Fans page in <a reasonable period>. In the meantime, I found it on Amazon; here’s the link.” >

As it is, “personal research” was strike three.

Oh, and my return query about whether I could get a refund wasn’t even refused. It wasn’t even answered.

My first reaction to this sequence of blunders was to leave a scathingly bad review of Simply NUC on TrustPilot. My second reaction was to think that, in fairness, the company deserves a full account of the blunders directed at somebody with the authority to fix what is broken.

Your move.

——————————————————-

Here’s the reply I got:

——————————————————-

Mr. Raymond, while I always welcome customer feedback and analyze it for
opportunities to improve our operations, I will not entertain customers who
verbally berate, belittle, or otherwise use profanity directed at my
employees or our company. That is a more important core value of our
company than the pursuit of revenue of any size.

I’ve instructed Sean to cancel the return shipping label as we’ve used enough of each other’s time in this transaction. You may retain the blower if it can be of any use to you or one of your friends in the future, or dispose of it in an environmentally friendly manner.

I will request a refund to your credit card for the $15 price of the product ASAP.


I don’t think any of this needs further elaboration on my part, but I note that Simply NUC has since modified its fans page to be a bit more informative.

Jul 04

Some PSAs for NUC owners

I’ve written before, in Contemplating the Cute Brick, that I’m a big fan of Intel’s NUC line of small-form-factor computers. Over the last week I’ve been having some unpleasant learning experiences around them. I’m still a fan, but I’m shipping this post where the search engines can see it in support of future NUC owners in trouble.

Two years ago I bought an NUC for my wife Cathy to replace her last tower-case PC – the NUC8i3BEH1. This model was semi-obsolete even then, but I didn’t want one of the newer i5 or i7 NUCs because I didn’t think it would fit my wife’s needs as well.

What my wife does with her computer doesn’t tax it much. Web browsing, office work, a bit of gaming that does not extend to recent AAA titles demanding the latest whizzy graphics card. I thought her needs would be best served by a small, quiet, low-power-consumption machine that was cheap enough to be considered readily disposable at the end of its service life. The exact opposite of my Great Beast…

The NUC was an experiment that made Cathy and me happy. She especially likes the fact that it’s small and light enough to be mounted on the back of her monitor, so it effectively takes up no desk space or floor area in her rather crowded office. I like the NUC’s industrial design and engineering – lots of nice little details like the four case screws being captive to the baseplate so you cannot lose them during disassembly.

Also. Dammit, NUCs are pretty. I say dammit because I feel like this shouldn’t matter to me and am a bit embarrassed to discover that it does. I like the color and shape and feel of these devices. Someone did an amazing job of making them unobtrusively attractive.

However…

Last week, Cathy registered a complaint that her NUC was making a funny noise. I went and listened and, alas, it was clearly the sound of the fan bearing in the NUC, screaming. That sound means you have worn or dirty bearing surfaces and the fan could fail at any time, forcing the device to shut down before it roasts its own components.

PSA #1: If you web-search for “NUC fan replacement”, you may well land at the website of a company specializing in NUC sales and support, named “Simply NUC”; I did. Do not buy from these people; they are lazy jerks.

First reason I know this: the “Fans” subpage in their Accessories section carries a link to exactly one model of fan. No indication of the range of NUC variants it matches, and not even a general warning that there are NUC models that require a different-sized fan. I had to find this out the hard way by pulling out the innards of Cathy’s NUC and sitting the fan I bought from Simply NUC next to it.

Two fans side by side

Second reason I know this: Simply NUC tech support was unhelpful, telling me they only carry that one fan and suggesting that I RMA Cathy’s machine back to Intel for repair, because obviously there could be no conceivable problem with it being out of service for an indefinite amount of time.

When I asked if Simply NUC knew of a source for a fan that would fit my 8i3BEH1 – a reasonable question, I think, to ask a company that loudly claims to be a one-stop shop for all NUC needs – the reply email told me I’d have to do “personal research” on that.

It turns out that if the useless drone who was Simply NUC “service” had cared about doing his actual job, he could have the read the fan’s model number off the image I had sent him into a search box and found multiple sources within seconds, because that’s what I then did. Of course this would have required caring that a customer was unhappy, which apparently they don’t do at Simply NUC.

Third reason I know this: My request for a refund didn’t even get refused; it wasn’t even answered.

It actually took some work to get the NUC board and fan out if its case. I watched some YouTube videos purporting to illuminate the process; none of them quite matched the hardware I was looking at and none told me the One Weird Trick I actually needed to know. Therefore:

PSA #2: If you’ve taken out both hold-down screws and the board still seems mechanically locked in place, it may well be because the NUC case is designed like that. On some NUCs you need to flex the two case walls with connector ports outwards by about a millimeter on each side so the connectors will pop out of their exit holes. The case is made of thin, springy metal; thumb pressure will do it.

So now I’m waiting on a second replacement fan to arrive. But there is good news; while I had the thing disassembled I blew out all the dust I could see with a can of air, playing it liberally over the fan. And since I reassembled it, it hasn’t screamed once. So:

PSA#3: Your NUC fan noise problem might be solvable just by blowing out the moondust under and around the fan bearings.

We’ll see. If I’m feeling lazy when the new fan arrives, I’ll leave it in the parts drawer until and unless the one now in the NUC fails. If I’m feeling energetic, I’ll swap in the new one, then disassemble and thoroughly clean and oil the old one before putting it in the drawer.

Jun 26

A user story about user stories

The way I learned to use the term “user story”, back in the late 1990s at the beginnings of what is now called “agile programming”, was to describe a kind of roleplaying exercise in which you imagine a person and the person’s use case as a way of getting an outside perspective on the design, the documentation, and especially the UI of something you’re writing.

For example:

Meet Joe. He works for Randomcorp, who has a nasty huge old Subversion repository they want him to convert to Git. Joe is a recent grad who got thrown at the problem because he’s new on the job and his manager figures this is a good performance test in a place where the damage will be easily contained if he screws up. Joe himself doesn’t know this, but his teammates have figured it out.

Joe is smart and ambitious but has little experience with large projects yet. He knows there’s an open-source culture out there, but isn’t part of it – he’s thought about running Linux at home because the more senior geeks around him all seem to do that, but hasn’t found a good specific reason to jump yet. In truth most of what he does with his home machine is play games. He likes “Elite: Dangerous” and the Bioshock series.

Joe knows Git pretty well, mainly through the Tortoise GUI under Windows; he learned it in school. He has only used Subversion just enough to know basic commands. He found reposurgeon by doing web searches. Joe is fairly sure reposurgeon can do the job he needs and has told his boss this, but he has no idea where to start.

What does Joe’s discovery process looks like? Read the first two chapters of “Repository Editing with Reposurgeon” using Joe’s eyes. Is he going to hit this wall of text and bounce? If so, what could be done to make it more accessible? Is there some way to write a FAQ that would help him? If so, can we start listing the questions in the FAQ?

Joe has used gdb a little as part of a class assignment but has not otherwise seen programs with a CLI resembling reposurgeon’s. When he runs it, what is he likely to try to do first to get oriented? Is that going to help him feel like he knows what’s going on, or confuse him?

“Repository Editing…” says he ought to use repotool to set up a Makefile and stub scripts for the standard conversion workflow. What will Joe’s eyes tell him when he looks at the generated Makefile? What parts are likeliest to confuse him? What could be done to fix that?

Joe, my fictional character, is about as little like me as as is plausible at a programming shop in 2020, and that’s the point. If I ask abstractly “What can I do to improve reposurgeon’s UI?”, it is likely I will just end up spinning my wheels; if, instead, I ask “What does Joe see when he looks at this?” I am more likely to get a useful answer.

It works even better if, even having learned what you can from your imaginary Joe, you make up other characters that are different from you and as different from each other as possible. For example, meet Jane the system administrator, who got stuck with the conversion job because her boss thinks of version-control systems as an administrative detail and doesn’t want to spend programmer time on it. What do her eyes see?

In fact, the technique is so powerful that I got an idea while writing this example. Maybe in reposurgeon’s interactive mode it should issue a first like that says “Interactive help is available; type ‘help’ for a topic menu.”

However. If you search the web for “design by user story”, what you are likely to find doesn’t resemble my previous description at all. Mostly, now twenty years after the beginnings of “agile programming”, you’ll see formulaic stuff equating “user story” with a one-sentence soundbite of the form “As an X, I want to do Y”. This will be surrounded by a lot of talk about processes and scrum masters and scribbling things on index cards.

There is so much gone wrong with this it is hard to even know where to begin. Let’s start with the fact that one of the original agile slogans was “Individuals and Interactions Over Processes and Tools”. That slogan could be read in a number of different ways, but under none of them at all does it make sense to abandon a method for extended insight into the reactions of your likely users for a one-sentence parody of the method that is surrounded and hemmed in by bureaucratic process-gabble.

This is embedded in a larger story about how “agile” went wrong. The composers of the Agile Manifesto intended it to be a liberating force, a more humane and effective way to organize software development work that would connect developers to their users to the benefit of both. A few of the ideas that came out of it were positive and important – besides design by user story, test-centric development and refactoring leap to mind,

Sad to say, though, the way “user stories” became trivialized in most versions of agile is all too representative of what it has often become under the influence of two corrupting forces. One is fad-chasers looking to make a buck on it, selling it like snake oil to managers forever perplexed by low productivity, high defect rates, and inability to make deadlines. Another is the managers’ own willingness to sacrifice productivity gains for the illusion of process control.

It may be too late to save “agile” in general from becoming a deadening parody of what it was originally intended to be, but it’s not too late to save design by user story. To do this, we need to bear down on some points that its inventors and popularizers were never publicly clear about, possibly because they themselves didn’t entirely understand what they had found.

Point one is how and why it works. Design by user story is a trick you play on your social-monkey brain that uses its fondness for narrative and characters to get you to step out of your own shoes.

Yes, sure, there’s a philosophical argument that stepping out of your shoes in this sense is impossible; Joe, being your fiction, is limited by what you can imagine. Nevertheless, this brain hack actually works. Eppure, si muove; you can generate insights with it that you wouldn’t have had otherwise.

Point two is that design by user story works regardless of the rest of your methodology. You don’t have to buy any of the assumptions or jargon or processes that usually fly in formation with it to get use out of it.

Point three is that design by user story is not a technique for generating code, it’ s a technique for changing your mind. If you approach it in an overly narrow and instrumental way, you won’t imagine apparently irrelevant details like what kinds of video games Joe likes. But you should do that sort of thing; the brain hack works in exact proportion to how much imaginative life you give your characters.

(Which in particular, is why stopping at a one-sentence “As an X, I want to do Y” is such a sadly reductive parody. This formula is designed to stereotype the process, but stereotyping is the enemy of novelty, and novelty is exactly what you want to generate.)

A few of my readers might have the right kind of experience for this to sound familiar. The mental process is similar to what in theater and cinema is called “method acting.” The goal is also similar – to generate situational responses that are outside your normal habits.

Once again: you have to get past tools and practices to discover that the important part of software design – the most difficult and worthwhile part – is mindset. In this case, and temporarily, someone else’s.

Jun 06

Yeet not, unless ye be yoten upon

This is an answer I posted to Stack Overflow: Linguistics that was so much fun to write that I feel like sharing it with my blog audience. The question is: What is the past tense of ‘yeet’?

I have a field sighting of the form “yoten” to report.

In January I was involved with the organizing for the big pro-Second-Amendment demonstration in Richmond, VA. One of the central concerns of the organizers, in view of the extreme hostility of the media against firearms rights, was to keep the demonstration strictly nonviolent.

Among the many memes and images posted on social media to express concurrence with this goal was a sort of cartoon of plucky musket-bearing rebels in Revolutionary War costume. The caption read:

“Yeet not unless ye be yoten upon!

You can’t tell gun-culture folks to be passively nonviolent; they’ll just laugh at you. You can preach an ethic of alert nonaggression, and that’s what this memester did. That fits their values, and works.

So we see “yeet” being used for the act of firing a weapon (no surprise; I already knew videogamers used it that way before this). We also have “yoten” as past tense in perfective aspect.

Subsequently someone else else posted a chart of the full conjugation of “yeet”. It did include “yote” as the simple past, and in general was a meticulous and knowing parody of the Germanic strong verb.

Watching culture being invented is a marvellous thing!

I concur with previous answers that the strong-verb system is not as yet entirely nonproductive in English. We should not expect new production of ablaut to obey historical rules for OE verb classes, but rather to template itself on surviving strong verbs.

I offer in this connection the following models: “speak/spoke/spoken”, “freeze/froze/frozen”, and “break/broke/broken”.

I further note that I have previously observed a tendency for such irregular inflections to flourish where humor is intended, and if the message of “Yeet not unless ye be yoten upon!” was serious the form of it was intentionally funny.

In another subculture that I am involved with, the name of a now obsolete minicomputer called a “VAX” was pluralized to “vaxen”, not “vaxes”, allegedly because the machine was as slow as an ox. This joke became productive: today, people in that culture may pluralize “box” (used in the sense of a computer, e.g a Unix box) as “boxen”.

The meme was successful. Enough armed gunfolks showed up to overmatch an infantry division, and the day was entirely peaceful.

Jun 04

Rules for rioters

I had business outside today. I needed to go in towards Philly, closer to the riots, to get a new PSU put into the Great Beast. I went armed; I’ve been carrying at all times awake since Philadelphia started to burn and there were occasional reports of looters heading into the suburbs in other cities.

I knew I might be heading into civil unrest today. It didn’t happen. But it still could.

Therefore I’m announcing my rules of engagement should any of the riots connected with the atrocious murder of George Floyd reach the vicinity of my person.

  1. I will shoot any person engaging in arson or other life-threatening behavior, issuing a warning to cease first if safety permits.
  2. Blacks and other minorities are otherwise safe from my gun; they have a legitimate grievance in the matter of this murder, and what they’re doing to their own neighborhoods and lives will be punishment enough for the utter folly of their means of expression once the dust settles.
  3. White rioters, on the other hand, will be presumed to be Antifa Communists attempting to manipulate this tragedy for Communist political ends; them I consider “enemies-general of all mankind, to be dealt with as wolves are” and will shoot immediately, without mercy or warning.

UPDATE: I didn’t mention white nationalists because I judge my chances of encountering any member of that tiny, ineffectual movement to be effectively zero, and I refuse to cooperate with the mass-media fiction that they are a significant factor in this crisis.

We don’t have a problem with white nationalists attempting to burn down our country using black people as tools and proxies. We have a problem with Communists doing that. I insist on naming – and if necessary, shooting – the real enemy.

May 28

Looking for C-to-anything transpilers

I’m looking for languages that have three properties:

(1) Must have weak memory safety. The language is permitted to crash on an out -of-bounds array reference or null pointer, but may not corrupt or overwrite memory as a result.

(2) Must have a transpiler from C that produces human-readable, maintainable code that preserves (non-perverse) comments. The transpiler is allowed to not do a 100% job, but it must be the case that (a) the parts it does translate are correct, and (b) the amount of hand-fixup required to get to complete translation is small.

(3) Must not be Go, Rust, Ada, or Nim. I already know about these languages and their transpilers.

May 17

Designing tasteful CLIs: a case study

Yesterday evening my apprentice, Ian Bruene, tossed a design question at me.

Ian is working on a utility he calls “igor” intended to script interactions with GitLab, a major public forge site. Like many such sites, it has a sort of remote-procedure-call interface that allows you, as an alternative to clicky-dancing on the visible Web interface, to pass it JSON datagrams and get back responses that do useful things like – for example – publishing a release tarball of a project where GitLab users can easily find it.

Igor is going to have (actually, already has) one mode that looks like a command interpreter for a little minilanguage, with each command being an action verb like “upload” or “release”. The idea is not so much for users to drive this manually as for them to be able to write scripts in the minilanguage which become part of a project’s canned release procedure. (This is why GUIs are irrelevant to this whole discussion; you can’t script a GUI.)

Ian, quite reasonably, also wants users to be able to run simple igor commands in a fire-and-forget mode by typing “igor” followed by command-line arguments. Now, classically, under Unix, you would expect a single-line “release” command to be designed to look something like this:

$ igor -r -n fooproject -t 1.2.3 foo-1.2.3.tgz

(To be clear, the dollar sign on the left is a shell prompt, put in to emphasize that this is something you type direct to a shell.)

In this invocation, the “-r” option says “I want to do a release”, the -n option says “This is the GitLab name of the project I’m shipping a release of”, the -t option specifies a release tag, and the following filename argument is the name of the tarball you want to publish.

It might not look exactly like this. Maybe there’d be yet another switch that lets you attach a release notes file. Maybe you’d have the utility deduce the project name from the directory it’s running in. But the basic style of this CLI (= Command Line Interface), with option flags like -r that act as command verbs and other flags that exist to attach their arguments to the request, is very familiar to any Unix user. This what most Unix system commands look like.

One of the design rules of the old-school style is that the first token on the line that is not a switch argument terminates recognition of switches. It, and all tokens after it, are treated as arguments to be passed to the program and are normally expected to be filenames (or, in the 21st century, filename-like things like URLs).

Another characteristic of this style is that the order of the switch clauses is not fixed. You could write

$ igor -t 1.2.3 -n fooproject -r foo-1.2.3.tgz

and it would mean the same thing. (Order of the following arguments, on the other hand, will usually be significant if there is more than one.)

For purposes of this post I’m going to call this style old-school UNIX CLI, because Ian’s puzzlement comes from a collision he’s having with a newer style of doing things. And, actually, with a third interface style, also ancient but still vigorous.

When those of us in Unix-land only had the old-school CLI style as a model it was difficult to realize that all of those switches, though compact and easy to type, imposed a relatively high cognitive load. They were, and still are, difficult to remember. But we couldn’t really notice this until we had something to contrast it with that met similar functional requirements with lower cognitive effort.

Though there may have been earlier precedents, the first well-known program to use something recognizably like what I will call new-school CLI was the CVS version control system. The distinguishing trope was this: Each CVS command begins with a subcommand verb, like “cvs update” or “cvs checkout”. If there are switches, they normally follow the subcommand rather than preceding it. And there are fewer switches.

Later version-control systems like Subversion and Mercurial picked up on the subcommand idea and used it to further reduce the number of arbitrary-looking switches users had to remember. In Subversion, especially, your normal workflow could consist of a sequence of svn add, svn update, svn status, and svn commit commands during which you’d never type anything that looked like an old-school Unixy switch at all. This was easy to remember, easy to document, and users liked it.

Users liked it because humans are used to remembering associations between actions and natural-language verbs; “release” is less of a memory load than “-r” even if it takes longer to type. Which illuminates one of the drivers of the old-school style; it was shaped back in the 1970s by 110-baud Teletypes on which terseness and only having to type few characters was a powerful virtue.

After Subversion and Mercurial Git came along, with its CLI written in a style that, though it uses leading subcommand verbs, is rather more switch-heavy. From the point of view of being comfortable for users (especially new users), this was a pretty serious regression from Subversion. But then the CLI of git wasn’t really a design at all, it was an accretion of features that there was little attempt to simplify or systematize. It’s fair to say that git has succeeded despite its rather spiky UI rather than because of it.

Git is, however a digression here; I’ve mainly described it to make clear that you can lose the comfort benefits of the new-school CLI if a lot of old-school-style switches crowd in around the action verbs.

Next we need to look at a third UI style, which I’m going to call “GDB style” because the best-known program that uses it today is the GNU symbolic debugger. It’s almost as ancient as old-school CLIs, going back to the early 1980s at least.

A program like GDB is almost never invoked as a one-liner at all; a command is something you type to its internal command prompt, not the shell. As with new-school CLIs like Subversuon’s, all commands begin with an action verb, but there are no switches. Each space-separated token after the verb on the command line is passed to the command handler as a positional argument.

Part of Igor’s interface is intended to be a GDB-style interpreter. In that, the release command should logically look something like this, with igor’s command prompt at the left margin.

igor> release fooproject 1.2.3 foo-1.2.3.tgz

Note that this is the same arguments in the same order as our old-school “igor -r” command, but now -r has been replaced by a command verb and the order of what follows it is fixed. If we were designing Igor to be Subversion-like, with a fire-and-forget interface and no internal command interpreter at all, it would correspond to a shell command line like this:

$ igor release fooproject 1.2.3 foo-1.2.3.tgz

This is where we get to the collision of design styles I referred to earlier. What was really confusing Ian, I think, is that part of his experience was pulling for old-school fire-and-forget with switches, part of his experience was pulling for new-school as filtered through git’s rather botched version of it, and then there is this internal GDB-like interpreter to reconcile with how the command line works.

My apprentice’s confusion was completely reasonable. There’s a real question here which the tradition he’s immersed in has no canned, best-practices answer for. Git and GDB evade it in equal and opposite ways – Git by not having any internal interpreter like GDB, GDB by not being designed to do anything in a fire-and-forget mode without going through its internal interpreter.

The question is: how do you design a tool that (a) has a GDB like internal interpreter for a command minilanguage, (b) also allows you to write useful fire-and-forget one-liners in the shell without diving into that interpreter, (c) has syntax for those one liners that looks like an old-school CLI, and (d) has only one syntax for each command?

And the answer is: you can’t actually satisfy all four of those constraints at once. One of them has to give. It’s trivially true that if you abandon (a) or (b) you evade the problem, the way Git and GDB do. The real problem is that an old-school CLI wants to have terse switch clauses with flexible order, a GDB-style minilanguage wants to have more verbose commands with positional arguments, and never these twain shall meet.

The only one-syntax-for-each-command choice you can make is to have the same command interpreter parse your command line and what the user types to the internal prompt.

I bit this bullet when I designed reposurgeon, which is why a fire-and-forget command to read a stream dump of a Subversion repository and build a live repository from it looks like this:

$ reposurgeon "read <project .svn" "prefer git" "rebuild ../overthere"

Each of those string arguments is just fed to reposurgeon’s internal interpreter; any attempt to look like an old-school CLI has been abandoned. This way, I can fire and forget multiple reposurgeon commands; for Igor, it might be more appropriate to pass all the tokens on the command line as a single command.

The other possible way Igor could go is to have a command language for the internal interpreter in which each line looks like a new-school shell command with a command verb followed by switch clusters:

igor> release -t 1.2.3 -n fooproject foo-1.2.3.tgz

Which is fine except that now we’ve violated some of the implicit rules of the GDB style. Those aren’t simple positional arguments, and we’re back to the higher cognitive load of having to remember cryptic switches.

But maybe that’s what your prospective users would be comfortable with, because it fits their established habits! This seems to me unlikely but possible.

Design questions like these generally come down to having some sense of who your audience is. Who are they? What do they want? What will surprise them the least? What will fit into their existing workflows and tools best?

I could launch into a panegyric on the agile-programming practice of design-by-user-story at this point; I think this is one of the things agile most clearly gets right. Instead, I’ll leave the reader with a recommendation to read up on that idea and learn how to do it right. Your users will be grateful.

May 13

Two graceful finishes

I’m having a rather odd feeling.

Reposurgeon. It’s…done; it’s a finished tool, fully fit for its intended purposes. After nine years of work and thinking, there’s nothing serious left on the to-do list. Nothing to do until someone files a bug or something in its environment changes, like someone writing an exporter/importer pair it doesn’t know about and should.

When you wrestle with a problem that is difficult and worthy for long enough, the problem becomes part of you. Having that go away is actually a bit disconcerting, like putting your foot on a step that’s not there. But it’s OK; there are lots of other interesting problems out there and I’m sure one will find me to replace reposurgeon’s place in my life.

I might try to write a synoptic look back on the project at some point.

Looking over some back blog posts on reposurgeon, I became aware that I never told my blog audience the last bit of the saga following my ankle surgery. That’s because there was no drama. The ankle is now fully healed and as solidly functional as though I never injured it at all – I’ve even stopped having residual aches in damp weather.

Evidently the internal cartilage healed up completely, which is far from a given with this sort of injury. My thanks to everyone who was supportive when I literally couldn’t walk.

May 08

Term of the day: builder gloves

Another in my continuing series of attempts to coin, or popularize, terms that software engineers don’t know they need yet. This one comes from my apprentice, Ian Bruene.

“Builder gloves” is the special knowledge possessed by the builder of a tool which allows the builder to use it without getting fingers burned.

Software that requires builder gloves to use is almost always faulty. There are rare exceptions to this rule, when the application area of the software is so arcane that the builder’s specialist knowledge is essential to driving it. But usually the way to bet is that if your code requires builder gloves it is half-baked, buggy, has a poorly designed UI or is poorly documented.

When you ship software that you know requires builder gloves, or someone else tells you that it seems to require builder gloves, it could ruin someone else’s day and reflect badly on you. But if you believe in releasing early and often, sometimes half-baked is going to happen. Here’s how to mitigate the problem.

1. Warn the users what’s buggy and unstable in your release notes and the rest of your documentation.

2. Document your assumptions where the user can see them,

3. Work harder at not being a terrible UI designer.

4. Watch the issues list/user’s forum/mailing list, and actually respond.

5. When someone tells you it requires builder gloves, believe them. And fix it so it doesn’t.

Becoming really good at software engineering requires that you care about the experience the user sees, not just the code you can see.

Apr 27

The feel for weapons

I read Scientists Have Recreated Ancient Battles to Solve Debate Over Ancient Bronze Swords and was annoyed.

Not because the study wasn’t worth doing for its own sake – I applaud archeologists with the good sense to use historical reenactors to learn more about how combat in bygone times must have worked. But it seems to have been done to refute a theory that shouldn’t have been entertained seriously for three seconds – namely that Bronze Age swords were mainly ceremonial items with little use in actual combat.

I am, as Charles Babbage might have said, not able rightly to apprehend the kind of confusion of ideas that could provoke such a theory.

Continue reading

Apr 26

Lassie errors

I didn’t invent this term, but boosting the signal gives me a good excuse for a rant against its referent.

Lassie was a fictional dog. In all her literary, film, and TV adaptations the most recurring plot device was some character getting in trouble (in the print original, two brothers lost in a snowstorm; in popular memory “Little Timmy fell in a well”, though this never actually happened in the movies or TV series) and Lassie running home to bark at other humans to get them to follow her to the rescue.

In software, “Lassie error” is a diagnostic message that barks “error” while being comprehensively unhelpful about what is actually going on. The term seems to have first surfaced on Twitter in early 2020; there is evidence in the thread of at least two independent inventions, and I would be unsurprised to learn of others.

In the Unix world, a particularly notorious Lassie error is what the ancient line-oriented Unix editor “ed” does on a command error. It says “?” and waits for another command – which is especially confusing since ed doesn’t have a command prompt. Ken Thompson had an almost unique excuse for extreme terseness, as ed was written in 1973 to run on a computer orders of magnitude less capable than the embedded processor in your keyboard.

Herewith the burden of my rant: You are not Ken Thompson, 1973 is a long time gone, and all the cost gradients around error reporting have changed. If you ever hear this term used about one of your error messages, you have screwed up. You should immediately apologize to the person who used it and correct your mistake.

Part of your responsibility as a software engineer, if you take your craft seriously, is to minimize the costs that your own mistakes or failures to anticipate exceptional conditions inflict on others. Users have enough friction costs when software works perfectly; when it fails, you are piling insult on that injury if your Lassie error leaves them without a clue about how to recover.

Really this term is unfair to Lassie, who as a dog didn’t have much of a vocabulary with which to convey nuances. You, as a human, have no such excuse. Every error message you write should contain a description of what went wrong in plain language, and – when error recovery is possible – contain actionable advice about how to recover.

This remains true when you are dealing with user errors. How you deal with (say) a user mistake in configuration-file syntax is part of the user interface of your program just as surely as the normally visible controls are. It is no less important to get that communication right; in fact, it may be more important – because a user encountering an error is a user in trouble that he needs help to get out of. When Little Timmy falls down a well you constructed and put in his path, your responsibility to say something helpful doesn’t lessen just because Timmy made the immediate mistake.

A design pattern I’ve seen used successfully is for immediate error messages to include both a one-line summary of the error and a cookie (like “E2317”) which can be used to look up a longer description including known causes of the problem and remedies. In a hypothetical example, the pair might look like this:

Out of memory during stream parsing (E1723)

E1723: Program ran out of memory while building the deserialized internal representation of a stream dump. Try lowering the value of GOGC to cause more frequent garbage collections, increasing the size of your swap partition, or moving to hardware with more RAM.

The key point here is that the user is not left in the lurch. The messages are not a meaningless bark-bark, but the beginning of a diagnosis and repair sequence.

If the thought of improving user experience in general leaves you unmoved, consider that the pain you prevent with an informative error message is rather likely to be your own, as you use your software months or years down the road or are required to answer pesky questions about it.

As with good comments in your code, it is perhaps most motivating to think of informative error messages as a form of anticipatory mercy towards your future self.

Apr 19

Payload, singleton, and stride lengths

Once again I’m inventing terms for useful distinctions that programmers need to make and sometimes get confused about because they lack precise language.

The motivation today is some issues that came up while I was trying to refactor some data representations to reduce reposurgeon’s working set. I realized that there are no fewer than three different things we can mean by the “length” of a structure in a language like C, Go, or Rust – and no terms to distinguish these senses.

Before reading these definitions, you might to do a quick read through The Lost Art of Structure Packing.

The first definition is payload length. That is the sum of the lengths of all the data fields in the structure. No padding is included in this length.

The second is stride length. This is the length of the structure with any interior padding and with the trailing padding or dead space required when you have an array of them. This padding is forced by the fact that on most hardware, an instance of a structure normally needs to have the alignment of its widest member for fastest access. If you’re working in C, sizeof gives you back a stride length in bytes.

I derived the term “stride length” for individual structures from a well-established traditional use of “stride” for array programming in PL/1 and FORTRAN that is decades old.

Stride length and payload length coincide if the structure has no interior or trailing padding. This can sometimes happen when you get an arrangement of fields exactly right, or your compiler might have a pragma to force tight packing even though fields may have to be accessed by slower multi-instruction sequences.

“Singleton length” is the term you’re least likely to need. It’s the length of a structure with interior padding but without trailing padding. The reason I’m dubbing it “singleton” length is that it might be relevant in situations where you’re declaring or passing a single instance of a struct not in an array.

Consider the following declarations in C on a 64-bit machine:

struct {int64_t a; int32_t b} x;
char y

That structure has a payload length of 12 bytes. Instances of it in an array would normally have a stride length of 16 bytes, with the last four bytes being padding. But your compiler might generate a 12-byte copy when you ask it to assign the value of x.

This struct has a singleton length of 12, same as its payload length. But these are not necessarily identical, Consider this:

struct {int64_t a; char b[6]; int32_t c} x;

The way this is normally laid out in memory it will have two bytes of interior padding after b, then 4 bytes of trailing padding after c. Its payload length is 8 + 6 + 4 = 18; its stride length is 8 + 8 + 8 = 24; and its singleton length is 8 + 6 + 2 + 4 = 20.

To avoid confusion, you should develop a habit: any time someone speaks or writes about the “length” of a structure, stop and ask: is this payload length, stride length, or singleton length?

Most usually the answer will be stride length. But someday, most likely when you’re working close to the metal on some low-power embedded system, it might be payload or singleton length – and the difference might actually matter.

Even when it doesn’t matter, having a more exact mental model is good for reducing the frequency of times you have to stop and check yourself because a detail is vague. The map is not the territory, but with a better map you’ll get lost less often.

Apr 14

Insights need you to keep your nerve

This is a story I’ve occasionally told various friends when one of the subjects it touches comes up. I told it again last night, and it occurred to me that I ought to put in the blog. It’s about how, if you want to have productive insights, you need a certain kind of nerve or self-belief.

Many years ago – possibly as far back as the late 80s – I happened across a film of a roomful of Sufi dervishes performing a mystical/devotional exercise called “dhikr”. The film was very old, grainy B&W footage from the early 20th century. It showed a roomful of bearded, turbaned, be-robed men swaying, spinning, and chanting. Some were gazing at bright objects that might have been lamps, or polished metal or jewelry reflecting other lamps – it wasn’t easy to tell from the footage.

I can’t find the footage I saw, but the flavor was a bit like this. No unison movement in what I saw, though – individuals doing different things and ignoring each other, more inward-focused.

The text accompanying the film explained that the intention of “dhikr” is to shut out the imperfect sensory world so the dervish can focus on the pure and holy name of Allah. “Right,” I thought, already having had quite a bit of experience as an experimental mystic myself, “I get this. In Zen language, they’re shutting down the drunken monkeys. Autohypnosis inducing a serene mind, nothing surprising here.”

But there was something else. Something about the induction methods they were using. It all seemed oddly familiar, more than it ought to. I had seen behaviors like this before somewhere, from people who weren’t wearing pre-Kemalist Turkish garb. I watched the film…and it hit me. This was exactly like watching a roomful of people with serious autism!

The rocking. The droning. The fixated behavior, or in the Sufi case the behavior designed to induce fixation. Which immediately led to the next question: why? I think the least hypothesis in cases where you observe parallel behaviors is that they have parallel causation. We know what the Sufis tell us about what they’re doing; might it tell us what the autists are doing what they’re doing?

The Sufis are trying to shut out sense data. What if the autists are too? That would imply that the autists live in a state of what is, for them, perpetual sensory overload. Their dhikr-like behaviors are a coping mechanism, an attempt to turn down the gain on their sensors so they can have some peace inside their own skulls.

The first applications of nerve I want to talk about here are (a) the nerve to believe that autistic behaviors have an explanation more interesting than “uhhh…those people are randomly broken”, and (b) the nerve to believe that you can apply a heuristic like “parallel behavior, parallel causes” to humans when you picked it up from animal ethology.

Insights need creativity and mental flexibility, but they also need you to keep your nerve. I think there are some very common forms of failing to keep your nerve that people who would like to have good and novel ideas self-sabotage with. One is “If that were true, somebody would have noticed it years ago”. Another is “Only certified specialists in X are likely to have good novel ideas about X, and I’m not a specialist in X, so it’s a bad risk to try following through.”

You, dear reader, are almost certainly browsing this blog because I’m pretty good at not falling victim to those, and duly became famous by having a few good ideas that I didn’t drop on the floor. However, in this case, I failed to keep my nerve in another bog-standard way: I believed an expert who said my idea was silly.

That was decades ago. Nowadays, the idea that autists have a sensory-overload problem is not even controversial – in fact it’s well integrated into therapeutic recommendations. I don’t know when that changed, because I haven’t followed autism research closely enough. Might even be the case that somewhere in the research literature, someone other than me has noticed the similarity between semi-compulsive autistic behaviors and Sufi dhikr, or other similar autohypnotic practices associated with mystical schools.

But I got there before the experts did. And dropped the idea because my nerve failed.

Now, it can be argued that there were good reasons for me not to have pursued it. Getting a real hearing for a heterodox idea is difficult in fields where the experts all have their own theories they’re heavily invested in, and success is unlikely enough that perhaps it wasn’t an efficient use of my time to try. That’s a sad reason, but in principle a sound one.

But losing my nerve because an expert laughed at me, that was not sound. I think I wouldn’t make that mistake today; I’m tougher and more confident than I used to be, in part because I’ve had “crazy” ideas that I’ve lived to see become everyone’s conventional wisdom.

You can read this as a variation on a theme I developed in Eric and the Quantum Experts: A Cautionary Tale. But it bears repeating. If you want to be successfully creative, your insights need you to keep your nerve.

Apr 03

On the implausibility of a war with China

In the wake of the PRC’s actions around the COVID-19 pandemic, there has been increasing speculation in some circles that the PRC might be preparing to wage war against the United States, or at least some sort of regional war (such as an invasion of Taiwan) in which treaty obligations would involve the U.S.

I’ve actually been considering this possibility, from my perspective as a wargamer and military-history buff, for over a decade – ever since China began seriously flexing its muscles in the South China Sea. And the risk of war has undoubtedly been rising recently.

The PRC has given U.S. and other trade partners ample reason to conclude that they need to decouple their economies from Chinese supply chains. Threats by China to use its control of most of rare-earth production for economic blackmail have been followed by much more serious threats to use its dominance of the manufacturing of basic pharmaceuticals as a weapon.

Post-COVID-19, it’s now strategically vital for other nations to develop supply chains for critical goods that are domestic, or at least better guarded against the political and epidemiological risk of relying on Chinese manufacturing. While necessary, this shift does mean the PRC has less to lose in the event of going to war.

Nevertheless, I continue to judge that the odds of China launching a war are very low. Nobody can entirely rule out enraged, irrational behavior by the PRC, but in the remainder of this post I will attempt to demonstrate why the war options available to the PRC hold out little or no prospect of a satisfying victory and entail severe terminal risks.

Continue reading

Mar 24

Choosing your weapons wisely

In the comments to my last post advising people not to panic-buy guns because of COVID-19, I got a request from a regular wondering how to choose a first firearm wisely, and about safe storage practices.

He said: “I’m thinking in the next year of getting a gun for home defense, and I’d like myself and my spouse to train with it. […] I have young kids, and want to make sure the gun is accessible enough to be reachable in the event of a break-in but hard enough to access that my daughter doesn’t get into it.”

Credentials for anybody new here: I have several decades of experience as a self-defense and firearms instructor. I’m grateful that I haven’t had to shoot a human being yet. I’m not a professional in this stuff, but people who are treat me as a peer. As you keep reading, I think you will recognize the common sense in my advice.

Content warning: if you are easily offended by cold-blooded consideration of violent outcomes or Damned Facts about statistical patterns in criminality, this post will offend you.

I’m going to address the second sentence first. For basic physical security, you may want to consider getting a pistol-sized gun safe with a biometric lock. However…and I cannot emphasize this enough…do not rely on this to protect your children. Children are curious and ingenious and if they consider your security a challenge to be defeated you could have a tragedy.

The only safety lock that reliably protects your child is the one you install in your child’s head by teaching him or her that a gun is a dangerous tool that should only be used with adult supervision. Explain the danger. Do not make your weapons taboo forbidden fruit or surround them with mystery; if your child is curious, take him or her to the range with you.

If your child becomes very interested, this is good. Shooting sports are an effective way to develop discipline and concentration. And very safe (safer than golf, for example) except in the extremely unlikely case that you’ve raised a sociopath or some other kind of minimal-brain-damage victim, in which case you have larger problems than I’m going to try to address here.

Now I’ll talk about intelligent choice of weapons. This depends on your threat model and where you live.

I’m going to go into different threat models more later in this post, but I’ll start with advice that is common to all of them. The single most universally useful firearm you can have – and the least dangerous in case of accident or misuse – is a reliable carry pistol which you do, in fact, carry daily.

Do not get hung up on caliber or type. Gunfolks love to argue about stopping power and bullet ballistics but it turns out that once you get out of the mouse-gun range (.22, .25 and .32) all pistol calibers have essentially indistinguishable statistics on two-shot stops.

Therefore, keep it simple. Rent several different pistols at a range. To use your time efficiently, exclude monster hand cannons like .44 Magnum; that is certainly not a good beginner’s choice. You should be looking at calibers from 9mm up to .45ACP (11mm). Shoot them all, and pay attention to which one fits your hand the best and feels most comfortable for you to shoot. That is almost certainly the one you should buy.

I myself prefer medium-caliber semiautomatics like a .40 or .45 because I don’t enjoy the snappy recoil of a 9mm. But other people can be best suited for lighter-caliber pistols or revolvers; there are a lot of relevant variables including the shape and size of your hands and what kind of upper-body strength you have.

For home defense, it’s probably a good idea to fit a laser sight on your pistol; I got an aftermarket one recently for my .45. Then you can train in point shooting using the laser – makes you faster responding because you don’t have to pause to get a sight picture.

Because this post is about choice of weapons, I’m not going to talk a lot about training methods except to say “do one”. Train, train, train. Get comfortable with firing your weapon, learn how to be accurate at normal pistol engagement ranges of 7-10 feet.

That’s feet, not yards. It’s pretty close. Accuracy at that range is easy. More important than crisply perfect technique is the ability to handle the psychological stress of clutch situations so your accuracy doesn’t go to shit when you’re tired, rattled, and in low-light conditions. Read up on “stress inoculation” and try to get some.

Don’t be daunted by the thought that it takes years to master shooting. As with all skills, the more you put into it, the more you can get out. But any competent instructor can teach you how to handle firearms safely in 20 minutes, and you can develop the competence for basic self-defense shooting in a few hours.

You should lock that in with at least semi-regular practice, though. The newer you are, the more regular it should be; eventually (after years) you may get to the point where your muscle memory is solid enough to weather long periods without practice.

You’ll need a holster so you can carry. A gunbelt – which is just an extra-stiff leather belt that helps distribute the weight of you weapon – is a good investment. Alas, choosing good gunleather is an entire topic in itself. Expect that the first holster you buy will not be optimal and that you’ll probably need to experiment a few times before finding one that suits you for long-term use.

One area in which I think the gun culture can be unhelpful is in helping you judge how much ammunition to keep around. The problem is that a lot of us gunfolks end to treat the size of our ammo stockpiles as a sort of tongue-in-cheek competitive studliness display. The truth is that for almost everyone a 250-round reserve per weapon (exclusive of what you buy and shoot at the range) is just fine – generous, in fact.

Beyond that first pistol, what else you should buy starts to diverge based on where you live and what your threat model is. I’m going to start by assuming the most common and simplest one, which I’ll call the Standard Threat Model: you want to defend yourself and your family against low-level criminal violence, with a side order of hedging against a temporary (on the order of a few days) condition of civil disorder due to, e.g., natural disaster.

In that case a lot depends on whether you live in Switzerland or Swaziland. Most of the U.S. has violent-crime statistics like Switzerland – very low base rate of crime, law-abiding neighbors, high levels of legal gun ownership. In Switzerland, even temporary disaster conditions don’t induce looting, arson, and crime spikes. Therefore they do not raise your threat level much.

Unfortunately, parts of the U.S. – some major urban cores, and some drug-corridor rural areas near the Mexican border – are Swaziland. In Swaziland base violent-crime rates are high. Rates of legal gun ownership are low. Your neighbors are unhelpful, and a high-deviant cohort of them is actively dangerous.

If you live in Switzerland (easily 95% of the U.S. by land area), rational assessment of the Standard Threat Model does not require you to be heavily armed. I’d start with a carry pistol for each adult household member, and one shotgun for fixed-point defense. Whether you should also get a rifle depends on where you live. If you’re urban or suburban there’s not a lot of physical point to it because you won’t have long enough sightlines for distance shooting to matter much.

If you’re rural, on the other hand, you want a rifle. How serious a rifle depends on whether you have dangerous critters like bears or mountain lions in your area. Most people can get away with what gunfolks call a “varmint rifle” – a light-caliber rifle that shoots cheap ammo like .22LR. This is fine if your typical animal threat is something like a rabid skunk. It will take care of threatening humans too, in the extremely unlikely event you assess enough threat to have to shoot them at distance.

If, on the other hand, you have heavy threats like cougars or bears, you need a heavier rifle and a bigger bullet. Detailed discussion of these is out of scope for this post. Besides, I don’t know much about heavy rifles and wouldn’t want to give bad advice.

If you live near enough to a Swaziland, your threat profile is entirely different. Here’s how to tell if you do: (1) you live in a rural area with the Mexican border or concentrations of illegal immigrants within a two-hour drive. (2) you live in an urban area and within 20 minutes’ walk of you are places where groups of black or Hispanic males aged 15-35 carrying intoxicants routinely gather.

Yes, I can hear you lefties screaming already. All I have to say is: study the crime statistics. We can tell all kinds of stories about why those numbers look the way they do. Some of the stories we could tell are racist and irrational (but I repeat myself). The fact that shitty people tell toxic stories about the numbers doesn’t change the numbers, and it doesn’t change what the rational response to the numbers should be.

In American Swaziland, unlike African Swaziland, there’s no dangerous-animal threat at all, so you don’t need a heavy rifle. However, you have a banditry problem – not just individual muggers and home invaders but gangs of feral predators who routinely commit crimes ranging from mass shoplifting raids upwards to savage monkey-dance beatings that cripple or kill their victims. Civil disorder in Swaziland is quite dangerous, not only because of direct threat from mobs of ferals but due to indirect threats like arson.

In Swaziland you also need to assume that any assailant will be high off his ass on something like PCP or bath salts – a disassociative anesthetic. Pistol rounds do not reliably stop such a person before they can get close enough to kill you unless you luck out with a heart or brain shot.

If you’re living in Swaziland, the best thing you can do for yourself and society is arm up to the level where you pose a credible threat not just to individual criminals but to a mob of drunk or drugged ferals with a low average IQ and poor impulse control. Because riots or natural disaster could require you to step up like a roof Korean.

That means we’re in scary-black-rifle territory. You want an AR-15 or something like one. Understand that functionally an AR-15 is not very different in how accurately can shoot and what it can stop from your granddad’s hunting rifle. However, what it does to a mob’s threat assessment is very different.

Granddad’s hunting rifle says to a mob “Stupid ofay probably hasn’t fired that thing in years.” Black rifle says “Uh oh, gun nut. Prepared. Would probably rather shoot than not.” Ironically, this means that if what you’re showing is granddad’s hunting rifle, you’re more likely to have to actually shoot it. I’d consider actually having to shoot a human a less than optimal outcome; if you don’t, I probably don’t want to know you.

And that pretty much wraps up the Standard Threat Model. Now I’ll briefly cover a couple of other possibilities you might want to arm against which group together because they push your weapons mix in a similar direction.

One is longer-term civil disorder, ranging upwards to what gunfolks and preppers call “SHTF” (Shit Hits The Fan) scenarios. Worrying about these changes your optimal weapons mix – basically, you have to assume mob-feral violence as a prompt threat even in Switzerland. You’ll want scary black rifles, at least one per military-age household member.

However…I urge you not to worry about the weapons themselves so much that you neglect other needs. One is ammunition. Anywhere near SHTF conditions ammunition is going to become scarce and valuable; you want at least a thousand rounds per weapon and 10K would not be excessive.

More importantly, however, you need to lay in serious amounts of food and medical supplies before going on any gun-buying sprees. You can’t eat bullets, and raiding your neighbors for food would get terminally risky pretty fast.

I myself do not prep for SHTF very seriously, for reasons which I could explain but which are beyond the scope of this post. However, there is a different reason for me to have a SHTF-like weapons mix: the Second Amendment. I take my Constitutional duty to be part of the nation in arms seriously, and I insist on having the weapons that would-be tyrants foreign and domestic fear and want to take away from me precisely because they want to take them.

Generalizing, a sufficient reason to keep weapons beyond what the Standard Threat Model requires is as a move in the political power game, with the goal of ensuring that they are never actually needed.