The lower-post-volume people behind the software in Debian. (List of feeds.)

I just released the first release candidate for libinput 1.8. Aside from the build system switch to meson one of the more visible things is that the helper tools have switched from a "libinput-some-tool" to the "libinput some-tool" approach (note the space). This is similar to what git does so it won't take a lot of adjustment for most developers. The actual tools are now hiding in /usr/libexec/libinput. This gives us a lot more flexibility in writing testing and debugging tools and shipping them to the users without cluttering up the default PATH.

There are two potential breakages here, one is that the two existing tools libinput-debug-events and libinput-list-devices have been renamed too. We currently ship compatibility wrappers for those but expect those wrappers to go away with future releases. The second breakage is of lesser impact: typing "man libinput" used to bring up the man page for the xf86-input-libinput driver. Now it brings up the man page for the libinput tool which then points to the man pages of the various features. That's probably a good thing, it puts the documentation a bit closer to the user. For the driver, you now have to type "man 4 libinput" though.

Posted Mon Jun 19 01:47:00 2017 Tags:
This paper was long overdue. But software papers are not easy to write, particularly not follow up papers. That actually seems a lot easier for databases. Moreover, we already publish too much. However, the scholarly community does not track software citations (data citations neither, but there seems to be a bit more momentum there; larger user group?). So, we need these kind of papers, and just a version, archived software release (e.g. on Zenodo) is not enough. But, there it is, the third CDK paper (doi:10.1186/s13321-017-0220-4). Fifth, if you include the two papers about ring finding, also describing CDK functionality.


Of course, I could detail what this paper has to offer, but let's not spoil the article. It's CC-BY so everyone can readily access it. You don't even need Sci-Hub (but are allowed for this paper; it's legal!).

A huge thanks to all co-authors, John's work as release manager and great performance improvements as well as code improvement, code clean up, etc, and all developers who are not co-author on this paper but contributed bigger or smaller patches over time (plz check the full AUTHOR list!). That list does not include the companies that have been supporting the project in kind, tho. Also huge thanks to all the users, particularly those who have used the CDK in downstream projects, many of which are listed in the introduction of the paper.

And, make sure to follow John Mayfield's blog with tons of good stuff.
Posted Sat Jun 10 10:53:00 2017 Tags:

Colossal Cave Adventure, that venerable classic, is back and better than ever! The page for downloads is here.

The game is fully playable. It would be astonishing if it were otherwise, since it has been stable since 1995. One minor cosmetic change a lot of people used to the non-mainline variants will appreciate is that there’s a command prompt.

The most substantial thing Jason Ninneman and I have done to the code itself is that the binary is now completely self-contained, with no need for it to refer to an external adventure.text file. That file is now compiled to C structures that are linked to the rest of the game at build time.

The other major thing we’ve done is that the distribution now includes a pretty comprehensive regression-test suite. We do coverage analysis to verify that it visits most of the code. This clears the way to do serious code cleanup (gotos and stale FORTRAN idioms must die!) while being sure we’re not breaking things,

We gratefully acknowledge the support of the game’s original authors, Don Woods and Will Crowther.

Posted Mon Jun 5 19:46:08 2017 Tags:

I need a bit of hands-on time with someone who knows how to troubleshoot WordPress installations.

I recently had to upgrade my WordPress installation due to an exploit that inserted a malicious URL in one of my widgets. Since then, my spam filter has not been operating correctly. I am not sure whether Akismet is working very slowly or not working at all, but the net result is that I am having to approve every post by hand.

There is a suspicious thing visible from my admin account. It looks as though Akismet is installed twice.

I suspect the fix for this is something simple, but I don’t know what it is. Can anybody help?

Posted Mon Jun 5 10:54:44 2017 Tags:

Henry Spencer, upon reading my previous post, had this to say by email:

I won’t argue with Eric about the significance of that little event, but I do have to interject a few corrections about the historical details.

For this, I’ve got a couple of advantages over him. First, being somewhat of a packrat, I still have my notes from conferences 30+ years ago! (Not in digital form, alas.) And second, being the one who saved most of what we have of Usenet’s early days, I also have some relevant old Usenet postings to consult.

The precise date was 19 January 1984, in the second morning session of the Uniforum conference (a joint meeting of the Usenix Association and the /usr/group trade association). Kathleen Hemenway of Bell Labs gave a talk on work she and Helene Armitage had done: “A proposed syntax standard for UNIX system commands”. This was basically an attempt to codify and slightly tighten the rules already implemented several years earlier by AT&T’s getopt() library function.

As you might gather from that wording, getopt() wasn’t actually new then: it had existed within Bell for some time, but hadn’t made it out in any widely-available Unix release. In late 1981, I’d seen a Bell-internal Unix manual page describing it. I wanted it, and I couldn’t have it, so I wrote my own. It was indeed quite helpful, and on 11 Jan. 1982, I posted it to Usenet newsgroup net.sources. By the time of that conference, a fair number of people were using it.

Ms. Hemenway’s talk was mostly about the syntax issues, but she did mention that there were plans for enhancements to getopt() to help support the new spec. And during Q&A, she was indeed evasive about availability of the enhanced code — probably nobody had made any decisions about that. So I got up and said that I would upgrade my public-domain implementation to match any enhancements AT&T made. Didn’t seem like a big deal; to be honest, I was a bit surprised that it got cheers and applause. I guess a lot of people hadn’t yet grasped that we *had a choice* about this.

No…no, we hadn’t. And my having got the date wrong explains the particularly high anxiety about the Bell breakup; the consent decree would have taken effect not nine months previously but just eight days before.

Ever since, on the occasions that I remembered this before returning to a busy life, the second or third thought in my mind was always that I ought to find Henry and let him know what an influence he had on me in that moment.

Henry Spencer, you set an example that day for all of us, and we cheered you not for promising to write a few lines of code but because we heard the call behind the promise. The details grew dim in my memory, but the power of that example struck me again each time I recalled it. Value your craft and pursue excellence in it; share what you create; never fear to go up against a monopolist; be loyal to your peers in the work. This is how a hacker – how any kind of maker, really – does his duty to the future.

A lot of us have been trying to live out that lesson ever since. Thank you, Henry. I think the world owes you more than it knows for that inspiration.

Posted Mon Jun 5 01:01:06 2017 Tags:

Back in 2003, the pressure range value for Wacom pen tablets was set to 2048. That's a #define in the driver, but it shouldn't matter because we also advertise this range as part of the device description in the X Input protocol. Clients should be using that advertised min/max range and scale appropriately, in the same way as they should be doing this for the x/y axis ranges.

Fast-forward to 2017 and we changed the pressure range. New Wacom devices now use ~8000 levels, but we opted to just #define it in the driver to 65536 and be done with it. We now scale all models into that range, with varying granularity based on the physical hardware. It shouldn't matter because it's not tied to a reliable physical property anyway and the only thing that matters is the percentage of the max value (which is why libinput just gives you a [0, 1] range. Hindsight is a bliss.).

Slow-forward to 2017-and-a-bit and we received complaints that pressure handling is now broken. Turns out that some applications hardcoded the old 2048 range and now get confused, because virtually any pressure will now hit that maximum. Since those applications are largely proprietary ones and cannot be updated easily, we needed a workaround to this. Jason Gerecke from Wacom got busy and we now have a "Pressure2K" option available in the driver. If set, this option will scale everything into the 2048 range to make sure those applications still work. To get this to work, the following xorg.conf.d snippet is recommended:


Section "InputClass"
Identifier "Wacom pressure compatibility"
MatchDriver "wacom"
Option "Pressure2K" "true"
EndSection
Put it in a file that sorts higher than the wacom driver itself (e.g. /etc/X11/xorg.conf.d/99-wacom-pressure2k.conf) and restart X. Check the Xorg.log/journal for a "Using 2K pressure levels" message, then verify it works by running xinput list "device name". xinput should show a range of 0 to 2048 on the third valuator.

$> xinput list "Wacom Intuos4 6x9 Pen stylus"
Wacom Intuos4 6x9 Pen stylus id=25 [slave pointer (2)]
Reporting 8 classes:
Class originated from: 25. Type: XIButtonClass
Buttons supported: 9
Button labels: None None None None None None None None None
Button state:
Class originated from: 25. Type: XIKeyClass
Keycodes supported: 248
Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 0:
Label: Abs X
Range: 0.000000 - 44704.000000
Resolution: 200000 units/m
Mode: absolute
Current value: 22340.000000
Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 1:
Label: Abs Y
Range: 0.000000 - 27940.000000
Resolution: 200000 units/m
Mode: absolute
Current value: 13970.000000
Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 2:
Label: Abs Pressure
Range: 0.000000 - 2048.000000
Resolution: 1 units/m
Mode: absolute
Current value: 0.000000

Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 3:
Label: Abs Tilt X
Range: -64.000000 - 63.000000
Resolution: 57 units/m
Mode: absolute
Current value: 0.000000
Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 4:
Label: Abs Tilt Y
Range: -64.000000 - 63.000000
Resolution: 57 units/m
Mode: absolute
Current value: 0.000000
Class originated from: 25. Type: XIValuatorClass
Detail for Valuator 5:
Label: Abs Wheel
Range: -900.000000 - 899.000000
Resolution: 1 units/m
Mode: absolute
Current value: 0.000000

This is an application bug, but this workaround will make sure new versions of the driver can be used until those applications have been fixed. The option will be available in the soon-to-be-released xf86-input-wacom 0.35. The upstream commit is d958ab79d21b57141415650daac88f9369a1c861.

Edit 02/06/2017: wacom devices have 8k pressure levels now.

Posted Thu Jun 1 01:24:00 2017 Tags:

I excavated a bit of hacker history from old memories today. Not dead history either, but an important beginning of some large good things.

Here’s how it happened. I got email from a person requesting me to identify a source for the following allegedly famous quote: “All operating systems eventually turn into Unix”.

I told him that I’d never heard that quote as written but that it reminded me of a line by Henry Spencer: “Those who do not understand Unix are condemned to reinvent it, poorly.” This is of course a take on a well-known aphorism by Santayana.

I googled Henry Spencer’s name and some keywords to make sure I had the quote correct. The top hit, from which I verified the quote, was Henry’s Wikipedia page. I read it and something caught my eye – the assertion that Henry wrote his well-known and widely distributed regex package around 1987. And suddenly I remembered something.

Years ago I wrote Eminent Domains: The First Time I Changed History in which I mentioned Henry Spencer’s presence at the conference where I persuaded the NWG not to drop the functional domains in favor of a strict geographic system.

What I just remembered today is something else that happened at that conference that may have been as important, though in a less obvious way. It also pins the date to 1984, because I remember now that anxiety about what AT&T’s recent commercialization of Unix meant to us was very much in the air at that conference (IIRC the big announcement had happened in January or February). We were all worried that this meant the Bell Labs source code would no longer be shared with universities and research labs…as was indeed to become the case.

To fully understand this story you need to know that Henry was already a hugely respected figure in the tiny, not yet very conscious hacker culture of the time (there were, at a guess, somewhere around 500 of us then in the U.S. and Canada – maybe 700 worldwide). I knew him from USENET; I’d seen some of his code, in the relatively small amounts we passed around back then.

It also matters that the GNU Manifesto hadn’t been written yet – wouldn’t be until the following year, though I had some idea what was coming because Richard Stallman had discussed his plans with me at Boskone in February. (That was when I suggested to him that the flagship project for the FSF ought to be Emacs. Duh!) Stallman’s concept of “Free Software” was not yet part of the hacker zeitgeist,

So in October 1984 I was in a crowd of people watching a presentation by a woman from Bell Labs describing the then-new getopt(3) library, written by AT&T as a way to regularize the processing of command-line arguments in C programs. The custom up to then had been to write ad-hoc code for each application, leading to some odd and unpredictable variations in behavior.

Everybody thought this was a fine idea, and several people asked questions probing whether AT&T was going to let anyone else use the getopt code they had written. These questions related to the general anxiety about Unix source code distributions drying up.

Frustration mounted as the woman gave evasive answers which seemed to add up to “No, we refuse to commit to allowing general access to this code.” Which seemed to confirm everyone’s worst fears about what was going to happen to Unix source code access in general.

At which point Henry Spencer stands up and says (not in these exact words) “I will write and share a conforming implementation.” – and got a cheer from the assembled.

If you’re thinking “That’s not a big deal, we do this sort of thing all the time,” my actual point is that in October 1984 this was indeed a big deal. It took an actual imaginative leap for Henry Spencer to, in effect, say “Screw AT&T and its legalisms and evasions, if they’re going to cut off source access we hackers are gonna do it for ourselves.”

Yes, we had DECUS tapes and the Unix sources newsgroups on Unix already; Henry was building on that. But he got an actual cheer exactly because he was pushing forward, exposing the possibility of doing not just small projects and demos and quirky little tools but at competing with the likes of AT&T itself at software production.

Of course RMS was, as I was then one of the very few to already know, thinking along the same lines. But RMS’s was a more personal, ideological vision. Henry didn’t have any grand plan to save the world; he was just being a hacker, seeing where the solution to the problem posed by AT&T’s source-code lockdown had to lie, pointing it out just a bit sooner than anyone else, and putting his own skin and considerable reputation in the game.

So yeah. In retrospect I think this was a historically pivotal moment. A respected elder of our tiny tribe (so tiny that probably a substantial fraction of it was in the room listening) declared his independence from what AT&T chose to do about restricting its code. And they took that possibility home with them.

I’m put in mind of the historian Oswald Spengler’s notion that cultures are born at a moment of what Nietzsche called “transvaluation of all values”. Arguably this was one, rather like the one I semi-accidentally triggered in the late 1990s, at which the hacker culture woke up some – became aware of the possibilities implied by its existing folkways.

And there’s a more direct and personal aspect I had half-forgotten. I was a young, unknown programmer then – just 27, still figuring out what I wanted. I watched Henry make that promise. I heard the cheer, and felt the change in the air as culturally, we realized what the solution to AT&T fscking us over had to be.

And I thought “I want to be like that guy.”

Posted Fri May 26 20:29:42 2017 Tags:

TLDR: If you see devices like "xwayland-pointer" show up in your xinput list output, then you are running under a Wayland compositor and debugging/configuration with xinput will not work.

For many years, the xinput tool has been a useful tool to debug configuration issues (it's not a configuration UI btw). It works by listing the various devices detected by the X server. So a typical output from xinput list under X could look like this:


:: whot@jelly:~> xinput list
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ SynPS/2 Synaptics TouchPad id=22 [slave pointer (2)]
⎜ ↳ TPPS/2 IBM TrackPoint id=23 [slave pointer (2)]
⎜ ↳ ELAN Touchscreen id=20 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ Power Button id=6 [slave keyboard (3)]
↳ Video Bus id=7 [slave keyboard (3)]
↳ Lid Switch id=8 [slave keyboard (3)]
↳ Sleep Button id=9 [slave keyboard (3)]
↳ ThinkPad Extra Buttons id=24 [slave keyboard (3)]
Alas, xinput is scheduled to go the way of the dodo. More and more systems are running a Wayland session instead of an X session, and xinput just doesn't work there. Here's an example output from xinput list under a Wayland session:

$ xinput list
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ xwayland-pointer:13 id=6 [slave pointer (2)]
⎜ ↳ xwayland-relative-pointer:13 id=7 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ xwayland-keyboard:13 id=8 [slave keyboard (3)]
As you can see, none of the physical devices are available, the only ones visible are the virtual devices created by XWayland. On a Wayland session, the X server doesn't have access to the physical devices. Instead, it talks via the Wayland protocol to the compositor. This image from the Wayland documentation shows the architecture:
In the above graphic, devices are known to the Wayland compositor (1), but not to the X server. The Wayland protocol doesn't expose physical devices, it merely provides a 'pointer' device, a 'keyboard' device and, where available, a touch and tablet tool/pad devices (2). XWayland wraps these into virtual devices and provides them via the X protocol (3), but they don't represent the physical devices.

This usually doesn't matter, but when it comes to debugging or configuring devices with xinput we run into a few issues. First, configuration via xinput usually means changing driver-specific properties but in the XWayland case there is no driver involved - it's all handled by libinput inside the compositor. Second, debugging via xinput only shows what the wayland protocol sends to XWayland and what XWayland then passes on to the client. For low-level issues with devices, this is all but useless.

The takeaway here is that if you see devices like "xwayland-pointer" show up in your xinput list output, then you are running under a Wayland compositor and debugging with xinput will not work. If you're trying to configure a device, use the compositor's configuration system (e.g. gsettings). If you are debugging a device, use libinput-debug-events. Or compare the behaviour between the Wayland session and the X session to narrow down where the failure point is.

Posted Tue May 23 00:56:00 2017 Tags:

Colossal Cave Adventure was the origin of many things; the text adventure game, the dungeon-crawling D&D (computer) game, the MOO, the roguelike genre. Computer gaming as we know it would not exist without ADVENT (as it was known in its original PDP-10 incarnation).

Long ago, you might have played this game. Or maybe you’ve just heard stories about it, or vaguely know that “xyzzy” is a magic word, or have heard people say “You are in a maze of twisty little passages, all alike”,

Though there’s a C port of the original 1977 game in the BSD game package, and the original FORTRAN sources could be found if you knew where to dig, Crowther & Woods’s final version – Adventure 2.5 from 1995 – has never been packaged for modern systems and distributed under an open-source license.

Until now, that is.

With the approval of its authors, I bring you Open Adventure. And with it some thoughts about what it means to be respectful of an important historical artifact when it happens to be software.

This is code that fully deserves to be in any museum of the great artifacts of hacker history. But there’s a very basic question about an artifact like this: should a museum preserve it in a static form as close to the original as possible, or is it more in the right spirit to encourage the folk process to continue improving the code?

Modern version control makes this question easier; you can have it both ways, keeping a pristine archival version in the history and improving it. Anyway, I think the answer to the general question is clear; if heritage code like this is relevant at all, it’s as a living and functional artifact. We respect our history and the hackers of the past best by carrying on their work and their playfulness.

I’ve actually dealt with this question before, which is why Super Star Trek has both interface improvements and a -t option that invokes the original TTY interface in all its brutal simplicity. Open Adventure will have a similar option.

When Willie Crowther wrote the very first version in 1976 he was also writing firmware for the earliest generation of routers on the operational ARPANET – he was one of the most ingenious and capable programmers of his time. The code makes a fascinating study in how to push the limits of the primitive tools then available to him.

It’s very strange to the modern eye just to see a simulation like ADVENT written with no analogue even of C structures, let alone objects. But the FORTRAN Crowther wrote in didn’t have such things; game state was all global variables, a “feature” preserved in the mechanically translated C of 2.5. (Which, alas, is extremely ugly code full of gotos.)

It also looks odd, 40 years after the fact, to see the amount of code complexity devoted to space/time optimization so that (for example) you don’t have to re-parse the text master of the dungeon-defining database on every startup. That’s what you had to do then, when a room-filling minicomputer cranked many fewer instructions per second than the controller in your microwave oven.

Despite all the energy Crowther and Woods had to spend fighting ancient constraints, ADVENT was a tremendous imaginative leap; there had been nothing like it before, and no text adventure that followed it would be innovative to quite the same degree.

Thus, one of my goals moving forward will be to make the logical design of the dungeon simulation easier to understand for a modern reader. This will probably entail something parallel to what was done in the BSD Games port – gradually massaging the code into more idiomatic C. This reflects my judgment that it’s more important to make Crowther & Wood’s algorithmic thinking more visible than it is to, in any sense, preserve the details of their code.

It’s actually more difficult to decide how much to change their data structures. In idiomatic C one would collect all the saveable game state into a structure rather than leaving it as a mess of global variables that have to be saved and restored one at a time. And a lot of the strange bit-packing hacks in the database representation could probably be done away with today. But would doing that sort of thing obscure their achievement too much?

Probably not – and the baseline version will still be visible in the repository, after all. Likely what I’ll end up doing (if I can allocate the time) is cleaning up stuff like that and leaving comments explaining what it looked like in the FORTRAN original.

I don’t plan on any gameplay changes yet. Any such things will be reversible by the switch to restore historic behavior.

One thing I definitely plan is to modify the code so it can be regression-tested by capturing command logs, replaying them to the game and checking that the output is as expected – testing a full walkthrough should be possible. Matter of fact that needs to happen before anything else so that the correctness of changes can be verified.

This is not a strategy that would likely have occurred to anyone (well, maybe outside of a handful of people at Bell Labs) in 1977. Finding it thinkable depends on a mindset created by Unix pipes and redirection that was only gestating then.

Replay also busts one of Crowther & Woods’s original goals – thwarting attempts to get around obstacles in the game with any investment less than quitting and playing again. Towards this end they deliberately obscured the encoding of save files to make point tweaks of the game state difficult.

That, however, is a hack that fails the Kerckhoffs’s Law test – it’s useless or near useless in an environment where the “adversary” has the source code. When ADVENT (so called because PDP-10 filenames were limited to six characters of uppercase) was written, that was a pretty safe assumption, but today it will never be safe again. No harm, then, in dropping their obscurifying measures.

Source code obscurity is a “feature” from 1977 I miss less than even FORTRAN…

Posted Mon May 22 16:59:12 2017 Tags:

A marvellous thing has just occurred.

Colossal Cave Adventure, the original progenitor of the D&D-like dungeon-crawling game genre from 1977 and fondly remembered as ADVENT by those of us who played it on PDP-10s, is one of the major artifacts of hacker history.

The earliest version by Crowther and Woods (sometimes known as 350-point Adenture) was ported to C by Jim Gillogly in ’77 just after it first shipped. That has been part of the bsd-games collection forever.

What I have have just received Crowther & Wood’s encouragement to polish up and ship under a modern open-source license is not the Gillogly port; it’s Crowther & Woods’s last version from 1995. It has 18 years of work in it that the Gillogly version doesn’t.

I feel rather as though I’d been given a priceless Old Master painting to restore and display. Behooves me to be careful stripping off the oxidized varnish.

Posted Sun May 14 22:53:33 2017 Tags:
Often there is the need to perform arithmetic on time values. Take as an example the following code which could be part of the inner loop of a server application. I know I wrote code like this in C many times in the past and it was always a pain.
Posted Wed May 10 22:00:00 2017 Tags:

There’s been a lot of public talk about “identity” lately, stimulated by high-profile cases of transsexuality (notably the athlete now named Caitlyn Jenner) and transracialism (Rachel Dolezal). It needs to be said: most of the talk, on all sides of these disputes, has been obvious nonsense – utter drivel that should not have survived five minutes of thought.

I thought we had reached the limit of absurdity with the flap over Rebecca Tuvel’s paper In Defense of Transracialism, about which it can only be said that while Tuvel seems marginally less insane than her attackers, everyone involved in that dispute has obviously been huffing unicorn farts for so long that oxygen no longer reaches their brains in appreciable quantities.

But that’s in a corner of academia where one rather expects postmodernism to have shut down rational thought. In its own way, the following statement in an exudation of mainstream journalism is much sillier, and has finally pushed me into writing on the topic. I quote it not because it’s a unique error but because it’s representative of a very common category mistake.

Thus should there be a weighty presumption against so blocking people, against subordinating them by substituting our judgments about their identity for their own.

This would seem to be a rather uncontroversial point, based on ordinary liberal arguments in favor of tolerance and respect for the dignity of others.

Ah, yes. So, what then would be amiss if I stood up in a public place and claimed to be the Queen of England? Who are you to substitute your judgment about my identity for my own?

There would actually be two different kinds of things wrong with this claim. One is that I can’t grant peerages – the people who administer the English honors system wouldn’t recognize my authority. The other is that the claim to be “Queen” (as opposed, to, say, “Prince-Consort”) implies an observably false claim that I am biologically female.

These criticisms imply a theory of “identity” that is actually coherent and useful. Here it is:

Your “identity” is a set of predictive claims you assert about yourself, mostly (though not entirely) about what kinds of transactions other people can expect to engage in with you.

As an example of an exception to “mostly”, the claim “I am white” implies that I sunburn easily. But usually, an “identity” claim implies the ability and willingness to meet behavioral expectations held by other people. For example, if I describe my “identity” as “male, American, computer programmer, libertarian” I am in effect making an offer that others can expect me to need to shave daily, salute the Stars and Stripes, sling code, and argue for the Non-Aggression Principle as an ethical fundamental.

Thus, identity claims can be false (not cashed out in observed behavior) or fraudulent (intended to deceive). You don’t get to choose your identity; you get to make an offer and it’s up to others whether or not to accept.

There was a very silly news story recently about “Claire”, a transsexual “girl” with a penis who complains that she is rejected by straight guys for ‘having male parts’. Er, how was “she” expecting anything different? By trying to get dates with heterosexual teenage boys using a female presentation, she was making an offer that there is about her person the sort of sexual parts said boys want to play with. Since “she” does not in fact have a vagina, this offer was fraudulent and there’s no wonder the boys rejected it.

More to the point, why is this “girl” treated as anything but a mental case? Leaving aside the entire question of how real transgenderism is as a neuropsychological phenomenon, “she” clearly suffers from a pretty serious disconnect with observable reality. In particular, those delusions about teenage boys…

I can anticipate several objections to this transactional account of identity. One is that is cruel and illiberal to reject an offer of “I claim identity X” if the person claiming feels that identity strongly enough. This is essentially the position of those journalists from The Hill.

To which I can only reply: you can feel an identity as a programmer as strongly as you want, but if you can’t either already sling code or are visibly working hard on repairing that deficiency, you simply don’t make the nut. Cruelty doesn’t enter into this; if I assent to your claim I assist your self-deceit, and if I repeat it I assist you in misleading or defrauding others.

It is pretty easy to see how this same analysis applies to “misgendering” people with the “wrong” pronouns. People who use the term “misgender” generally follow up with claims about the subject’s autonomy and feelings. Which is well enough, but such considerations do not justify being complicit in the deceit of others any more than they do with respect to “I am a programmer”.

A related objection is that I have stolen the concept of “identity” by transactionalizing it. That is, true “identity” is necessarily grounded not in public performance but private feelings – you are what you feel, and it’s somehow the responsibility of the rest of the world to keep up.

But…if I’m a delusional psychotic who feels I’m Napoleon, is it the world’s responsibility to keep up? If I, an overweight clumsy shortish white guy, feel that I’m a tall agile black guy under the skin, are you obligated to choose me to play basketball? Or, instead, are you justified in predicting that I can’t jump?

You can’t base “identity” on a person’s private self-beliefs and expect sane behavior to emerge any more than you can invite everyone to speak private languages and expect communication to happen.

Racial identity is fuzzier than gender identity becuse, leaving aside “white men can’t jump”, it’s at first sight more difficult to tie it to a performance claim. Also, people who are genetically interracial are far more common than physical intersexes. Although this may mean less than you think; it turns out that peoples’ self-ascribed race correlates very accurately with race-associated genetic markers.

Nevertheless, here’s a very simple performance claim that solves the problem: if you are a man or woman who claims racial identity X, and I do too, and we were to marry, can we expect our children to claim racial identity X and, without extraordinary attempts at deceit, be believed?

This test neatly disposes of Rachel Dolezal – it explains not just why most blacks think she’s a fraud but why she’s an actual fraud. To apply it, we don’t even have to adhere to an “essentialist” notion of what race is. But the test becomes stronger if we note that (see link above) a genetic essentialist notion of race is probably justified by the facts. Among other applications, genetic racial identity turns out to matter for medical diagnosticians in assessing vulnerability to various diseases – for example, if you are black but claim to be white, your doctor may seriously underweight the possibility that you have hypertension.

As a culture, we got to the crazy place we’re at now by privileging feelings over facts. The whole mess around “identity” is only one example of this. It’s time to say this plainly: people who privilege feelings over facts are not sane, and the facts always win in the end. Though, unfortunately, often not before the insanity has inflicted a great deal of unnecessary suffering.

Posted Tue May 9 15:27:58 2017 Tags:

[ This blog was crossposted on Software Freedom Conservancy's website. ]

I am honored to be a co-author and editor-in-chief of the most comprehensive, detailed, and complete guide on matters related to compliance of copyleft software licenses such as the GPL. This book, Copyleft and the GNU General Public License: A Comprehensive Tutorial and Guide (which we often call the Copyleft Guide for short) is 155 pages filled with useful material to help everyone understand copyleft licenses for software, how they work, and how to comply with them properly. It is the only document to fully incorporate esoteric material such as the FSF's famous GPLv3 rationale documents directly alongside practical advice, such as the pristine example, which is the only freely published compliance analysis of a real product on the market. The document explains in great detail how that product manufacturer made good choices to comply with the GPL. The reader learns by both real-world example as well as abstract explanation.

However, the most important fact about the Copyleft Guide is not its useful and engaging content. More importantly, the license of this book gives freedom to its readers in the same way the license of the copylefted software does. Specifically, we chose the Creative Commons Attribution Share-Alike 4.0 license (CC BY-SA) for this work. We believe that not just software, but any generally useful technical information that teaches people should be freely sharable and modifiable by the general public.

The reasons these freedoms are necessary seem so obvious that I'm surprised I need to state them. Companies who want to build internal training courses on copyleft compliance for their employees need to modify the materials for that purpose. They then need to be able to freely distribute them to employees and contractors for maximum effect. Furthermore, like all documents and software alike, there are always “bugs”, which (in the case of written prose) usually means there are sections that are fail to communicate to maximum effect. Those who find better ways to express the ideas need the ability to propose patches and write improvements. Perhaps most importantly, everyone who teaches should avoid NIH syndrome. Education and science work best when we borrow and share (with proper license-compliant attribution, of course!) the best material that others develop, and augment our works by incorporating them.

These reasons are akin to those that led Richard M. Stallman to write his seminal essay, Why Software Should Be Free. Indeed, if you reread that essay now — as I just did — you'll see that much of the damage and many of the same problems to the advancement of software that RMS documents in that essay also occur in the world of tutorial documentation about FLOSS licensing. As too often happens in the Open Source community, though, folks seek ways to proprietarize, for profit, any copyrighted work that doesn't already have a copyleft license attached. In the field of copyleft compliance education, we see the same behavior: organizations who wish to control the dialogue and profit from selling compliance education seek to proprietarize the meta-material of compliance education, rather than sharing freely like the software itself. This yields an ironic exploitation, since the copyleft license documented therein exists as a strategy to assure the freedom to share knowledge. These educators tell their audiences with a straight face: Sure, the software is free as in freedom, but if you want to learn how its license works, you have to license our proprietary materials! This behavior uses legal controls to curtail the sharing of knowledge, limits the advancement and improvement of those tutorials, and emboldens silos of know-how that only wealthy corporations have the resources to access and afford. The educational dystopia that these organizations create is precisely what I sought to prevent by advocating for software freedom for so long.

While Conservancy's primary job provides non-profit infrastructure for Free Software projects, we also do a bit of license compliance work as well. But we practice what we preach: we release all the educational materials that we produce as part of the Copyleft Guide project under CC BY-SA. Other Open Source organizations are currently hypocrites on this point; they tout the values of openness and sharing of knowledge through software, but they take their tutorial materials and lock them up under proprietary licenses. I hereby publicly call on such organizations (including but not limited to the Linux Foundation) to license materials such as those under CC BY-SA.

I did not make this public call for liberation of such materials without first trying friendly diplomacy first. Conservancy has been in talks with individuals and staff who produce these materials for some time. We urged them to join the Free Software community and share their materials under free licenses. We even offered volunteer time to help them improve those materials if they would simply license them freely. After two years of that effort, it's now abundantly clear that public pressure is the only force that might work0. Ultimately, like all proprietary businesses, the training divisions of Linux Foundation and other entities in the compliance industrial complex (such as Black Duck) realize they can make much more revenue by making materials proprietary and choosing legal restrictions that forbid their students from sharing and improving the materials after they complete the course. While the reality of this impasse regarding freely licensing these materials is probably an obvious outcome, multiple sources inside these organizations have also confirmed for me that liberation of the materials for the good of general public won't happen without a major paradigm shift — specifically because such educational freedom will reduce the revenue stream around those materials.

Of course, I can attest first-hand that freely liberating tutorial materials curtails revenue. Karen Sandler and I have regularly taught courses on copyleft licensing based on the freely available materials for a few years — most recently in January 2017 at LinuxConf Australia and at at OSCON in a few weeks. These conferences do kindly cover our travel expenses to attend and teach the tutorial, but compliance education is not a revenue stream for Conservancy. While, in an ideal world, we'd get revenue from education to fund our other important activities, we believe that there is value in doing this education as currently funded by our individual Supporters; these education efforts fit withour charitable mission to promote the public good. We furthermore don't believe that locking up the materials and refusing to share them with others fits a mission of software freedom, so we never considered such as a viable option. Finally, given the institutionally-backed FUD that we've continue to witness, we seek to draw specific attention to the fundamental difference in approach that Conservancy (as a charity) take toward this compliance education work. (My recent talk on compliance covered on LWN includes some points on that matter, if you'd like further reading.)


0One notable exception to these efforts was the success of my colleague, Karen Sandler (and others) in convincing the OpenChain project to choose CC-0 licensing. However, OpenChain is not officially part of the LF training curriculum to my knowledge, and if it is, it can of course be proprietarized therein, since CC-0 is not a copyleft license.

Posted Wed Apr 26 00:07:00 2017 Tags:

I support the open letter by Drupal developers protesting the attempted expulsion of Larry Garfield from the Drupal commmunity.

As a Drupal contributor who has never in any respect attempted to tie the project to his beliefs or lifestyle, Garfield deserves the right to be judged by his code alone. That is the hacker way; competence is all that matters, and no irrelevance like skin color or shape of genitals or political beliefs or odd lifestyle preference should be allowed to matter.

That I even need to say this in 2017 is something of a disgrace. The hacker culture already had judge-by-the-code-alone figured out forty years ago when I was a n00b; the only reason it needs to be said now is that there’s been a recent fashion for “social justice” witch hunting which, inevitably, has degenerated into the sort of utter fiasco the Drupal devs are now protesting.

Thomas Paine said it best: “He that would make his own liberty secure, must guard even his enemy from oppression; for if he violates this duty, he establishes a precedent that will reach to himself.”

It doesn’t matter how much you dislike Larry Garfield’s personal kinks. If you don’t defend him now, you may have nobody to defend you when some self-declared commissar of political and sexual correctness – or just a censorious project lead like Dries Buytaert – decides that you should be declared an unperson.

You shall judge by the code alone. That is the only social equilibrium that doesn’t degenerate into an ugly bitchfest with expulsions controlled by whatever happens to be politically on top this week. It was the right community norm forty years ago, and remains so today.

Posted Tue Apr 18 16:26:03 2017 Tags:

My last G+ post reported this:

Something out there kills about one oceangoing ship a week.

It is probably freakishly large waves – well outside the ranges predicted by simple modeling of fluid dynamics and used to set required force-tolerance levels in ship design. Turns out these can be produced by nonlinear interactions in which one crest in a wave train steals energy from its neighbors.

Much more in the video.

So go watch the video – this BBC documentary from 2002 on Rogue Waves. It’s worth your time, and you’ll learn some interesting physics.

As I’m watching, I’m thinking that the really interesting word they’re not using is “soliton”. And then, doing some followup, I learn two things: the solutions to the nonlinear Schrödinger equation that describe rogue waves are labeled “Peregrine solitons”, despite not actually having the non-dissipative property of your classical soliton; and it is now believed that the S.S. Edmund Fitzgerald was probably wrecked by a rogue wave back in ’75.

In a weird way this made it kind of personal for me. I used to joke, back when people knew who he was, that Gordon Lightfoot and I have exactly the same four-note singing range. It is a fact that anything he wrote I can cover effectively; I’ve sung and played The Wreck of the Edmund Fitzgerald many times.

So, I’m texting my friend Phil Salkie (he who taught me to solder, and my reference for the Tinker archetype of hacker) about this, and we started filking. And here’s what eventually came out: Wreck of the Edmund Fitzgerald, the science! version:

The lads in the crew saw that soliton come through
It stove in the hatches and coamings
Her hull broached and tore, she was spillin’ out ore
That rogue put an end to her roamings.

Does anyone know where the Gaussian goes
When the sea heights go all superlinear?
A Schrödinger wave for a watery grave
It’ll drown both the saint and the sinner.

That is all.

Posted Mon Apr 17 00:34:49 2017 Tags:
At the American Chemical Society meetings drug companies disclose recent new drugs to the world. Normally, the chemical structures are already out in the open, often as part of patents. But because these patents commonly discuss many compounds, the disclosures are a big thing.

Now, these disclosure meetings are weird. You will not get InChIKeys (see doi:10.1186/s13321-015-0068-4) or something similar. No, people sit down with paper, manually redraw the structure. Like Carmen Drahl has done in the past. And Bethany Halford has taken over that role at some point. Great work from both! The Chemical & Engineering News has aggregated the tweets into this overview.

Of course, a drug structure disclosure is not complete if it does not lead to deposition in databases. The first thing is to convert the drawings into something machine readable. And thanks to the great work from John May on the Chemistry Development Kit and the OpenSMILES team, I'm happy with this being SMILES. So, we (Chris Southan and me) started a Google Spreadsheet with CCZero data:



I drew the structures in Bioclipse 2.6.2 (which has CDK 1.5.13) and copy-pasted the SMILES and InChIKey into the spreadsheet. Of course, it is essential to get the stereochemistry right. The stereochemistry of the compounds was discussed on Twitter, and we think we got it right. But we cannot be 100% sure. For that, it would have been hugely helpful if the disclosures included the InChIKeys!

As I wrote before, I see Wikidata as a central resource in a web of linked chemical data. So, using the same code I used previously to add disclosures to Wikidata, I created Wikidata items for these compounds, except for one that was already in the database (see the right image). The code also fetches PubChem compound IDs, which are also listed in this spreadsheet.

The Wikidata IDs link to the SQID interface, giving a friendly GUI, one that I actually brought up before too. That said, until people add more information, it may be a bit sparsely populated:


But others are working on this series of disclosures too, and keep an eye on this blog post, as others may follow up with further information!
Posted Fri Apr 14 13:39:00 2017 Tags:

In ancient lore, a molly-guard was a shield to prevent tripping of some Big Red Switch by clumsy or ignorant hands. Originally used of the plexiglass covers improvised for the BRS on an IBM 4341 after a programmer’s toddler daughter (named Molly) frobbed it twice in one day

The Great Beast of Malvern, the computer designed on this blog for performing repository surgery, sits to the left of my desk. This is Zola the cat sitting on it, as he sometimes does to hang out near one of his humans.

What you cannot quite see in that picture is the Power Switch of the Beast, located near the right front corner of the case top – alas, where an errant cat foot can land on it. Dilemma! I do not want to shoo away the Zola, for he is a wonderfully agreeable cat. On the other hand, it is deucedly inconvenient to have one’s machine randomly power-cycled while hacking.

Fortunately, I am a tool-using sophont and there is an elegant solution to this problem.

The solution is an ordinary refrigerator magnet. Being a magnet, it adheres to the ferrous-metal case top just firmly enough not to be easily pushed aside by a cat foot, but is readily moved by human fingers. Here it is in the ‘unsafe’ position:

And in the ‘safe’ position:

EDIT: Thanks to a brilliant comment by andyjpb. I have retitled this post and changed the following lexical item.

Thus, your word of the day: “moggy-guard”. Like a molly-guard, but for cats.

Posted Thu Apr 13 14:32:22 2017 Tags:

This is a higher-level explanation of the inputfd protocol RFC I sent to the list on March 31. Note that this is a first draft, this protocol may never see the light of the day or may be significantly altered before it lands.

First, what is it? inputfd is a protocol for a Wayland compositor to pass a file descriptor (fd) for an input device directly to the client. The client can then read events off this fd and process them, without any additional buffering in between. In the ideal case, the compositor doesn't care (or even notice) that events are flowing between the kernel and the client. Because the compositor sets up the fd, the client does not need any special privileges.

Why is this needed? There are a few input devices that will not be handled by libinput. libinput is the stack for those devices that have a direct interaction with the desktop - mice, keyboards, touchpads, ... But plenty of devices do not want or require desktop interactions. Joysticks/gamepads are the prime example here, but 3D mice or VR input devices also come to mind. These devices don't control the cursor on the desktop, so the compositor doesn't care about the device's events. Note that the current draft only caters for joysticks, 3D mice etc. are TBD.

Why not handle these devices in libinput? Joysticks in particular are so varied that a library like libinput would have to increase the API surface area massively to cater for every possibility. And in the end it is likely that libinput would merely buffer events and pass them on almost as-is anyway. From a maintenance POV - I don't want to deal with that. And from a technical POV - there's little point to have libinput in between anyway. Furthermore, it's already the case that gaming devices are opened directly by the application rather than going through X. So just connecting the clients with the device directly has a advantages.

What does the compositor do? The compositor has two jobs: device filtering and focus management. In the current draft, a client can say "give me all gaming devices" but it's the compositor that decides which device is classified as such (see below for details). Depending on seat assignment or other policies, a client may only see a fraction of the devices currently connected.

Focus management is relatively trivial: the compositor decides that a client has focus now (e.g. by clicking into the window) and hands that client an fd. On focus out (e.g. alt-tab) the fd is revoked and the client cannot read any more events. Rinse, wash, repeat as the focus changes. Note that the protocol does not define how the focus is decided, that task is up to the compositor. Having focus management in the compositor gives us a simple benefit: the client can't hog the device or read events when it's out of focus. This makes it easy to have multiple clients running that need access to the same device without messing up state in a backgrounded client. The security benefit is minimal, few people enter their password with a joystick. But it's still a warm fuzzy feeling to know that a client cannot read events when it's not supposed to.

What devices are applicable devices? This is one of the TBD of the protocol. The compositor needs to know whether a device is a gaming device or not, but the default udev tag ID_INPUT_JOYSTICK is too crude and provides false positives (e.g. some Wacom tablets have that tag set). The conversion currently goes towards having a database that can define this better, but this point is far from decided on.

There are a couple of other points that are still being discussed. If you have knowledge in game (framework) development, do join the discussion, we need input. Personally I am far from a game developer, so I cannot fathom all the weird corner cases we may have to deal with here. Flying blind while designing a protocol is fun, but not so much when you then have to maintain it for the next 10 years. So some navigational input would be helpful.

And just to make sure you join the right discussion, I've disabled comments here :)

Posted Sat Apr 1 06:12:00 2017 Tags:

When you've emptied a cupboard, put masking tape across it, ideally in a colour that's highly visible. This way you immediately see know which ones are finished and which ones still need attention. You won't keep opening the cupboard a million times to check and after the move it takes merely seconds to undo.

Posted Fri Mar 17 02:00:00 2017 Tags:

Was looking at using zstd for backup, and wanted to see the effect of different compression levels. I backed up my (built) bitcoin source, which is a decent representation of my home directory, but only weighs in 2.3GB. zstd -1 compressed it 71.3%, zstd -22 compressed it 78.6%, and here’s a graph showing runtime (on my laptop) and the resulting size:

zstandard compression (bitcoin source code, object files and binaries) times and sizes

For this corpus, sweet spots are 3 (the default), 6 (2.5x slower, 7% smaller), 14 (10x slower, 13% smaller) and 20 (46x slower, 22% smaller). Spreadsheet with results here.

Posted Thu Mar 9 00:53:39 2017 Tags: