The lower-post-volume people behind the software in Debian. (List of feeds.)

The net benefit of having anything that can be called a software development methodology is in inverse proportion to the quality of your developers – and possibly inverse-squared.

Any software-development methodology works well on sufficiently small projects, but all scale very badly to large ones. The good ones scale slightly less badly.

One thing all methodologies tagged “agile” have in common is that they push developers away from the median. Competent developers work better; mediocre developers don’t change; incompetent ones get worse.

Software metrics have their uses. Unfortunately, their principal use is to create the illusion that you know what’s going on.

Structured development methodologies have their uses, too. Unfortunately, their principal use is to create an illusion of control.

Trust simple, crude metrics over complex ones because the simple ones are less brittle. KLOC is best, though a poor best.

Agile development is efficient only in environments where the cost of flag days is low. Otherwise, slow down, and take time to think and write about your architecture.

Good programmers are difficult to control; great ones are nearly impossible to control. Different methodologies require different kinds and degrees of control; match them to your developers wisely.

Process is not a good substitute for judgment; when you have to use it as one, your project is probably too large. Sometimes this is unavoidable, but don’t fool yourself about what that will cost you.

The difference between O(n**2) and O(n log n) really matters. It’s why lots of small teams working coordinated small projects works better than one big team building a monolith.

A million dollars is roughly a 55-gallon oil drum full of five-dollar bills. Large-scale software development is such a difficult and lossy process that it’s like setting fire to several of these oil drums and hoping a usable product flutters out of the smoke. If this drives you to despair, find a different line of work.

Posted Thu Aug 18 04:24:28 2016 Tags:

Last Friday, I gave the first keynote at GUADEC 2016. I was delighted for the invitation from the GNOME Foundation to deliver this talk, which I entitled Confessions of a command line geek: why I don’t use GNOME but everyone else should.

The Chaos Computer Club assisted the GUADEC organizers in recording the talks, so you can see here a great recording of my talk here (and also, the slides). Whether the talk itself is great — that's for you to watch and judge, of course.

The focus of this talk is why the GNOME desktop is such a central component for the future of software freedom. Too often, we assume that the advent of tablets and other mobile computing platforms means the laptop and desktop will disappear. And, maybe the desktop will disappear, but the laptop is going nowhere. And we need a good interface that gives software freedom to the people who use those laptops. GNOME is undoubtedly the best system we have for that task.

There is competition. The competition is now, undeniably, Apple. Unlike Microsoft, who hitherto dominated desktops, Apple truly wants to make beautifully designed, and carefully crafted products that people will not just live with, but actually love. It's certainly possible to love something that harms you, and Apple is so carefully adept creating products that not only refuse to give you software freedom, but Apple goes a step further to regularly invent new ways to gain lock-down control and thwarting modification by their customers.

GUADEC 2016 trip sponsored by the GNOME Foundation!

We have a great challenge before us, and my goal in the keynote was to express that the GNOME developers are best poised to fight that battle and that they should continue in earnest in their efforts, and to offer my help — in whatever way they need it — to make it happen. And, I offer this help even though I readily admit that I don't need GNOME for myself, but we as a community need it to advance software freedom.

I hope you all enjoy the talk, and also check out Werner Koch's keynote, We want more centralization, do we?, which was also about a very important issue. And, finally, I thank the GNOME Foundation for covering my travel expenses for this trip.

Posted Tue Aug 16 10:00:00 2016 Tags:
A new paper in PeerJ (doi:10.7717/peerj.2322) caught my eye for two reasons. First, it's nice to see a paper using the CDK in PeerJ, one of the journals of an innovative, gold Open Access publishing group. Second, that's what I call a graphical abstract (shown on the right)!

The paper describes a collection of Alzheimer-related QSAR models. It primarily uses fingerprints and the PaDeL-Descriptor software (doi:10.1002/jcc.21707) for it particularly. I just checked the (new) PaDeL-Descriptor website and it still seems to use CDK 1.4. The page has the note "Hence, there are some compatibility issues which will only be resolved when PaDEL-Descriptor updates to CDK 1.5.x, which will only happen when CDK 1.5.x becomes the new stable release." and I hope Yap Chun Wei will soon find time to make this update. I had a look at the source code, but with no NetBeans experience and no install instructions, I was unable to compile the source code. AMBIT is now up to speed with CDK 1.5, so the migration should not be too difficult.

Mind you, PaDEL is used quite a bit, so the impact of such an upgrade would be substantial. The Wiley webpage for the article mentions 184 citations, Google Scholar counts 369.

But there is another thing. The authors of the Alzheimer paper compare various fingerprints and the predictive powers of models based on them. I am really looking forward to a paper where the authors compare the same fingerprint (or set of descriptors) but with different CDK versions, particularly CDK 1.4 against 1.5. My guess is that the models based on 1.5 will be better, but I am not entirely convinced yet that the increased stability of 1.5 is actually going to make a significant impact on the QSAR performance... what do you think?


Simeon, S., Anuwongcharoen, N., Shoombuatong, W., Malik, A. A., Prachayasittikul, V., Wikberg, J. E. S., Nantasenamat, C., Aug. 2016. Probing the origins of human acetylcholinesterase inhibition via QSAR modeling and molecular docking. PeerJ 4, e2322+. 10.7717/peerj.2322

Yap, C. W., May 2011. PaDEL-descriptor: An open source software to calculate molecular descriptors and fingerprints. Journal of Computational Chemistry 32 (7), 1466-1474. 10.1002/jcc.21707

Posted Mon Aug 15 10:30:00 2016 Tags:
My Groovy Cheminformatics with the Chemistry Development Kit book sold more than 100 times via Lulu.com now. An older release can be downloaded as CC-BY from Figshare and was "bought" 39 times. That does not really make a living, but does allow me to financially support CiteULike, for example, where you can find all the references I use in the book.

The content of the book is not unique. The book exists for convenience, it explains things around the APIs, gives tips and tricks. In the first place for myself, to help me quickly answer questions on the cdk-user mailing list. This list is a powerful source of answers, and the archive covers 14 years of user support:


One of the design goals of the book was to have many editions allowing me to keep all scripts updated. In fact, all scripts in the book are run each time I make a new release of the book, and, therefore, which each release of the CDK that I make a book release for. That also explains why a new release of the book currently takes quite a bit of time, because there are so many API updates at the moment, as you can read about in the draft CDK 3 paper.

Now, I had for a long time also the plan to make the scripts freely available. However, I never got around to making the website to go with that. I have given up on the idea of a website and now use GitHub. So, you can now, finally, find the scripts for the two active book releases on GitHub. Of course, without the explanations and context. For that you need the book.

Happy CDK hacking!
Posted Sat Aug 13 08:01:00 2016 Tags:

The time has come that I must speak out against the inappropriate rhetoric used by those who (ostensibly) advocate for FLOSS usage in automotive applications.

There was a catalyst that convinced me to finally speak up. I heard a talk today from a company representative of a software supplier for the automotive industry. He said during his talk: putting GPLv3 software in cars will kill people and opening up the source code to cars will cause more harm than good. These statements are completely disingenuous. Most importantly, it ignores the fact that proprietary software in cars is at least equally, if not more, dangerous. At least one person has already been killed in a crash while using a proprietary software auto-control system. Volkswagen decided to take a different route; they decided to kill us all slowly (rather than quickly) by using proprietary software to lie about their emissions and illegally polluting our air.

Meanwhile, there has been not a single example yet about use of GPLv3 software that has harmed anyone. If you have such an example, email it to me and I promise to add it right here to this blog post.

So, to the auto industry folks and vendors who market to/for them: until you can prove that proprietary software assures safety in a way that FLOSS cannot, I will continue to tell you this: in the long and sad tradition of the Therac 25, your proprietary software has killed people, both quickly and slowly, and your attacks on GPLv3 and software freedom are not only unwarranted, they are clearly part of a political strategy to divert attention from your own industry's bad behavior and graft unfair blame onto FLOSS.

As a side note, during the talk's Q&A session, I asked this company's representatives how they assure compliance with the GPLv2 — particularly their compliance with provision of scripts used to control compilation and installation of the executable, which are so often missing for many products, including vehicles. The official answer was: Oh, I don't know. Not only does this company publicly claim security through obscurity is a viable solution, and accuse copyleft advocates of endangering the public safety, they also seem to have not fully learned the lessons of making FLOSS license compliance a clear part of their workflow.

This is, unfortunately, my general impression of the status of the automotive industry.

Posted Sat Aug 13 04:00:00 2016 Tags:

A common issue with users typing on a laptop is that the user's palms will inadvertently get in contact with the touchpad at some point, causing the cursor to move and/or click. In the best case it's annoying, in the worst case you're now typing your password into the newly focused twitter application. While this provides some general entertainment and thus makes the world a better place for a short while, here at the libinput HQ [1] we strive to keep life as boring as possible and avoid those situations.

The best way to avoid accidental input is to detect palm touches and simply ignore them. That works ok-ish on some touchpads and fails badly on others. Lots of hardware is barely able to provide an accurate touch location, let alone enough information to decide whether a touch is a palm. libinput's palm detection largely works by using areas on the touchpad that are likely to be touched by the palms.

The second-best way to avoid accidental input is to disable the touchpad while a user is typing. The libinput marketing department [2] has decided to name this feature "disable-while-typing" (DWT) and it's been in libinput for quite a while. In this post I'll describe how exactly DWT works in libinput.

Back in the olden days of roughly two years ago we all used the synaptics X.Org driver and were happy with it [3]. Disable-while-typing was featured there through the use of a tool called syndaemon. This synaptics daemon [4] has two modes. One was to poll the keyboard state every few milliseconds and check whether a key was down. If so, syndaemon sends a command to the driver to tell it to disable itself. After a timeout when the keyboard state is neutral again syndaemon tells the driver to re-enable itself. This causes a lot of wakeups, especially during those 95% of the time when the user isn't actually typing. Or missed keys if the press + release occurs between two polls. Hence the second mode, using the RECORD extension, where syndaemon opens a second connection to the X server and end checks for key events [5]. If it sees one float past, it tells the driver to disable itself, and so on and so forth. Either way, you had a separate process that did that job. syndaemon had a couple of extra options and features that I'm not going to discuss here, but we've replicated the useful ones in libinput.

libinput has no external process, DWT is integrated into the library with a couple of smart extra features. This is made easier by libinput controlling all the devices, so all keyboard events are passed around internally to the touchpad backend. That backend then decides whether it should stop sending events. And this is where the interesting bits come in.

First, we have different timeouts: if you only hit a single key, the touchpad will re-enable itself quicker than after a period of typing. So if you use the touchpad, hit a key to trigger some UI the pointer only stops moving for a very short time. But once you type, the touchpad disables itself longer. Since your hand is now in a position over the keyboard, moving back to the touchpad takes time anyway so a longer timeout doesn't hurt. And as typing is interrupted by pauses, a longer timeout bridges over those to avoid accidental movement of the cursor.

Second, any touch started while you were typing is permanently ignored, so it's safe to rest the palm on the touchpad while typing and leave it there. But we keep track of the start time of each touch so any touch started after the last key event will work normally once the DWT timeout expires. You may feel a short delay but it should be well in the acceptable range of a tens of ms.

Third, libinput is smart enough to detect which keyboard to pair with. If you have an external touchpad like the Apple Magic Trackpad or a Logitech T650, DWT will never enable on those. Likewise, typing on an external keyboard won't disable the internal touchpad. And in the rare case of two internal touchpads [6], both of them will do the right thing. As of systemd v231 the information of whether a touchpad is internal or external is available in the ID_INPUT_TOUCHPAD_INTEGRATION udev tag and thus available to everyone, not just libinput.

Finally, modifier keys are ignored for DWT, so using the touchpad to do shift-clicks works unimpeded. This also goes for the F-Key row and the numpad if you have any. These keys are usually out of the range of the touchpad anyway so interference is not an issue here. As of today, modifier key combos work too. So hitting Ctrl+S to save a document won't disable the touchpad (or any other modifiers + key combination). But once you are typing DWT activates and if you now type Shift+S to type the letter 'S' the touchpad remains disabled.

So in summary: what we've gained from switching to libinput is one external process less that causes wakeups and the ability to be a lot smarter about when we disable the touchpad. Coincidentally, libinput has similar code to avoid touchpad interference when the trackpoint is in use.

[1] that would be me
[2] also me
[3] uphill, both ways, snow, etc.
[4] nope. this one wasn't my fault
[5] Yes, syndaemon is effectively a keylogger, except it doesn't do any of the "logging" bit a keylogger would be expected to do to live up to its name
[6] This currently happens on some Dell laptops using hid-i2c. We get two devices, one named "DLL0704:01 06CB:76AE Touchpad" or similar and one "SynPS/2 Synaptics TouchPad". The latter one will never send events unless hid-i2c is disabled in the kernel

Posted Fri Aug 5 05:15:00 2016 Tags:

[ This blog was crossposted on Software Freedom Conservancy's website. ]

Monday 1 February 2016 was the longest day of my life, but I don't mean that in the canonical, figurative, and usually negative sense of that phrase. I mean it literally and in a positive way. I woke up that morning Amsterdam in the Netherlands — having the previous night taken a evening train from Brussels, Belgium with my friend and colleague Tom Marble. Tom and I had just spent the weekend at FOSDEM 2016, where he and I co-organize the Legal and Policy Issues DevRoom (with our mutual friends and colleagues, Richard Fontana and Karen M. Sandler).

Tom and I headed over to AMS airport around 07:00 local time, found some breakfast and boarded our flights. Tom was homeward bound, but I was about to do the crazy thing that he'd done in the reverse a few years before: I was speaking at FOSDEM and LinuxConf Australia, back-to-back. In fact, because the airline fares were substantially cheaper this way, I didn't book a “round the world” flight, but instead two back-to-back round-trip tickets. I boarded the plane at AMS at 09:30 that morning (local time), and landed in my (new-ish) hometown of Portland, OR as afternoon there began. I went home, spent the afternoon with my wife, sister-in-law, and dogs, washed my laundry, and repacked my bag. My flight to LAX departed at 19:36 local time, a little after US/Pacific sunset.

I crossed the Pacific ocean, the international dateline, left a day on deposit to pickup on the way back, and after 24 hours of almost literally chasing the sun, I arrived in Melbourne on the morning of Wednesday 3 February, road a shuttle bus, dumped my bags at my room, and arrived just in time for the Wednesday afternoon tea break at LinuxConf Australia 2016 in Geelong.

Nearly everyone who heard this story — or saw me while it was happening — asked me the same question: Why are you doing this?. The five to six people packed in with me in my coach section on the LAX→SYD leg are probably still asking this, because I had an allergic attack of some sort most of the flight and couldn't stop coughing, even with two full bags of Fisherman's Friends over those 15 hours.

But, nevertheless, I gave a simple answer to everyone who questioned my crazy BRU→AMS→PDX→LAX→SYD→MEL itinerary: FOSDEM and LinuxConf AU are two of the most important events on the Free Software annual calendar. There's just no question. I'll write more about FOSDEM sometime soon, but the rest of this post, I'll dedicate to LinuxConf Australia (LCA).

One of my biggest regrets in Free Software is that I was once — and you'll be surprised by this given my story above — a bit squeamish about the nearly 15 hour flight to get from the USA to Australia, and therefore I didn't attend LCA until 2015. LCA began way back in 1999. Keep in mind that, other than FOSDEM, no major, community-organized events have survived from that time. But LCA has the culture and mindset of the kinds of conferences that our community made in 1999.

LCA is community organized and operated. Groups of volunteers each year plan the event. In the tradition of science fiction conventions and other hobbyist activities, groups bid for the conference and offer their time and effort to make the conference a success. They have an annual hand-off meeting to be sure the organization lessons are passed from one committee to the next, and some volunteers even repeat their involvement year after year. For organizational structure, they rely on a non-profit organization, Linux Australia, to assist with handling the funds and providing infrastructure (just like Conservancy does for our member projects and their conferences!).

I believe fully that the success of software freedom and GNU/Linux in particular has not primarily come from companies that allow developers to spend some of their time coding on upstream. Sure, many Free Software projects couldn't survive without that component, but what really makes GNU/Linux, or any Free Software project, truly special is that there's a community of users and developers who use, improve, and learn about the software because it excites and interests them. LCA is one of the few events specifically designed to invite that sort of person to attend, and it has for almost an entire generation stood in stark contrast the highly corporate, for-profit/trade-association events that slowly took over our community in the years that followed LCA's founding. (Remember all those years of LinuxWorld Expo? I wasn't even sad when IDG stopped running it!)

Speaking particularly of earlier this year, LCA 2016 in Geelong, Australia was a particular profound event for me. LCA is one of the few events that accepts my rather political talks about what's happening in Open Source and Free Software, so I gave a talk on Friday 5 February 2016 entitled Copyleft For the Next Decade: A Comprehensive Plan, which was recorded, so you can watch it, or read the LWN article about it. I do warn everyone that the jokes did not go over well (mine never do), so after I finished, I was feeling a bit down that I hadn't made the talk entertaining enough. But then, something amazing happened: people started walking up to me and telling me how important my message was. One individual even came up and told me that he was excited enough that he'd like to match any donation that Software Freedom Conservancy received during LCA 2016. Since it was the last day of the event, I quickly went to one of the organizers, Kathy Reid, and asked if they would announce this match during the closing ceremonies; she agreed. In a matter of just an hour or two, I'd gone from believing my talk had fallen flat to realizing that — regardless of whether I'd presented well — the concepts I discussed had connected with people.

Then, I sat down in the closing session. I started to tear up slightly when the organizers announced the donation match. Within 90 seconds, though, that turned to full tears of joy when the incoming President of Linux Australia, Hugh Blemings, came on stage and said:

[I'll start with] a Software Freedom Conservancy thing, as it turns out. … I can tell that most of you weren't at Bradley's talk earlier on today, but if there is one talk I'd encourage you to watch on the playback later it would be that one. There's a very very important message in there and something to take away for all of us. On behalf of the Council I'd like to announce … that we're actually in the process of making a significant donation from Linux Australia to Software Freedom Conservancy as well. I urge all of you to consider contributing individual as well, and there is much left for us to be done as a community on that front.

I hope that this post helps organizers of events like LCA fully understand how much something like this means to us who run a small charities — and not just with regard to the financial contributions. Knowing that the organizers of community events feel so strongly positive about our work really keeps us going. We work hard and spend much time at Conservancy to serve the Open Source and Free Software community, and knowing the work is appreciated inspires us to keep working. Furthermore, we know that without these events, it's much tougher for us to reach others with our message of software freedom. So, for us, the feeling is mutual: I'm delighted that the Linux Australia and LCA folks feel so positively about Conservancy, and I now look forward to another 15 hour flight for the next LCA.

And, on that note, I chose a strategic time to post this story. On Friday 5 August 2016, the CFP for LCA 2017 closes. So, now is the time for all of you to submit a talk. If you regularly speak at Open Source and Free Software events, or have been considering it, this event really needs to be on your calendar. I look forward to seeing all of you Hobart this January.

Posted Fri Aug 5 00:30:28 2016 Tags:

Mobile developers are working with all kinds of graphics assets and until now, to preview them, we would use an external tool to browse them.

We have developed a plug-in for both Visual Studio and Xamarin Studio that will provide live previews of various assets right into the IDE. It works particularly well for UrhoSharp projects.

The previewer can display in the IDE previews of the following asset types.

  • Static Models (*.mdl files)
  • Materials, Skyboxes, Textures (*.dds)
  • Animations (*.ani)
  • 2D and 3D Particles (*.pex)
  • Urho Prefabs (*.xml if the XML is an Urho Prefab)
  • Scenes (*.xml if the XML is an Urho Scene)
  • SDF Fonts (*.sdf)
  • Post-process effects (*.xml if the XML is an Urho RenderPath)
  • Urho UI layouts (*.xml if the XML is an Urho UI).

For Visual Studio, just download the previwer from the Visual Studio Gallery.

For Xamarin Studio, go to Xamarin Studio > Add-ins go to the "Gallery" tab, and search for "Graphics asset previewer" and install.

Posted Thu Jul 28 22:30:25 2016 Tags:
I'm turning off commenting for my blogs. While I've enjoyed some feedback, the time wasted to moderate spam posts just isn't worth it. Thank you, spammers! :-(
Posted Sat Jul 23 21:11:00 2016 Tags:

As of tonight, I have a new challenge in my life.

Sifu Dale announced tonight that next year he plans to send a team to next year’s Quoshu, a national-level martial-arts competition held every year in northern Maryland in late summer.

I told Sifu I want to go and compete in weapons, at least. He was like, “Well, of course,” as though he’d been expecting it. Which is maybe a little surprising and flattering considering I’ll be pushing 60 then.

It’ll mean serious training for the next year, and maybe a pretty humiliating experience if it turns out I’m too old and slow. But I want to try, because it’s national-level competition against fighters from dozens of different styles, and…frankly, I’m tired of not having any clear idea how good I am. Winning would be nice, but what I really want is to measure myself against a bigger talent pool.children tracker

The thing is, on the limited evidence I have, the possibilities range from “Eric is a clumsy goof who powers through weak opposition just by being a little stronger and more aggressive” to “Eric is a genuinely powerful and clever fighter who even national-level competitors had better take seriously.” It’s really hard for me to tell.

I’ve tended to look pretty good at schools where the style matched my physical capabilities. I was a duffer at aikido and undistinguished at MMA, but me in a style that’s about hitting things or swinging weapons and I shine. You really, really don’t want to be in the way when I strike at full power; I never do it against my training partners because I don’t want to break them. On one occasion at my MMA school when I was practicing short punches against a padded structural beam I vibrated the building. Not kidding!

I also take hits very well when they happen. My sifu often tells new students “Hit him as hard as you can. You can’t hurt him,” which claim is funny because it’s largely true. By the time they can put enough mv**2 on target to make me flinch they’re well beyond being newbies. Generally if he doesn’t say this the student has trained in another striking style before.

On the other hand, I’m only tested against a relatively small population, and it’s not clear that my upper-body strength is the kind of advantage against genuinely skilled opponents that it is when you’re, say, trying to vibrate a building. I’m slow on my feet and my balance is iffy because cerebral palsy. And there are lots of people who can do technique better than me.

If I’m really good, then it’s because (a) I’m strong and tough, (b) I’m aggressive, and (c) I have a kind of low cunning about fighting and do things my opponents don’t expect and aren’t prepared for. I know where my tiger is. An awful lot of people who are better martial technicians than me don’t, and that is a fact.

But I don’t know what percentile this puts me in if you could match me against a hundred people who have also been training for years and are among the best at their schools. In a year and change maybe I will. It’s worth the effort to find out.

Posted Fri Jul 22 11:34:59 2016 Tags:

Don't panic. Of course it isn't. Stop typing that angry letter to the editor and read on. I just picked that title because it's clickbait and these days that's all that matters, right?

With the release of libinput 1.4 and the newest feature to add tablet pad mode switching, we've now finished the TODO list we had when libinput was first conceived. Let's see what we have in libinput right now:

  • keyboard support (actually quite boring)
  • touchscreen support (actually quite boring too)
  • support for mice, including middle button emulation where needed
  • support for trackballs including the ability to use them rotated and to use button-based scrolling
  • touchpad support, most notably:
    • proper multitouch support on touchpads [1]
    • two-finger scrolling and edge scrolling
    • tapping, tap-to-drag and drag-lock (all configurable)
    • pinch and swipe gestures
    • built-in palm and thumb detection
    • smart disable-while-typing without the need for an external process like syndaemon
    • more predictable touchpad behaviours because everything is based on physical units [2]
    • a proper API to allow for kinetic scrolling on a per-widget basis
  • tracksticks work with middle button scrolling and communicate with the touchpad where needed
  • tablet support, most notably:
    • each tool is a separate entity with its own capabilities
    • the pad itself is a separate entity with its own capabilities and events
    • mode switching is exported by the libinput API and should work consistently across callers
  • a way to identify if multiple kernel devices belong to the same physical device (libinput device groups)
  • a reliable test suite
  • Documentation!
The side-effect of libinput is that we are also trying to fix the rest of the stack where appropriate. Mostly this meant pushing stuff into systemd/udev so far, with the odd kernel fix as well. Specifically the udev bits means we
  • know the DPI density of a mouse
  • know whether a touchpad is internal or external
  • fix up incorrect axis ranges on absolute devices (mostly touchpads)
  • try to set the trackstick sensitivity to something sensible
  • know when the wheel click is less/more than the default 15 degrees
And of course, the whole point of libinput is that it can be used from any Wayland compositor and take away most of the effort of implementing an input stack. GNOME, KDE and enlightenment already uses libinput, and so does Canonical's Mir. And some distribution use libinput as the default driver in X through xf86-input-libinput (Fedora 22 was the first to do this). So overall libinput is already quite a success.

The hard work doesn't stop of course, there are still plenty of areas where we need to be better. And of course, new features come as HW manufacturers bring out new hardware. I already have touch arbitration on my todo list. But it's nice to wave at this big milestone as we pass it into the way to the glorious future of perfect, bug-free input. At this point, I'd like to extend my thanks to all our contributors: Andreas Pokorny, Benjamin Tissoires, Caibin Chen, Carlos Garnacho, Carlos Olmedo Escobar, David Herrmann, Derek Foreman, Eric Engestrom, Friedrich Schöller, Gilles Dartiguelongue, Hans de Goede, Jackie Huang, Jan Alexander Steffens (heftig), Jan Engelhardt, Jason Gerecke, Jasper St. Pierre, Jon A. Cruz, Jonas Ådahl, JoonCheol Park, Kristian Høgsberg, Krzysztof A. Sobiecki, Marek Chalupa, Olivier Blin, Olivier Fourdan, Peter Frühberger, Peter Hutterer, Peter Korsgaard, Stephen Chandler Paul, Thomas Hindoe Paaboel Andersen, Tomi Leppänen, U. Artie Eoff, Velimir Lisec.

Finally: libinput was started by Jonas Ådahl in late 2013, so it's already over 2.5 years old. And the git log shows we're approaching 2000 commits and a simple LOCC says over 60000 lines of code. I would also like to point out that the vast majority of commits were done by Red Hat employees, I've been working on it pretty much full-time since 2014 [3]. libinput is another example of Red Hat putting money, time and effort into the less press-worthy plumbing layers that keep our systems running. [4]

[1] Ironically, that's also the biggest cause of bugs because touchpads are terrible. synaptics still only does single-finger with a bit of icing and on bad touchpads that often papers over hardware issues. We now do that in libinput for affected hardware too.
[2] The synaptics driver uses absolute numbers, mostly based on the axis ranges for Synaptics touchpads making them unpredictable or at least different on other touchpads.
[3] Coincidentally, if you see someone suggesting that input is easy and you can "just do $foo", their assumptions may not match reality
[4] No, Red Hat did not require me to add this. I can pretty much write what I want in this blog and these opinions are my own anyway and don't necessary reflect Red Hat yadi yadi ya. The fact that I felt I had to add this footnote to counteract whatever wild conspiracy comes up next is depressing enough.

Posted Wed Jul 20 00:45:00 2016 Tags:

More and more distros are switching to libinput by default. That's a good thing but one side-effect is that the synclient tool does not work anymore [1], it just complains that "Couldn't find synaptics properties. No synaptics driver loaded?"

What is synclient? A bit of history first. Many years ago the only way to configure input devices was through xorg.conf options, there was nothing that allowed for run-time configuration. The Xorg synaptics driver found a solution to that: the driver would initialize a shared memory segment that kept the configuration options and a little tool, synclient (synaptics client), would know about that segment. Calling synclient with options would write to that SHM segment and thus toggle the various options at runtime. Driver and synclient had to be of the same version to know the layout of the segment and it's about as secure as you expect it to be. In 2008 I added input device properties to the server (X Input Extension 1.5 and it's part of 2.0 as well of course). Rather than the SHM segment we now had a generic API to talk to the driver. The API is quite simple, you effectively have two keys (device ID and property number) and you can set any value(s). Properties literally support just about anything but drivers restrict what they allow on their properties and which value maps to what. For example, to enable left-finger tap-to-click in synaptics you need to set the 5th byte of the "Synaptics Tap Action" property to 1.

xinput, a commandline tool and debug helper, has a generic API to change those properties so you can do things like xinput set-prop "device name" "property name" 1 [2]. It does a little bit under the hood but generally it's pretty stupid. You can run xinput set-prop and try to set a value that's out of range, or try to switch from int to float, or just generally do random things.

We were able to keep backwards compatibility in synclient, so where before it would use the SHM segment it would now use the property API, without the user interface changing (except the error messages are now standard Xlib errors complaining about BadValue, BadMatch or BadAccess). But synclient and xinput use the same API to talk to the server and the server can't tell the difference between the two.

Fast forward 8 years and now we have libinput, wrapped by the xf86-input-libinput driver. That driver does the same as synaptics, the config toggles are exported as properties and xinput can read and change them. Because really, you do the smart work by selecting the right property names and values and xinput just passes on the data. But synclient is broken now, simply because it requires the synaptics driver and won't work with anything else. It checks for a synaptics-specific property ("Synaptics Edges") and if that doesn't exists it complains with "Couldn't find synaptics properties. No synaptics driver loaded?". libinput doesn't initialise that property, it has its own set of properties. We did look into whether it's possible to have property-compatibility with synaptics in the libinput driver but it turned out to be a huge effort, flaky reliability at best (not all synaptics options map into libinput options and vice versa) and the benefit was quite limited. Because, as we've been saying since about 2009 - your desktop environment should take over configuration of input devices, hand-written scripts are dodo-esque.

So if you must insist on shellscripts to configure your input devices use xinput instead. synclient is like fsck.ext2, on that glorious day you switch to btrfs it won't work because it was only designed with one purpose in mind.

[1] Neither does syndaemon btw but it's functionality is built into libinput so that doesn't matter.
[2] xinput set-prop --type=int --format=32 "device name" "hey I have a banana" 1 2 3 4 5 6 and congratulations, you've just created a new property for all X clients to see. It doesn't do anything, but you could use those to attach info to devices. If anything was around to read that.

Posted Fri Jul 15 07:47:00 2016 Tags:

xinput is a commandline tool to change X device properties. Specifically, it's a generic interface to change X input driver configuration at run-time, used primarily in the absence of a desktop environment or just for testing things. But there's a feature of xinput that many don't appear to know: it resolves device and property names correctly. So plenty of times you see advice to run a command like this:


xinput set-prop 15 281 1
This is bad advice, it's almost impossible to figure out what this is supposed to do, it depends on the device ID never changing (spoiler: it will) and the property number never changing (spoiler: it will). Worst case, you may suddenly end up setting a different property on a different device and you won't even notice. Instead, just use the built-in name resolution features of xinput:

xinput set-prop "SynPS/2 Synaptics TouchPad" "libinput Tapping Enabled" 1
This command will work regardless of the device ID for the touchpad and regardless of the property number. Plus it's self-documenting. This has been possible for many many years, so please stop using the number-only approach.
Posted Thu Jul 14 05:44:00 2016 Tags:

I wrote a Twitter Bot that tweets real-time updates from FiveThirtyEight's 2016 U.S. presidential election forecast. This is the latest version of the model that correctly predicated 49 of 50 states in the 2008 election and all 50 states in 2012.

Follow @polling_2016 on Twitter!

Details: The system pulls updates from FiveThirtyEight throughout the day, but will only tweet when the model is updated and the probabilities change. It uses the polls-only model. Typically it generates between zero and three tweets a day. It is written using open source software and runs on Linux. This is unaffiliated with FiveThirtyEight or my employer.

Posted Wed Jul 13 17:00:00 2016 Tags:

In an earlier post, I explained how we added graphics tablet pad support to libinput. Read that article first, otherwise this article here will be quite confusing.

A lot of tablet pads have mode-switching capabilities. Specifically, they have a set of LEDs and pressing one of the buttons cycles the LEDs. And software is expected to map the ring, strip or buttons to different functionality depending on the mode. A common configuration for a ring or strip would be to send scroll events in mode 1 but zoom in/out when in mode 2. On the Intuos Pro series tablets that mode switch button is the one in the center of the ring. On the Cintiq 21UX2 there are two sets of buttons, one left and one right and one mode toggle button each. The Cintiq 24HD is even more special, it has three separate buttons on each side to switch to a mode directly (rather than just cycling through the modes).

In the upcoming libinput 1.4 we will have mode switching support in libinput, though modes themselves have no real effect within libinput, it is merely extra information to be used by the caller. The important terms here are "mode" and "mode group". A mode is a logical set of button, strip and ring functions, as interpreted by the compositor or the client. How they are used is up to them as well. The Wacom control panels for OS X and Windows allow mode assignment only to the strip and rings while the buttons remain in the same mode at all times. We assign a mode to each button so a caller may provide differing functionality on each button. But that's optional, having a OS X/Windows-style configuration is easy, just ignore the button modes.

A mode group is a physical set of buttons, strips and rings that belong together. On most tablets there is only one mode group but tablets like the Cintiq 21UX2 and the 24HD have two independently controlled mode groups - one left and one right. That's all there is to mode groups, modes are a function of mode groups and can thus be independently handled. Each button, ring or strip belongs to exactly one mode group. And finally, libinput provides information about which button will toggle modes or whether a specific event has toggled the mode. Documentation and a starting point for which functions to look at is available in the libinput documentation.

Mode switching on Wacom tablets is actually software-controlled. The tablet relies on some daemon running to intercept button events and write to the right sysfs files to toggle the LEDs. In the past this was handled by e.g. a callout by gnome-settings-daemon. The first libinput draft implementation took over that functionality so we only have one process to handle the events. But there are a few issues with that approach. First, we need write access to the sysfs file that exposes the LED. Second, running multiple libinput instances would result in conflicts during LED access. Third, the sysfs interface is decidedly nonstandard and quite quirky to handle. And fourth, the most recent device, the Express Key Remote has hardware-controlled LEDs.

So instead we opted for a two-factor solution: the non-standard sysfs interface will be deprecated in favour of a proper kernel LED interface (/sys/class/leds/...) with the same contents as other LEDs. And second, the kernel will take over mode switching using LED triggers that are set up to cover the most common case - hitting a mode toggle button changes the mode. Benjamin Tissoires is currently working on those patches. Until then, libinput's backend implementation will just pretend that each tablet only has one mode group with a single mode. This allows us to get the rest of the userstack in place and then, once the kernel patches are in a released kernel, switch over to the right backend.

Posted Mon Jul 11 01:28:00 2016 Tags:

The Washington Post is running a story alleging that surveys show gun ownership in the U.S,. is at a 40-year low. I won’t link to it.

This is at the same time gun sales are at record highs.

The WaPo’s explanation, is, basically, that all these guns are being bought by the same fourteen survivalists in Idaho.

Mine is that the number of gun owners with a justified fear that “surveys” are a data-gathering tool for confiscations is also at a record high, and therefore that the number lying to nosy strangers about having no guns is at a record high. videochat

I think there’s a way to discriminate between these cases on the evidence.

It’s not NICS records, because thoise get destroyed after a timeout. Thankfully…

In any consumer market, a reliable way to tell if it’s broadening or narrowing is whether manufacturers’ and retailers product ranges are expanding or contracting. SKUs are expensive; having more complicates everybodies’ supply chains and planning and accounting.

In a broadening market, the variety of consumer preferences is increasing. It makes sense to chase them with product variations. In a narrowing one the opposite is true, and you shed SKUs that no longer carry the overhead of their differentiation.

In early-stage technologies this effect can be masked by the normal culling of product types that happens as a technology stabilizes. There was much more variety in personal computers in 1980 than there is now! But firearms are not like this; they’re a mature technology.

So a productive question to ask is this: is the huge upswing in gun sales being accompanied by a broadening of product ranges? Google-fu did not provide a definite answer, but I can think of several indicators.

A big one is the explosion in sales of aftermarket parts for AR-15 customization. If that’s a sign of a contracting market, I’ll eat the grips on my Kimber. Another is the way new product classes keep coming out and being difficult to buy until gunmakers tool up to meet demand. The most recent case of this I know of was subcompact (3.5″-barrel) .45ACPs.

Open question for my blog regulars: can we find good public measures for SKU diversity in this space?

Posted Sun Jul 3 11:37:51 2016 Tags:

Haven’t been blogging for a while because I’ve been deep in coding and HOWTO-writing. Follows the (slightly edited) text of an email I wrote to the NTPsec devel list that I I think might be of interest to a lot of my audience.

One of the questions I get a lot is: How do you do it? And what is “it”, anyway? The question seems like an inquiry into the mental stance that a systems architect has to have to do his job.

So, um, this is it. If you read carefully, I think you’ll learn a fair bit even if you haven’t a clue about NTP itself.

Today, after a false start yesterday and a correction, I completed a patch sequence that makes a significant structural change to NTP that isn’t just removing cruft.

This is kind of a first. Yes, I’ve made some pretty dramatic changes to the code over the last year, but other than the not-yet-successful TESTFRAME scaffolding they were almost all bug fixes, refactorings, or creative removals. The one exception, JSON reporting from ntpdig, was rather trivial.

[What I didn’t say to the list, because they already know it, is that the code was such a rubble pile that it actually took that year to clean up to the point where a change like this was reasonable to attempt.]

What I’ve succeeded in doing is almost completely removing from the code the assumption that refclock addresses necessarily have the special form 127.127.t.u. The only code that still believes this is in the ntp.conf configuration parser, and the only reason *it* still believes this in order not to break the existing syntax of refclock declarations.

(In fact, clock addresses do still have this form internally, but that is only to avoid surprising older ntpq instances; nothing in the NTPsec code now requires it.)

I’ve also made substantial progress towards eliminating driver-type magic numbers from the code. The table that used to indirect from driver-type numbers to driver-type shortnames is gone; instead, the driver shortname string is what it should be – an element of the driver method table – and there is only one type-number-to-driver indirection, a table in refclock_conf.c.

This is all clearing the decks for a big user-visible change. I’m going to fix the frighteningly awful refclock declaration syntax. Consider this example:

# Uses the shared-memory driver, accepting fixes from a running gpsd
# instance watching one PPS-capable GPS. Accepts in-band GPS time (not
# very good, likely to have jitter in the 100s of milliseconds) on one
# unit, and PPS time (almost certainly good to 1 ms or less) on
# another.  Prefers the latter.
 
# GPS Serial data reference (NTP0)
server 127.127.28.0
fudge 127.127.28.0 refid GPS
 
# GPS PPS reference (NTP1)
server 127.127.28.1 prefer
fudge 127.127.28.1 refid PPS

The misleading “server” keyword for what is actually a reference clock. The magic 127.127.t.u address, which is the only way you *know* it’s a reference clock. Some attributes of the clock being specified in a mystery ‘fudge’ command only tied in by the magic server address. The magic driver type number 28. The fail is strong here. The only excuse for this garbage (and it’s not much of one – Mills was smart enough to know better) is that it was designed decades ago in a more primitive time.

Here’s how I think it should look:

refclock shm unit 0 refid GPS
refclock shm unit 1 prefer refid PPS

No magic IPv4 address, no split syntax, no driver type number (it’s been replaced by the driver shortname “shm”). It should be less work to get the rest of the way to this (while still supporting the old syntax for backward compatibility) than I’ve done already – I’ve already written the grammar, only the glue code still needs doing.

An unobvious benefit of this change is that the driver reference pages are going to become a lot less mystifying. I can still remember how and why my head hurt on first reading them. Removing the magic addresses and mystery numbers will help a lot.

Along the way I learned a lot about how ntpq and mode 6 responses work. (Like NTP in general, it’s an odd combination of elegant structural ideas with an astonishing accumulation of cruft on top.) In order to remove the magic-address assumptions from ntpq I had to add another variable, “displayname”, to the set you get back when you request information about a peer. In effect, ntpd gets to say “*this* is how you should label this peer”, and ntpq uses that to decorate the clock entries in its -p output.

This has the minor downside that new ntpqs will display 127.127.28.0 (rather than “SHM(0)”) when querying Classic ntpd, which doesn’t ship that variable. Oh well…almost everyone disables remote querying anyway. It was the right thing to do; ntpq has no business knowing about driver type numbers.

(Grrrrr…Actually, *nobody* has any business knowing about driver type numbers. Things that have names should be referred to by name. Making humans maintain a level of indirection from names to numbers is perverse, that’s the kind of detail we have computers to track. Or, to put it slightly differently, “1977 called – it wants its ugly kluge back.”)

It’s easy for codebases this size to wind up as huge balls of mud. There are several nearly equivalent ways to describe my job as a systems architect; one of them centers on enforcing proper separation of concerns so collapse-to- mudball is prevented. The changes I’ve just described are a significant step in the good direction.

Posted Sun Jun 26 02:51:12 2016 Tags:

I ran some quick numbers on the last retargeting period (blocks 415296 through 416346 inclusive) which is roughly a week’s worth.

Blocks were full: median 998k mean 818k (some miners blind mining on top of unknown blocks). Yet of the 1,618,170 non-coinbase transactions, 48% were still paying dumb, round fees (like 5000 satoshis). Another 5% were paying

dumbround-numbered per-byte fees (like 80 satoshi per byte).

The mean fee was 24051 satoshi (~16c), the mean fee rate 60 satoshi per byte. But if we look at the amount you needed to pay to get into a block (using the second cheapest tx which got in), the mean was 16.81 satoshis per byte, or about 5c.

tl;dr: It’s like a tollbridge charging vehicles 7c per ton, but half the drivers are just throwing a quarter as they drive past and hoping it’s enough. It really shows fees aren’t high enough to notice, and transactions don’t get stuck often enough to notice. That’s surprising; at what level will they notice? What wallets or services are they using?

Posted Wed Jun 15 03:00:05 2016 Tags:
A few notes on technology-fueled normalization of lynch mobs targeting both the accuser and the accused. #ethics #crime #punishment
Posted Tue Jun 7 01:00:47 2016 Tags:

I’ve been learning more about tinkering with electronics lately, soldering and casemodding and that sort of thing. The major reason for this is NTPsec-related and will be discussed in a near-future post, but here is an early consequence unrelated to that project:

Converting a PS/2 TrackMan Marble to USB

Posted Thu Jun 2 13:42:55 2016 Tags: