The lower-post-volume people behind the software in Debian. (List of feeds.)

And the latest revision:
Things Every Hacker Once Knew.

This time: The Break key. uuencode/uudecode. Why older Internet protocols only assume a 7-bit link. The original meanings of SO/SI. WRU and station ID on teletypes. BITNET and other pre-Internets.

There is one respect in which working on this is changing my historical perspective. The section now titled “WAN time gone: The forgotten pre-Internets” started out just being about UUCP but has gradually expanded to include the BBS scene, commercial timesharing, and academic networks in the period 1978-1996 (and especially 1981-1991).

At the time those of us exposed to more than one of these networks saw mostly differences – differences in capability, differences in addressing schemes, differences in underlying protocols.

Now, twenty years later, I’m finding that it’s the similarities that look more significant. These experiments were all evolving in parallel, offering services that converged over time.

Wide-area TCP/IP was the eventual winner, of course. It’s not hard to see why: being designed for internetworking and not being gated by proprietary IP gave it two insuperable advantages.

Posted Mon Feb 20 19:33:27 2017 Tags:

I’ve been thinking a lot about language design lately. Part of this comes from my quite successful acquisition of Go and my mostly failed attempt to learn Rust. These languages make me question premises I’ve held for a long time, and that questioning has borne some fruit.

In the remainder of this posting I will describe a simple syntax extension in C that could be used to support a trait-centered object system similar to Rust’s (or even Go’s). It is not the whole design, but it is a simple orthogonal piece that could fit with several different possible designs.

Suppose we have two structs named snark and boojum and a function that takes one of each. That is:

struct snark {
    /* stuff goes here */
}
 
struct boojum {
    /* other stuff goes here */
}
 
int conjugate(int x, struct snark *, int y, struct boojum *, int z);
 
struct snark s;
struct boojum b;

Then my proposal is this: under almost all circumstances, the compiler
should automatically perform the following transformations::

s.conjugate(1, 2, &b, 3) -> conjugate(1, &s, 2, &b, 3)
b.conjugate(1, &s, 2, 3) -> conjugate(1, &s, 2, &b, 3)

That is, if the compiler encounters an attempt to evaluate a structure member reference followed by an argument list, for a member that doesn’t exist in the structure (that’s the “almost”), it looks for a visible function with the right name, and then tries to apply it to transform the method-like syntax into an actual method call.

The rule will fail unless the actual arguments match all of the formal argument types of the function in the right order, except that one actual argument is missing. It will also fail unless the one formal without a corresponding actual has any type other than address of the structure for which we are synthesising the method call.

(It has to be address of, otherwise these method calls could never mutate the instance – they’d get an arg-stack copy of it instead.)

I think I got this idea by miscegenating a recent language called Julia with Rust’s method syntax and some dim memories of CLOS. It has a couple of interesting advantages:

1. No front-end modifications or new syntactic tokens are required – it can be implemented entirely as a transformation on abstract syntax after parsing.

2. Entirely upward-compatible with plain C – if you try to feed this to a compiler without the extension it will break noisily at compile time.

3. Same function can be a method of multiple structures without the requirement that you make an articial commitment to which one “really” owns it. (I find this comes up remarkably often.)

Of course, as I mentioned up front, this is not an entire object system. Conspicuous by its absence is inheritance or a trait/interface system. The final advantage of this proposal is that it would not foreclose or complicate any of those alternatives.

Posted Sun Feb 19 20:19:20 2017 Tags:

I’ve shipped another revision of Things Every Hacker Once Knew

The pace of suggested additions and corrections has slowed down a lot; I think this thing is stabilizing.

I gave in and added the one bit of paper-tape lore people have been bugging me to include, about why DEL is 0xb1111111. Learning that the NSA still distributed crypto keys on paper tape until last year smashed that one through my relevance filter.

There’s a short addition on the Trek family of games, a mention of xyzzy, and some minor corrections and typo fixes as well.

Posted Fri Feb 17 15:33:30 2017 Tags:

Heritage games. The legacy of all-uppercase terminals. Where README came from. What “core” is. The ARPANET. Monitoring your computer with a radio. And more…

Things Every Hacker Once Knew

The response to this document has been nothing short of astonishing. More than half of my non-spam mail over the last three weeks has been people writing to suggest additions and corrections or just to thank me. The count of respondents must be over a hundred by now.

A reminder: This document is not intended as a mere ramble down memory lane – it’s much more focused than that. New content has to pass three filters: (1) was common knowledge at the time, (2) has since been forgotten or seems very near forgotten, and (3) might be useful knowledge to younger hackers working on open-source platforms – or is, at the very least, entertaining.

Some matters of potential interest:

1. The largest category of suggested additions that has failed to pass my filters is facts of the form “specified set of obscure ASCII control characters was or is used in specified obscure point-of-sale or financial-transaction protocol”.

2. The most popular single suggested edition that I’ve rejected is that DEL is 7 1-bits because of the paper-tape rubout character. Sorry, guys, I had to draw the chronological/generational line somewhere; just forward of paper tape and punched cards is it.

3. I myself became a direct observer of all this in 1976, about a year after VDTs matured into their final form and at about the earliest point that they were replacing printing terminals in Ivy League computer labs (the rest of the universities lagged this by a few years).

4. I’ve had two mildly protesting emails out of a hundred or so from people who identify as hackers but came up through mainframe-land and never dealt with the whole scene around serial terminals, micros, BBSes, and so forth. Had to tell them that I can only write what I know – and that the ex post facto justification for caring about what I happen to know is that it morphed into today’s open-source culture.

And a final reminder: If gratitude were cash this document would make me a rich man. Since it isn’t, and I have no funding, please express your thanks more tangibly by joining my Patreon feed.

Posted Tue Feb 14 15:13:33 2017 Tags:

I encourage all of you to either listen to or read the transcript of Terry Gross' Fresh Air interview with Joseph Turow about his discussion of his book “The Aisles Have Eyes: How Retailers Track Your Shopping, Strip Your Privacy, And Define Your Power”.

Now, most of you who read my blog know the difference between proprietary and Free Software, and the difference between a network service and software that runs on your own device. I want all of you have a good understanding of that to do a simple thought experiment:

How many of the horrible things that Turow talks about can happen if there is no proprietary software on your IoT or mobile devices?

AFAICT, other than the facial recognition in the store itself that he talked about in Russia, everything he talks about would be mitigated or eliminated completely as a thread if users could modify the software on their devices.

Yes, universal software freedom will not solve all the worlds' problems. But it does solve a lot of them, at least with regard to the bad things the powerful want to do to us via technology.

(BTW, the blog title is a reference to Philip K. Dick's Minority Report, which includes a scene about systems reading people's eyes to target-market to them. It's not the main theme of that particular book, though… Dick was always going off on tangents in his books.)

Posted Tue Feb 14 03:30:00 2017 Tags:

Here’s my first new project in a while – loccount, inspired by David A. Wheeler’s sloccount tool but much faster and with broader language coverage.

I actually wrote this as a learning exercise in the Go language. You can find more details in my NTPsec blog post on Grappling With Go.

If you like it, please remember that open source may be free but my time is not and join my Patreon feed.

Posted Mon Feb 13 23:44:07 2017 Tags:

There are a lot of problems in our society, and particularly in the USA, right now, and plenty of charities who need our support. The reason I continue to focus my work on software freedom is simply because there are so few focused on the moral and ethical issues of computing. Open Source has reached its pinnacle as an industry fad, and with it, a watered-down message: “having some of the source code for some of your systems some of the time is so great, why would you need anything more?”. Universal software freedom is however further from reality than it was even a few years ago. At least a few of us, in my view, must focus on that cause.

I did not post many blog posts about this in 2016. There was a reason for that — more than any other year, work demands at Conservancy have been constant and unrelenting. I enjoy my work, so I don't mind, but blogging becomes low priority when there is a constant backlog of urgent work to support Conservancy's mission and our member projects. It's not just Conservancy's mission, of course, it's my personal one as well.

For our 2016 fundraiser, I wrote last year a blog post entitled “Do You Like What I Do For a Living?”. Last year, so many of you responded, that it not only made it possible for me to continue that work for one more year, but we were able to add our colleague Brett C. Smith to our staff, which brought Conservancy to four full-time staff for the first time. We added a few member projects (and are moving that queue to add more in 2017), and sure enough — the new work plus the backlog of work waiting for another staffer filled Brett's queue just like my, Karen's and Tony's was already filled.

The challenge now is sustaining this staffing level. Many of you came to our aid last year because we were on the brink of needing to reduce our efforts (and staffing) at Conservancy. Thanks to your overwhelming response, we not only endured, but we were able to add one additional person. As expected, though, needs of our projects increased throughout the year, and we again — all four of us full-time staff — must work to our limits to meet the needs of our projects.

Charitable donations are a voluntary activity, and as such they have a special place in our society and culture. I've talked a lot about how Conservancy's Supporters give us a mandate to carry out our work. Those of you that chose to renew your Supporter donations or become new Supporters enable us to focus our full-time efforts on the work of Conservancy.

On the signup and renewal page, you can read about some of our accomplishments in the last year (including my recent keynote at FOSDEM, an excerpt of which is included here). Our work does not follow fads, and it's not particularly glamorous, so only dedicated Supporters like you understand its value. We don't expect to get large grants to meet the unique needs of each of our member projects, and we certainly don't expect large companies to provide very much funding unless we cede control of the organization to their requests (as trade associations do). Even our most popular program, Outreachy, is attacked by a small group of people who don't want to see the status quo of privileged male domination of Open Source and Free Software disrupted.

Supporter contributions are what make Conservancy possible. A year ago, you helped us build Conservancy as a donor-funded organization and stabilize our funding base. I now must ask that you make an annual commitment to renewal — either by renewing your contribution now or becoming a monthly supporter, or, if you're just learning about my work at Conservancy from this blog post, reading up on us and becoming a new Supporter.

Years ago, when I was still only a part-time volunteer at Conservancy, someone who disliked our work told me that I had “invented a job of running Conservancy”. He meant it as an insult, but I take it as a compliment with pride. In fact, between me and my colleague (and our Executive Director) Karen Sandler, we've “invented” a total of four full-time jobs and one part-time one to advance software freedom. You helped us do that with your donations. If you donate again today, your donation will be matched to make the funds go further.

Many have told me this year that they are driven to give to other excellent charities that fight racism, work for civil and immigration rights, and other causes that seem particularly urgent right now. As long as there is racism, sexism, murder, starvation, and governmental oppression in the world, I cannot argue that software freedom should be made a priority above all of those issues. However, even if everyone in our society focused on a single, solitary cause that we agreed was the top priority, it's unlikely we could make quicker progress. Meanwhile, if we all single-mindedly ignore less urgent issues, they will, in time, become so urgent they'll be insurmountable by the time we focus on them.

Industrialized nations have moved almost fully to computer automation for most every daily task. If you question this fact, try to do your job for a day without using any software at all, or anyone using software on your behalf, and you'll probably find it impossible. Then, try to do your job using only Free Software for a day, and you'll find, as I have, that tasks that should take only a few minutes take hours when you avoid proprietary software, and some are just impossible. There are very few organizations that are considering the long-term implications of this slowly growing problem and making plans to build the foundations of a society that doesn't have that problem. Conservancy is one of those few, so I hope you'll realize that long-term value of our lifelong work to defend and expand software freedom and donate.

Posted Mon Feb 13 15:20:00 2017 Tags:

There are a lot of problems in our society, and particularly in the USA, right now, and plenty of charities who need our support. The reason I continue to focus my work on software freedom is simply because there are so few focused on the moral and ethical issues of computing. Open Source has reached its pinnacle as an industry fad, and with it, a watered-down message: “having some of the source code for some of your systems some of the time is so great, why would you need anything more?”. Universal software freedom is however further from reality than it was even a few years ago. At least a few of us, in my view, must focus on that cause.

I did not post many blog posts about this in 2016. There was a reason for that — more than any other year, work demands at Conservancy have been constant and unrelenting. I enjoy my work, so I don't mind, but blogging becomes low priority when there is a constant backlog of urgent work to support Conservancy's mission and our member projects. It's not just Conservancy's mission, of course, it's my personal one as well.

For our 2016 fundraiser, I wrote last year a blog post entitled “Do You Like What I Do For a Living?”. Last year, so many of you responded, that it not only made it possible for me to continue that work for one more year, but we were able to add our colleague Brett Smith to our staff, which brought Conservancy to four full-time staff for the first time. We added a few member projects (and are moving that queue to add more in 2017), and sure enough — the new work plus the backlog of work waiting for another staffer filled Brett's queue just like my, Karen's and Tony's was already filled.

The challenge now is sustaining this staffing level. Many of you came to our aid last year because we were on the brink of needing to reduce our efforts (and staffing) at Conservancy. Thanks to your overwhelming response, we not only endured, but we were able to add one additional person. As expected, though, needs of our projects increased throughout the year, and we again — all four of us full-time staff — must work to our limits to meet the needs of our projects.

Charitable donations are a voluntary activity, and as such they have a special place in our society and culture. I've talked a lot about how Conservancy's Supporters give us a mandate to carry out our work. Those of you that chose to renew your Supporter donations or become new Supporters enable us to focus our full-time efforts on the work of Conservancy.

On the signup and renewal page, you can read about some of our accomplishments in the last year (including my recent keynote at FOSDEM, an excerpt of which is included here). Our work does not follow fads, and it's not particularly glamorous, so only dedicated Supporters like you understand its value. We don't expect to get large grants to meet the unique needs of each of our member projects, and we certainly don't expect large companies to provide very much funding unless we cede control of the organization to their requests (as trade associations do). Even our most popular program, Outreachy, is attacked by a small group of people who don't want to see the status quo of privileged male domination of Open Source and Free Software disrupted.

Supporter contributions are what make Conservancy possible. A year ago, you helped us build Conservancy as a donor-funded organization and stabilize our funding base. I now must ask that you make an annual commitment to renewal — either by renewing your contribution now or becoming a monthly supporter, or, if you're just learning about my work at Conservancy from this blog post, reading up on us and becoming a new Supporter.

Years ago, when I was still only a part-time volunteer at Conservancy, someone who disliked our work told me that I had “invented a job of running Conservancy”. He meant it as an insult, but I take it as a compliment with pride. In fact, between me and my colleague (and our Executive Director) Karen Sandler, we've “invented” a total of four full-time jobs and one part-time one to advance software freedom. You helped us do that with your donations. If you donate again today, your donation will be matched to make the funds go further.

Many have told me this year that they are driven to give to other excellent charities that fight racism, work for civil and immigration rights, and other causes that seem particularly urgent right now. As long as there is racism, sexism, murder, starvation, and governmental oppression in the world, I cannot argue that software freedom should be made a priority above all of those issues. However, even if everyone in our society focused on a single, solitary cause that we agreed was the top priority, it's unlikely we could make quicker progress. Meanwhile, if we all single-mindedly ignore less urgent issues, they will, in time, become so urgent they'll be insurmountable by the time we focus on them.

Industrialized nations have moved almost fully to computer automation for most every daily task. If you question this fact, try to do your job for a day without using any software at all, or anyone using software on your behalf, and you'll probably find it impossible. Then, try to do your job using only Free Software for a day, and you'll find, as I have, that tasks that should take only a few minutes take hours when you avoid proprietary software, and some are just impossible. There are very few organizations that are considering the long-term implications of this slowly growing problem and making plans to build the foundations of a society that doesn't have that problem. Conservancy is one of those few, so I hope you'll realize that long-term value of our lifelong work to defend and expand software freedom and donate.

Posted Mon Feb 13 15:20:00 2017 Tags:

libinput has a couple of features that 'automagically' work on touchpads such as disable-while-typing and the lid switch triggered disabling of touchpads and disabling the touchpad when an external mouse is plugged in [1]. But not all of these features make sense on all touchpads. For example, an Apple Magic Trackpad doesn't need disable-while-typing because unless you have a creative arrangement of input devices [2], the touchpad won't be where your palm is likely to hit it. Likewise, a Logitech T650 connected over a unifying receiver shouldn't get disabled when the laptop lid closes.

For this to work, libinput has some code to figure out whether a touchpad is internal or external. Initially we had some code to detect this but eventually moved this to the ID_INPUT_TOUCHPAD_INTEGRATION property now set by udev's hwdb (systemd 231 and later). Having it in the hwdb makes it quite trivial to override locally where the current rules are insufficient (and until the hwdb is fixed, thanks for filing a bug). We still have the fallback code though in case the tag is missing. On a sufficiently modern distribution, udevadm info /sys/class/input/event4 for your touchpad device node should show something like ID_INPUT_TOUCHPAD_INTEGRATION=internal.

So for any feature that libinput adds for touchpads, we only enable it where it makes sense. That's why your external touchpad doesn't trigger disable-while-typing or the lid switch.

[1] ok, I admit, this is something we should've left to the client, but now we have the feature.
[2] yes, I'm sure there's at least one person out there that uses the touchpad upside down in front of the keyboard and is now angry that libinput doesn't allow arbitrary rotation of the device combined with configurable dwt. I think of you every night I cry myself to sleep.

Posted Fri Feb 10 00:27:00 2017 Tags:

Did I say Things Every Hacker Once Knew was stabilizing? Silly me…

Here’s the 1.7 version. Substantial new material on the BBS scene – this is my answer to the people who have been bugging me to at least mention XMODEM/YMODEM/ZMODEM.

The expository approach I’m taking is to bin all of UUCP, the BBS scene, and commercial dialup services like AOL as parallel contemporaneous attempts to figure out what kind of store-and-forward messaging people actually wanted.

Posted Thu Feb 9 17:16:06 2017 Tags:

The newest version is here.

I think it’s stabilizing. The rate of comments and submissions has been dropping.

Changelog:

     How VDTs explain some heritage programs, and how bitmapped
     displays eventually obsolesced them. Explain why the ADM-3
     was called "dumb" even though it was smart.

There’s also a mention of RS-323 on network gear.

Still nothing about XMODEM/YMODEM/ZMODEM – that’s probably the most requested addition left, but I really don’t see what could be interesting to say about them at this late date.

The reaction to this on my Patreon feed has been impressive. It seems to have driven $300-$400 of new subscriptions.

Posted Wed Feb 8 13:47:30 2017 Tags:

The work on NTPsec is going very well. Unfortunately, since we lost our CII funding in September, my personal situation isn’t.

Ever since the Destroy Full-time Employment Affordable Care Act smashed my wife’s legal job out of existence, we’ve felt under some pressure at Chez Raymond. While Cathy was full-time employed, my earnings from royalties and consulting contracts were what we saved against a rainy day or retirement. Since then we’ve had to live on what I make and occasional legal piecework.

The NTPsec funding came on-stream just as we were starting to worry seriously about money. That kept us out of trouble for a little over a year, but now that’s gone, and the legal piecework market is apparently flooded with attorneys whose jobs were also crushed by the ACA around the same time my wife’s was. It’s a buyer’s market and a lot of what she’s being offered these days is not just scutwork, but would barely cover her expenses after travel.

(For reasons I don’t understand, law seems to have been hit harder by the ACA employer mandate than any other professional field except higher education. I hear non-tenured faculty are even hungrier than lawyers these days.)

So now it’s been five months since either of us has been drawing anything like a salary. We’re burning savings, and Cathy – who grew up poor and thus finds a state of no income viscerally terrifying in a way I do not – has started to look like a shell-shock victim. This is damaging my morale.

The only bright spot is my Patreon feed. Right now it’s pulling $1,392 a month, which is actually rather a lot for Patreon but nowhere near enough to cover our living expenses. That would take about $3,000 a month; the big items are mortgage and medical insurance driven stratospherically up in cost by the same disastrous government bungling that cost my wife her job. Then, of course, there’s income tax; with that I figure we’d need $5K a month to be sure of keeping our heads above water.

Thus, even with the Patreon, we’re fast approaching the point where if someone were to offer me a day job, I’d have to take it. And that would be unfortunate for the long term; the infrastructure work I’m doing and expect to do in the future is tremendously important. Somebody’s going to need to design and field NTPv5 to fully cover the IPv6 transition, and it looks increasingly like that somebody needs to be me.

To everyone reading this who is already contributing to my Patreon feed as an individual, Cathy and I thank you. Someday the rest of the world may be grateful, too.

Now I’m appealing to those of you at corporations and institutions. If you use open-source software (including, say, the browser you’re reading this through) you almost certainly rely on code I wrote every day. Soon, you’ll very likely be relying on my code to provide time service for your computers.

If you have purchase authority for software, please use it to sign your company or institution up as an “Institutional Supporter” of my Patreon account. For $100 per month, you will help ensure that all 50+ of my projects stay under maintenance – and your organization will be listed as a sponsor every time I do a project release after you sign up.

Please don’t wait; do it now. Don’t kid yourself that essential infrastructure like time service is somebody else’s problem; it’s your problem, and this is how you can be sure it gets solved.

Posted Tue Feb 7 11:03:52 2017 Tags:

The 1.5 revision of Things Every Hacker Once Knew is out.

Alas, I had to drop the reference to the Space Cadet keyboard. Turns out it shipped a 32-bit status word and this had nothing to do with 9-bit bytes at all. The indirect reference to the SAIL extended ASCII keyboard is still in.

Patrick Maupin’s revelation about the AT prefix is summarized.

The fact that UUCP was a hack around the old two-tier structure of phone rates is mentioned.

There’s more about TTL serial. Gary Miller, my very hardware-savvy lieutenant and now acting lead on the GPSD project, thinks this didn’t become a common way to ship data off peripherals and daughterboards until after 2000, with GPS chips leading the way. This matches my recollection, but I was pretty oblivious about that sort of thing until the last decade so I don’t consider my recollection very good evidence. Commentary an correction invited.

I’d like to pin down the year cathode-ray tubes disappeared. I know the leading display vendors ceased production in 2005, but I think the transition might have been as much as two years sooner. Again, corrections welcomed.

Posted Mon Feb 6 12:46:56 2017 Tags:

One of my regular commenters on A&D, Random832, wrote the following in response to my inclusion criteria for Things Every Hacker Once Knew.

On “common knowledge at the time”, I think the problem with dismissing things as “fascinating but obscure trivia” means too much exclusion of the real facts behind things that were already forgotten or inaccurately known, and reduces its value as a historical document. There’s also the fact that the intersection between, say, the Lisp and Unix hacker spaces seems to have been tenuous enough to cause some things to have been lost in translation […] – maybe there was never really anything every hacker once knew, just some things some hackers once knew, and other things other hackers once knew, all worth preserving. I think arbitrarily drawing lines around “provides too much historical context” runs the risk of the document being better described as “Things Eric Once Knew”.

I think this comment raises real issues that deserve to be squarely engaged. I’ve omitted one sentence that I think is based on a a factual misunderstanding in order to focus on the large questions.

The first and most obvious point is that this is not a new set of issues. I already had to engage them in connection with the Jargon File a quarter century ago. My answer now, as it was then, is that of course every historian has unconscious biases and selectivities. But if we let that paralyze us we could never write any history at all. This would be a cure worse than the disease that Random832 is pointing out.

There are equally obvious ways to address this problem. One is doing now as I did in 1990: exposing my process to peer review and willingly engaging in a public conversation about both fact and interpretation – not just accepting correction from other witnesses but inviting it. This is more than most historians are willing or even practically able to do.

In this particular situation, I think I am a reliable negative filter (if I didn’t know it, it wasn’t common knowledge) but a less reliable positive one (if I knew it, and thought everyone else did, it was common knowledge).

The reason for this is precisely the position that Random832 sees as a possible weakness: in the crucial period 1976-1985 I lived at the intersection of three subcultures. They were: the Lispers, the Unix minicomputer guys, and the micro hobbyists. It wasn’t a perfectly symmetrical situation; I was deeper in the Unix culture than in in the Lispers and micro guys. But it gave me a breadth of perspective that nobody confined to those individual subcultures could quite match.

But putting it that way also risks overstating the divergence. Because, for example, essentially everybody who learned his chops before the early 1990s either had to deal with serial terminals day to day, or (if they were on a workstation or micro) had recent memory of doing so. There simply was no way for this not to be true, given the technological surround. Similarly for modems in the pre-DSL era.

I am, therefore, actually pretty confident about the ubiquity of most of what’s in there – that it isn’t just things Eric once knew. I get some confirmation of this from the rather high volume of traffic related to the document in my mailbox. The overwhelmingly dominant tone is “Thanks for the trip down memory lane”; I don’t think anybody has yet said “I didn’t know that”, or suggested that I’m too focused on one of the subcultures. I have been alert for such criticism because I understood the issue going in.

Looking over the whole document with this in mind, these are actually just two pieces that worry me that way. One is the ‘graph about the Space Cadet keyboard; the other is the discussion of standard vs. TTL serial. They present opposite problems.

On the one hand, I’m not certain the Space Cadet keyboard was common knowledge at the time. It became so later, but that might have been a retrospective effect of the Jargon File.

On the other hand, I now think the details of the level distinction in RS-232 were common knowledge then – even though I, less hardware-savvy then than now, only vaguely knew of it – but I could still be wrong; that might have been been micro-culture only, which is why I was only vaguely aware.

Whereof one cannot speak, one must remain silent; but if one chooses to bear witness, the hazards of imperfect memory and limited perspective are ever present. The best we can hope for is to mitigate them by cultivating humility and openness.

Posted Sun Feb 5 15:36:00 2017 Tags:

My New Years resolution is to restart blogging.jigsawfish2

Bufferbloat is the most common underlying cause of most variable bad performance on the Internet; it is called “lag” by gamers.

Trying to steer anything the size of the Internet into a better direction is very slow and difficult at best. From the time changes in the upstream operating systems are complete to when consumers can buy new product is typically four years caused by the broken and insecure ecosystem in the embedded device market. Chip vendors, box vendors, I’m looking at you… So much of what is now finally appearing in the market is based on work that is often four years old. Market pull may do what push has not.

The fq_codel & cake work going on in the bufferbloat project is called SQM – “smart queue management.”

See What to do About Bufferbloat for general information. And the DSLReports Speedtest makes it easy to test for bufferbloat. But new commercial products are becoming increasingly available.  Here’s some of them.

Evenroute IQrouter

First up, I’d like call out the Evenroute IQrouter, which has a variant of SQM that deals with “sag”. DSL users have often suffered more than other broadband users, due to bad bloat in the modems compounded by minimal bandwidth, so the DSL version of the IQrouter is particularly welcome.   Often DSL ISP’s seem to have the tendency (seemingly more often than ISPs with other technologies) to under provision their back haul, causing “sag” at different times of day/week.  This makes the static configuration techniques we’ve used in LEDE/OpenWrt SQM ineffective, as you have to give away too much bandwidth if a fixed bandwidth is used.  I love the weasel words “up to” some speed used by many ISPs. It is one thing for your service to degrade for a short period of days or weeks while an ISP takes action to provision more bandwidth to an area; it is another for your bandwidth to routinely vary by large factors for weeks/months and years.

I sent a DSL Evenroute IQrouter to my brother in Pennsylvania recently and arranged for one for a co-worker, and they are working well, and Rich Brown has had similarly good experiences. Evenroute has been working hard to make the installation experience easy. Best yet, is that the IQrouter is autoconfiguring and figures out for you what to do in the face of “sag” in your Internet service, something that may be a “killer feature” if you suffer lots of “sag” from your ISP. The IQrouter is therefore the first “out of the box” device I can recommend to almost anyone, rather than just my geek friends.

The IQRouter does not yet have the very recent wonderful WiFi results of Toke and Dave (more about coming this in a separate post), but has the capability for over the air updates and one hopes debloated WiFi and ATF will come to it reasonably soon. The new WiFi stack is just going upstream into Linux and LEDE/OpenWRT as I write this post.  DSL users seldom have enough bandwidth for the WiFi hop to be the bottleneck; so the WiFi work is much more important for Cable and fiber users at higher bandwidth than for DSL users stuck at low bandwidth.

Ubiquiti Edgerouter

I’ve bought an Ubiquiti Edgerouter X on recommendation of Dave Taht but not yet put it into service. Router performance can be an issue on high end cable or fiber service. It is strictly an Ethernet router, lacking WiFi interfaces; but in my house, where the wiring is down in the basement, that’s what I need.  The Edgerouter starts at around $50; the POE version I bought around $75.

The Edgerouter story is pretty neat – Dave Taht did the backport 2? years back. Ubiquti’s user community jumped all over it and polished it up, adding support to their conf tools and GUI, and Ubiquiti recognized what they had and shipped it as part of their next release.

SQM is available in recent releases of Ubituiti’s Edgerouter firmware.  SQM itself is easy to configure. But the Edgerouter overall requires considerable configuration before it is useful in the home environment, however, and its firmware web interface is aimed at IT people rather than most home users. I intend this to replace my primary router TP-Link Archer C7v2 someday soon, as it is faster than the TP-Link since Comcast keeps increasing my bandwidth without asking me.  I wish the Ubiquiti had a “make me into a home router” wizard that would make it immediately usable for most people, as its price is low enough for some home users to be interested in it.   I believe one can install LEDE/OpenWrt on the Edgerouter, which I may do if I find its IT staff oriented web interface too unusable.

LEDE/OpenWrt and BSD for the Geeks

If you are adventurous enough to reflash firmware, anything runnable on OpenWrt/LEDE of the last few years has SQM available. You take the new LEDE release in testing for a spin. If your router has an Ath9k WiFi chip (or a later version of the Ath10k WiFi chip), or you buy a new router with the right chips in them, you can play with the new WiFi goodness now in LEDE (noted above). There is a very wide variety of home routers that can benefit from reflashing. Its web UI is tolerably decent, better than many commercial vendors I have seen.

WiFi chip vendors should take careful note of the stupendous improvements available in the Linux mac802.11 framework for bufferbloat elimination and air time fairness. If you don’t update to the new interfaces and get your code into LEDE, you’re going to be at a great disadvantage to Atheros in the market.

dd-wrt, asuswrt, ipfire, all long ago added support for SQM. It will be interesting to see how long it takes them to pick up the stunning WiFi work.

The pcengines APU2 is a good “DIY” router for higher speeds. Dave has not yet tried LEDE on it yet, but will. He uses it presently on Ubuntu….

BSD users recently got fq_codel in opnsense, so the BSD crowd are making progress.

Other Out of the Box Devices

The Turris Omnia is particularly interesting for very fast broadband service and can run LEDE as well; but unfortunately,  it seems only available in Europe at this time.  We think the Netduma router has SQM support, though it is not entirely clear what they’ve done; it is a bit pricey for my taste, and I don’t happen to know anyone who has one.

Cable Modems

Cable users may find that upgrading to a new DOCSIS 3.1 modem is helpful (though that does not solve WiFi bufferbloat).  The new DOCSIS 3.1 standard requires AQM.  While I don’t believe PIE anywhere as good as fq_codel (lacking flow queuing), the DOCSIS 3.1 standard at least requires an AQM, and PIE should help and does not require manual upstream bandwidth tuning.  Maybe someday we’ll find some fq_codel (or fq_pie) based cable modems.  Here’s hoping…

Under the Covers, Hidden

Many home routers vendors make bold claims they have proprietary cool features, but these are usually smoke and mirrors. Wireless mesh devices without bufferbloat reduction are particularly suspect and most likely to require manual RF engineering beyond most users. They require very high signal strength and transfer rates to avoid the worst of bufferbloat. Adding lots more routers without debloating and not simultaneously attacking transmit power control is a route to WiFi hell for everyone. The LEDE release is the first to have the new WiFi bits needed to make wireless mesh more practical. No one we know of has been working on minimizing transmit power to reduce interference between mesh nodes. So we are very skeptical of these products.

There are now a rapidly increasing number of products out there with SQM goodness under the covers, sometimes implemented well, and sometimes not so well, and more as the months go by.

One major vendor put support for fq_codel/SQM under the covers of one product using a tradename, promptly won an award, but then started using that tradename on inferior products in their product line that did not have real queue management. I can’t therefore vouch for any product line tradename that does not acknowledge publicly how it works and that the tradename means that it really has SQM under the covers. Once burned, three times shy. That product therefore does not deserve a mention due to the behavior of the vendor. “Bait and switch” is not what anyone needs.

Coming Soon…

We have wind of a number of vendors’ plans who have not quite reached the market, but it is up to them to announce their products.

If you find new products or ISP’s that do really well, let us know, particularly if they actually say what they are doing. We need to start some web pages to keep track of commercial products.


Posted Thu Feb 2 08:00:00 2017 Tags:

I merged a patchset from James Ye today to add support for switch events to libinput, specifically: lid switch events. This feature is scheduled for libinput 1.7.

First, what are switches and how are they different so keys? A key's state is transient with a neutral state of "key is up". The state itself is expected to change frequently. Switches don't always have a defined logical neutral state and the state changes only infrequently. This requires different handling in applications and thus libinput exposes a new interface (and capability) for switches.

The interface itself is trivial. A switch event has two properties, the switch type (e.g. "lid") and the switch state (on/off). See the libinput-debug-events source code for a simple code to print the state and type.

In libinput, we generally try to restrict ourselves to the cases we know how to handle. So in the first iteration, we'll support a single switch event: the lid switch. This is the toggle that changes when you close the lid on your laptop.

But libinput uses this internally too: touchpads are disabled automatically whenever the lid is closed. Indeed, this functionally was the main motivation for this patchset. On a number of devices, we get ghost touches when the lid is closed. Even though the touchpad is unreachable by the user interference with the screen still causes events, moving the pointer in unexpected ways and generally being a nuisance. Some trackpoints suffer from the same issue. But now that libinput knows about the lid switch it can transparently disable the touchpad whenever the lid is closed and thus discard the events.

Lid switches on some devices are unreliable. There are some devices where the lid is permanently closed and other devices where the lid can be closed, but we'll never see the open event. So we change behaviour based on a few factors. After all, no-one likes a dysfunctional touchpad because the lid switch is broken (if you do, seek help). For one, whenever we detect keyboard events while in logically closed state we'll assume that the lid is open after all and adjust state accordingly. Unless the lid switch is reliable, we don't sync the initial state. That's annoying for those who start libinput in closed mode, but it filters out all devices that set the lid switch to "on" and then never change again. On the Surface 3 devices we go even further: we know those devices needs a bit of hand-holding. So whenever we detect activity on the keyboard, we also write the EV_SW/SW_LID state to the device node, thus updating the kernel to be correct again (and thus help everyone else who may be listening).

The exact behaviours will likely change slightly over time as we have to deal with corner-cases one-by-one. But meanwhile, it's even easier for compositors to listen to switch events and users don't have to deal with ghost touches anymore. Many thanks to James Ye for implementing this.

Posted Wed Feb 1 04:51:00 2017 Tags:

In order to read events and modify devices, libinput needs a file descriptor to the /dev/input/event node. But those files are only accessible by the root user. If libinput were to open these directly, we would force any process that uses libinput to have sufficient privileges to open those files. But these days everyone tries to reduce a processes privileges wherever possible, so libinput simply delegates opening and closing the file descriptors to the caller.

The functions to create a libinput context take a parameter of type struct libinput_interface. This is an non-opaque struct with two function pointers: "open_restricted" and "close_restricted". Whenever libinput needs to open or close a file, it calls the respective function. For open_restricted() libinput expects the caller to return an fd with the given flags.

In the simplest case, a caller can merely call open() and close(). This is what the debugging tools do (and the test suite). But obviously this means you have to run those as root. The main wayland compositors (weston, mutter, kwin, ...) instead forward the request to systemd-logind. That then opens the event node and returns the fd which is passed to libinput. And voila, the compositors don't need to run as root, libinput doesn't have to know how the fd is opened and everybody wins. Plus, logind will mute the fd on VT-switch, so we can't leak keyboard events.

In the X.org case it's a combination of the two. When the server runs with systemd-logind enabled, it will open the fd before the driver initialises the device. During the init stage, libinput asks the xf86-input-libinput driver to open the device node. The driver forwards the request to the server which simply returns the already-open fd. When the server runs without systemd-logind, the server opens the file normally with a standard open() call.

So in summary: you can easily run libinput without systemd-logind but you'll have to figure out how to get the required privileges to open device nodes. For anything more than a test or debugging program, I recommend using systemd-logind.

Posted Mon Jan 30 00:00:00 2017 Tags:

We're in the middle of the 1.7 development cycle and one of the features merged already is support for "wheel tilt", i.e. support for devices that don't have a separate horizontal wheel but instead rely on a tilt motion for horizontal event. Now, the way this is handled in the kernel is that the events are sent via REL_WHEEL (or REL_DIAL) so we don't actually need special code in libinput to handle tilt. But libinput tries to to make sense of input devices so the upper layers have a reliable base to build on - and that's why we need tilt-wheels to be handled.

For 'pointer axis' events (i.e. scroll events) libinput provides scroll sources. These specify how the scroll event was generated, allowing a caller to handle things accordingly. A finger-based scroll for example can trigger kinetic scrolling while a mouse wheel would not usually do so. The value for a pointer axis is also dependent on the scroll source - for continuous/finger based scrolling the value is in pixels. For a mouse wheel, the value is in degrees. This obviously doesn't work for a tilt event because degrees don't make sense in this context. So the new axis source is just that, an indicator that the event was caused by a wheel tilt rather than a rotation. Its value matches the default wheel rotation (i.e. 15 degrees) just to make use of it easier.

Of course, a device won't tell us whether it provides a proper wheel or just tilt. So we need a hwdb property and I've added that to systemd's repo. To make this work, set the MOUSE_WHEEL_TILT_HORIZONTAL and/or MOUSE_WHEEL_TILT_VERTICAL property on your hardware and you're off. Yay.

Patches for the wayland protocol have been merged as well, so this is/will be available to wayland clients.

Posted Thu Jan 26 05:03:00 2017 Tags:

We created the ObjectiveSharpie tool to automate the mapping of Objective-C APIs to the .NET world. This is the tool that we use to keep up with Apple APIs.

One of the lesser known features of ObjectiveSharpie, is that it is not limited to binding Objective-C header files. It is also capable of creating definitions for C APIs.

To do this, merely use the "bind" command for ObjectiveSharpie and run it on the header file for the API that you want to bind:

	sharpie bind c-api.h -o binding.cs

The above command will produce the binding.cs that contains the C# definitions for both the native data structures and the C functions that can be invoked.

Since C APIs are ambiguous, in some cases ObjectiveSharpie will generate some diagnostics. In most cases it will flag methods that have to be bound with the [Verify]. This attribute is used as an indicator on your source code that you need to manually audit the binding, perhaps checking the documentation and adjust the P/Invoke signature accordingly.

There are various options that you can pass to the bind command, just invoke sharpie bind to get an up-to-date list of configuration options.

This is how I quickly bootstrapped the TensorFlowSharp binding. I got all the P/Invoke signatures done in one go, and then I started to do the work to surface an idiomatic C# API.

Posted Wed Jan 18 23:21:00 2017 Tags:

This post describes the synclient tool, part of the xf86-input-synaptics package. It does not describe the various options, that's what the synclient(1) and synaptics(4) man pages are for. This post describes what synclient is, where it came from and how it works on a high level. Think of it as a anti-bus-factor post.

Maintenance status

The most important thing first: synclient is part of the synaptics X.Org driver which is in maintenance mode, and superseded by libinput and the xf86-input-libinput driver. In general, you should not be using synaptics anymore anyway, switch to libinput instead (and report bugs where the behaviour is not correct). It is unlikely that significant additional features will be added to synclient or synaptics and bugfixes are rare too.

The interface

synclient's interface is extremely simple: it's a list of key/value pairs that would all be set at the same time. For example, the following command sets two options, TapButton1 and TapButton2:


synclient TapButton1=1 TapButton2=2
The -l switch lists the current values in one big list:

$ synclient -l
Parameter settings:
LeftEdge = 1310
RightEdge = 4826
TopEdge = 2220
BottomEdge = 4636
FingerLow = 25
FingerHigh = 30
MaxTapTime = 180
...
The commandline interface is effectively a mapping of the various xorg.conf options. As said above, look at the synaptics(4) man page for details to each option.

History

A decade ago, the X server had no capabilities to change driver settings at runtime. Changing a device's configuration required rewriting an xorg.conf file and restarting the server. To avoid this, the synaptics X.Org touchpad driver exposed a shared memory (SHM) segment. Anyone with knowledge of the memory layout (an internal struct) and permission to write to that segment could change driver options at runtime. This is how synclient came to be, it was the tool that knew that memory layout. A synclient command would thus set the correct bits in the SHM segment and the driver would use the newly updated options. For obvious reasons, synclient and synaptics had to be the same version to work.

Atoms are 32-bit unsigned integers and created for each property name at runtime. They represent a unique string (the property name) and can be created by applications too. Property name to Atom mappings are global. Once any driver initialises a property by its name (e.g. "Synaptics Tap Actions"), that property and the corresponding Atom will exist globally until the server resets. Atoms unknown to a driver are simply ignored.

8 or so years ago, the X server got support for input device properties, a generic key/value store attached to each input device. The keys are the properties, identified by an "Atom" (see box on the side). The values are driver-specific. All drivers make use of this now, being able to change a property at runtime is the result of changing a property that the driver knows of.

synclient was converted to use properties instead of the SHM segment and eventually the SHM support was removed from both synclient and the driver itself. The backend to synclient is thus identical to the one used by the xinput tool or tools used by other drivers (e.g. the xsetwacom tool). synclient's killer feature was that it was the only tool that knew how to configure the driver, these days it's merely a commandline argument to property mapping tool. xinput, GNOME, KDE, they all do the same thing in the backend.

How synclient works

The driver has properties of a specific name, format and value range. For example, the "Synaptics Tap Action" property contains 7 8-bit values, each representing a button mapping for a specific tap action. If you change the fifth value of that property, you change the button mapping for a single-finger tap. Another property "Synaptics Off" is a single 8-bit value with an allowed range of 0, 1 or 2. The properties are described in the synaptics(4) man page. There is no functional difference between this synclient command:


synclient SynapticsOff=1
and this xinput command

xinput set-prop "SynPS/2 Synaptics TouchPad" "Synaptics Off" 1
Both set the same property with the same calls. synclient uses XI 1.x's XChangeDeviceProperty() and xinput uses XI 2.x's XIChangeProperty() if available but that doesn't really matter. They both fetch the property, overwrite the respective value and send it back to the server.

Pitfalls and quirks

synclient is a simple tool. If multiple touchpads are present it will simply pick the first one. This is a common issue for users with a i2c touchpad and will be even more common once the RMI4/SMBus support is in a released kernel. In both cases, the kernel creates the i2c/SMBus device and an additional PS/2 touchpad device that never sends events. So if synclient picks that device, all the settings are changed on a device that doesn't actually send events. This depends on the order the devices were added to the X server and can vary between reboots. You can work around that by disabling or ignoring the PS/2 device.

synclient is a one-shot tool, it does not monitor devices. If a device is added at runtime, the user must run the command to change settings. If a device is disabled and re-enabled (VT-switch, suspend/resume, ...), the user must run synclient to change settings. This is a major reason we recommend against using synclient, the desktop environment should take care of this. synclient will also conflict with the desktop environment in that it isn't aware when something else changes things. If synclient runs before the DE's init scripts (e.g. through xinitrc), its settings may be overwritten by the DE. If it runs later, it overwrites the DE's settings.

synclient exclusively supports synaptics driver properties. It cannot change any other driver's properties and it cannot change the properties created by the X server on each device. That's another reason we recommend against it, because you have to mix multiple tools to configure all devices instead of using e.g. the xinput tool for all property changes. Or, as above, letting the desktop environment take care of it.

The interface of synclient is IMO not significantly more obvious than setting the input properties directly. One has to look up what TapButton1 does anyway, so looking up how to set the property with the more generic xinput is the same amount of effort. A wrong value won't give the user anything more useful than the equivalent of a "this didn't work".

TL;DR

If you're TL;DR'ing an article labelled "the definitive guide to" you're kinda missing the point...

Posted Tue Jan 3 05:45:00 2017 Tags: