This feed omits posts by jwz. Just 'cause.

*"I’m very afraid": same-sex couples express anxiety about a post-Roe landscape.*

Don't sit their expressing your fear! Organize, recruit, and fight!

Posted Tue Jun 28 19:35:27 2022 Tags:

*Techniques such as adding manure and compost to soils, growing nitrogen-fixing plants between crops, and cultivating a wide range of produce instead of sticking to the same crops, can all increase yields while protecting and improving the natural ecosystems of farms.* The result is to economize on chemical fertiliser.

Posted Tue Jun 28 19:35:27 2022 Tags:

Microplastics in fresh water keep intestinal viruses alive for longer, thus helping them infect more humans.

Posted Tue Jun 28 19:35:27 2022 Tags:

Orange rough reproduce very slowly -- at 55 years old, only 50% are ready to breed. This means fishing must be curtailed to save the species.

Posted Tue Jun 28 19:35:27 2022 Tags:

Oxfam's head calls for a windfall profits tax to help feed poor people.

This tax could certainly do a lot of good for poor people, and I am in favor of it. In the recent past, there was indeed enough food available, so money to help poor people get a bigger share would solve the problem of feeding them. However, given Putin's blockade of Ukraine, is that still true? Would more money for the poor be sufficient now?

Posted Tue Jun 28 19:35:27 2022 Tags:

States may prohibit in-vitro fertilization, because that tends to produce lots of extra embryos that do not get implanted.

Posted Tue Jun 28 19:35:27 2022 Tags:

Congresscritters who asked the corrupter for pardons recognized that their efforts to overturn the election might be criminal.

Posted Tue Jun 28 16:15:11 2022 Tags:

*We Americans are dancing on the Titanic. Our iceberg is not far away.*

I have to respond, "Which one?" There is the right-wing extemists iceberg, the China's world domination iceberg, and behind them the global heating disaster iceberg that can sink the US, China, and the rest of human civilization at the same time.

Posted Tue Jun 28 16:15:11 2022 Tags:
Posted Tue Jun 28 16:15:11 2022 Tags:

The US has returned a Guantanamo prisoner to Afghanistan after a court ruled that the US had no grounds to hold him prisoner.

That is a significant step towards bringing rule of law to the Guantanamo prisoner, but it does not get all the way there. The US must punish the torturers, and hold no one prisoner without a true, fair trial.

Posted Tue Jun 28 16:15:11 2022 Tags:

So, I’ve finally started implementing Pickhardt Payments in Core Lightning (#cln) and there are some practical complications beyond the paper which are worth noting for others who consider this!

In particular, the cost function in the paper cleverly combines the probability of success, with the fee charged by the channel, giving a cost function of:

? log( (ce + 1 ? fe) / (ce + 1)) + ? · fe · fee(e)

Which is great: bigger ? means fees matter more, smaller means they matter less. And the paper suggests various ways of adjusting them if you don’t like the initial results.

But, what’s a reasonable ? value? 1? 1000? 0.00001? Since the left term is the negative log of a probability, and the right is a value in millisats, it’s deeply unclear to me!

So it’s useful to look at the typical ranges of the first term, and the typical fees (the rest of the second term which is not ?), using stats from the real network.

If we want these two terms to be equal, we get:

? log( (ce + 1 ? fe) / (ce + 1)) = ? · fe · fee(e)
=> ? = ? log( (ce + 1 ? fe) / (ce + 1)) / ( fe · fee(e))

Let’s assume that fee(e) is the median fee: 51 parts per million. I chose to look at amounts of 1sat, 10sat, 100sat, 1000sat, 10,000sat, 100,000sat and 1M sat, and calculated the ? values for each channel. It turns out that, for almost all those values, the 10th percentile ? value is 0.125 the median, and the 90th percentile ? value is 12.5 times the median, though for 1M sats it’s 0.21 and 51x, which probably reflects that the median fee is not 51 for these channels!

Nonetheless, this suggests we can calculate the “expected ?” using the median capacity of channels we could use for a payment (i.e. those with capacity >= amount), and the median feerate of those channels. We can then bias it by a factor of 10 or so either way, to reasonably promote certainty over fees or vice versa.

So, in the internal API for the moment I accept a frugality factor, generally 0.1 (not frugal, prefer certainty to fees) to 10 (frugal, prefer fees to certainty), and derive ?:

? = -log((median_capacity_msat + 1 – amount_msat) / (median_capacity_msat + 1)) * frugality / (median_fee + 1)

The median is selected only from the channels with capacity > amount, and the +1 on the median_fee covers the case where median fee turns out to be 0 (such as in one of my tests!).

Note that it’s possible to try to send a payment larger than any channel in the network, using MPP. This is a corner case, where you generally care less about fees, so I set median_capacity_msat in the “no channels” case to amount_msat, and the resulting ? is really large, but at that point you can’t be fussy about fees!

Posted Mon May 9 05:37:40 2022 Tags:

SwiftTermApp: SSH Client for iOS

For the past couple of years, programming in Swift has been a guilty pleasure of mine - I would sneak out after getting the kids to sleep to try out the latest innovations in iOS, such as SwiftUI and RealityKit. I have decided to ship a complete app based on this work, and I put together an SSH client for iOS and iPadOS using my terminal emulator, which I call “SwiftTermApp.”

What it lacks in terms of an original name, it makes up for by having solid fundamentals in place: a comprehensive terminal emulator with all the features you expect from a modern terminal emulator, good support for international input and output, tasteful use of libssh2, keyboard accessories for your Unix needs, storing your secrets in the iOS keychain, extensive compatibility tests, an embrace of the latest and greatest iOS APIs I could find, and is fuzzed and profiled routinely to ensure a solid foundation.

While I am generally pleased with the application for personal use, my goal is to make this app generally valuable to users that routinely use SSH to connect to remote hosts - and nothing brings more clarity to a product than a user’s feedback.

I would love for you to try this app and help me identify opportunities and additional features for it. These are some potential improvements to the app, and I could use your help prioritizing them:

To reduce my development time and maximize my joy, I built this app with SwiftUI and the latest features from Swift and iOS, so it won't work on older versions of iOS. In particular, I am pretty happy with what Swift async enabled me to do, which I hope to blog about soon.

SwiftTermApp is part of a collection of open-source code built around the Unix command line that I have been authoring on and off for the past 15 years. First in C#, now also in Swift. If you are interested in some of the other libraries, check out my UI toolkits for console applications (gui.cs for C#, and TermKit for Swift) and my xterm/vt100 emulator libraries (XtermSharp for C# and SwiftTerm for Swift). I previously wrote about how they came to be.

Update: Join the discussion

For later:

For a few months during the development of the SwiftTerm library, I worked to ensure great compatibility with other terminal emulators using the esctest and vttest. I put my MacPro to good use during the evenings to run the Swift fuzzer and tracked down countless bugs and denial of service errors, used Instruments religiously to improve the performance of the terminal emulator and ensured a good test suite to prevent regressions.

Original intro: For the past few years, I have been hacking on assorted terminal tools in both C# and Swift, including a couple of UI toolkits for console applications (gui.cs for C#, and TermKit for Swift) and xterm/vt100 emulators (XtermSharp for C# and SwiftTerm for Swift). I previously wrote about how they came to be.

Posted Wed Apr 6 14:20:15 2022 Tags:

[ A version of this article was also posted on Software Freedom Conservancy's blog. ]

Bad Early Court Decision for AGPLv3 Has Not Yet Been Appealed

We at Software Freedom Conservancy proudly and vigilantly watch out for your rights under copyleft licenses such as the Affero GPLv3. Toward this goal, we have studied the Neo4j, Inc. v. PureThink, LLC ongoing case in the Northern District of California , and the preliminary injunction appeal decision in the Ninth Circuit Court this month. The case is complicated, and we've seen much understandable confusion in the public discourse about the status of the case and the impact of the Ninth Circuit's decision to continue the trial court's preliminary injunction while the case continues. While it's true that part of the summary judgment decision in the lower court bodes badly for an important provision in AGPLv3§7¶4, the good news is that the case is not over, nor was the appeal (decided this month) even an actual appeal of the decision itself! This lawsuit is far from completion.

A Brief Summary of the Case So Far

The primary case in question is a dispute between Neo4j, a proprietary relicensing company, against a very small company called PureThink, run by an individual named John Mark Suhy. Studying the docket of the case, and a relevant related case, and other available public materials, we've come to understand some basic facts and events. To paraphrase LeVar Burton, we encourage all our readers to not take our word (or anyone else's) for it, but instead take the time to read the dockets and come to your own conclusions.

After canceling their formal, contractual partnership with Suhy, Neo4j alleged multiple claims in court against Suhy and his companies. Most of these claims centered around trademark rights regarding “Neo4j” and related marks. However, the claims central to our concern relate to a dispute between Suhy and Neo4j regarding Suhy's clarification in downstream licensing of the Enterprise version that Neo4j distributed.

Specifically, Neo4j attempted to license the codebase under something they (later, in their Court filings) dubbed the “Neo4j Sweden Software License” — which consists of a LICENSE.txt file containing the entire text of the Affero General Public License, version 3 (“AGPLv3”) (a license that I helped write), and the so-called “Commons Clause” — a toxic proprietary license. Neo4j admits that this license mash-up (if legitimate, which we at Software Freedom Conservancy and Suhy both dispute), is not an “open source license”.

There are many complex issues of trademark and breach of other contracts in this case; we agree that there are lots of interesting issues there. However, we focus on the matter of most interest to us and many FOSS activists: Suhy's permissions to remove of the “Commons Clause”. Neo4j accuses Suhy of improperly removing the “Commons Clause” from the codebase (and subsequently redistributing the software under pure AGPLv3) in paragraph 77 of their third amended complaint. (Note that Suhy denied these allegations in court — asserting that his removal of the “Commons Clause” was legitimate and permitted.

Neo4j filed for summary judgment on all the issues, and throughout their summary judgment motion, Neo4j argued that the removal of the “Commons Clause” from the license information in the repository (and/or Suhy's suggestions to others that removal of the “Commons Clause” was legitimate) constituted behavior that the Court should enjoin or otherwise prohibit. The Court partially granted Neo4j's motion for summary judgment. Much of that ruling is not particularly related to FOSS licensing questions, but the section regarding licensing deeply concerns us. Specifically, to support the Court's order that temporarily prevents Suhy and others from saying that the Neo4j Enterprise edition that was released under the so-called “Neo4j Sweden Software License” is a “free and open source” version and/or alternative to proprietary-licensed Neo4j EE, the Court held that removal of the “Commons Clause” was not permitted. (BTW, the court confuses “commercial” and “proprietary” in that section — it seems they do not understand that FOSS can be commercial as well.)

In this instance, we're not as concerned with the names used for the software; as much as the copyleft licensing question — because it's the software's license, not its name, that either assures or prevents users to exercise their fundamental software rights. Notwithstanding our disinterest in the naming issue, we'd all likely agree that — if “AGPLv3 WITH Commons-Clause” were a legitimate form of licensing — such a license is not FOSS. The primary issue, therefore, is not about whether or not this software is FOSS, but whether or not the “Commons Clause” can be legitimately removed by downstream licensees when presented with a license of “AGPLv3 WITH Commons-Clause”. We believe the Court held incorrectly by concluding that Suhy was not permitted to remove the “Commons Clause”. “Their order that enjoins Suhy from calling the resulting code “FOSS” — even if it's a decision that bolsters a minor goal of some activists — is problematic because the underlying holding (if later upheld on appeal) could seriously harm FOSS and copyleft.

The Confusion About the Appeal

Because this was an incomplete summary judgment and the case is ongoing, the injunction against Suhy's on making such statements is a preliminary injunction, and cannot be made permanent until the case actually completes in the trial court. The decision by the Ninth Circuit appeals court regarding this preliminary injunction has been widely reported by others as an “appeal decision” on the issue of what can be called “open source”. However, this is not an appeal of the entire summary judgment decision, and certainly not an appeal of the entire case (which cannot even been appealed until the case completes). The Ninth Circuit decision merely affirms that Suhy remains under the preliminary injunction (which prohibits him and his companies from taking certain actions and saying certain things publicly) while the case continues. In fact, the standard that an appeals Court uses when considering an appeal of a preliminary injunction differs from the standard for ordinary appeals. Generally speaking, appeals Courts are highly deferential to trial courts regarding preliminary injunctions, and appeals of actual decisions have a much more stringent standard.

The Affero GPL Right to Restriction Removal

In their partial summary judgment ruling, the lower Court erred because they rejected an important and (in our opinion) correct counter-argument made by Suhy's attorneys. Specifically, Suhy's attorneys argued that Neo4j's license expressly permitted the removal of the “Commons Clause” from the license. AGPLv3 was, in fact, drafted to permit such removal in this precise fact pattern.

Specifically, the AGPLv3 itself has the following provisions (found in AGPLv3§0 and AGPLv3§7¶4):

  • “This License” refers to version 3 of the GNU Affero General Public License.
  • “The Program” refers to any copyrightable work licensed under this License. Each licensee is addressed as “you”.
  • If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.

That last term was added to address a real-world, known problem with GPLv2. Frequently throughout the time when GPLv2 was the current version, original copyright holders and/or licensors would attempt to license work under the GPL with additional restrictions. The problem was rampant and caused much confusion among licensees. As an attempted solution, the FSF (the publisher of the various GPL's) loosened its restrictions on reuse of the text of the GPL — in hopes that would provide a route for reuse of some GPL text, while also avoiding confusion for licensees. Sadly, many licensors continued to take the confusing route of using the entire text a GPL license with an additional restriction — attached either before or after, or both. Their goals were obvious and nefarious: they wanted to confuse the public into “thinking” the software was under the GPL, but in fact restrict certain other activities (such as commercial redistribution). They combined this practice with proprietary relicensing (i.e., a sole licensor selling separate proprietary licenses while releasing a (seemingly FOSS) public version of the code as demoware for marketing). Their goal is to build on the popularity of the GPL, but in direct opposition to the GPL's policy goals; they manipulate the GPL to open-wash bad policies rather than give actual rights to users. This tactic even permitted bad actors to sell “gotcha” proprietary licenses to those who were legitimately confused. For example, a company would look for users operating commercially with the code in compliance with GPLv2, but hadn't noticed the company's code had the statement: “Licensed GPLv2, but not for commercial use”. The user had seen GPLv2, and knew from its brand reputation that it gave certain rights, but hadn't realized that the additional restriction outside of the GPLv2's text might actually be valid. The goal was to catch users in a sneaky trap.

Neo4j tried to use the AGPLv3 to set one of those traps. Neo4j, despite the permission in the FSF's GPL FAQ to “use the GPL terms (possibly modified) in another license provided that you call your license by another name and do not include the GPL preamble”, left the entire AGPLv3 intact as the license of the software — adding only a note at the front and at the end. However, their users can escape the trap, because GPLv3 (and AGPLv3) added a clause (which doesn't exist in GPLv2) to defend users from this. Specifically, AGPLv3§7¶4 includes a key provision to help this situation.

Specifically, the clause was designed to give more rights to downstream recipients when bad actors attempt this nasty trick. Indeed, I recall from my direct participation in the A/GPLv3 drafting that this provision was specifically designed for the situation where the original, sole copyright holder/licensor0 added additional restrictions. And, I'm not the only one who recalls this. Richard Fontana (now a lawyer at IBM's Red Hat, but previously legal counsel to the FSF during the GPLv3 process), wrote on a mailing list1 in response to the Neo4j preliminary injunction ruling:

For those who care about anecdotal drafting history … the whole point of the section 7 clause (“If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.”) was to address the well known problem of an original GPL licensor tacking on non-GPL, non-FOSS, GPL-norm-violating restrictions, precisely like the use of the Commons Clause with the GPL. Around the time that this clause was added to the GPLv3 draft, there had been some recent examples of this phenomenon that had been picked up in the tech press.

Fontana also pointed us to the FSF's own words on the subject, written during their process of drafting this section of the license (emphasis ours):

Unlike additional permissions, additional requirements that are allowed under subsection 7b may not be removed. The revised section 7 makes clear that this condition does not apply to any other additional requirements, however, which are removable just like additional permissions. Here we are particularly concerned about the practice of program authors who purport to license their works under the GPL with an additional requirement that contradicts the terms of the GPL, such as a prohibition on commercial use. Such terms can make the program non-free, and thus contradict the basic purpose of the GNU GPL; but even when the conditions are not fundamentally unethical, adding them in this way invariably makes the rights and obligations of licensees uncertain.

While the intent of the original drafter of a license text is not dispositive over the text as it actually appears in the license, all this information was available to Neo4j as they drafted their license. Many voices in the community had told them that provision in AGPLv3§3¶4 was added specifically to prevent what Neo4j was trying to do. The FSF, the copyright holder of the actual text of the AGPLv3, also publicly gave Neo4j permission to draft a new license, using any provisions they like from AGPLv3 and putting them together in a new way. But Neo4j made a conscious choice to not do that, but instead constructed their license in the exact manner that allowed Suhy's removal of the “Commons Clause”.

In addition, that provision in AGPLv3§3¶4 has little meaning if it's not intended to bind the original licensor! Many other provisions (such as AGPLv3§10¶3) protect the users against further restrictions imposed later in the distribution chain of licensees. This clause was targeted from its inception against the exact, specific bad behavior that Neo4j did here.

We don't dispute that copyright and contract law give Neo4j authority to license their work under any terms they wish — including terms that we consider unethical or immoral. In fact, we already pointed out above that Neo4j had permission to pick and choose only some text from AGPLv3. As long as they didn't use the name “Affero”, “GNU” or “General Public” or include any of the Preamble text in the name/body of their license — we'd readily agree that Neo4j could have put together a bunch of provisions from the AGPLv3, and/or the “Commons Clause”, and/or any other license that suited their fancy. They could have made an entirely new license. Lawyers commonly do share text of licenses and contracts to jump-start writing new ones. That's a practice we generally support (since it's sharing a true commons of ideas freely — even if the resulting license might not be FOSS).

But Neo4j consciously chose not to do that. Instead, they license their software “subject to the terms of the GNU AFFERO GENERAL PUBLIC LICENSE Version 3, with the Commons Clause”. (The name “Neo4j Sweden Software License” only exists in the later Court papers, BTW, not with “The Program” in question.) Neo4j defines “This License” to mean “version 3 of the GNU Affero General Public License.”. Then, Neo4j tells all licensees that “If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term”. Yet, after all that, Neo4j had the audacity to claim to the Court that they didn't actually mean that last sentence, and the Court rubber-stamped that view.

Simply put, the Court erred when it said: “Neither of the two provisions in the form AGPLv3 that Defendants point to give licensees the right to remove the information at issue.”. The Court then used that error as a basis for its ruling to temporarily enjoin Suhy from stating that software with “Commons Clause” removed by downstream is “free and open source”, or tell others that he disagrees with the Court's (temporary) conclusion about removing the “Commons Clause” in this situation.

What Next?

The case isn't over. The lower Court still has various issues to consider — including a DMCA claim regarding Suhy's removal of the “Commons Clause”. We suspect that's why the Court only made a preliminary injunction against Suhy's words, and did not issue an injunction against the actual removal of the clause! The issue as to whether the clause can be removed is still pending, and the current summary judgment decision doesn't address the DMCA claim from Neo4j's complaint.

Sadly, the Court has temporarily enjoined Suhy from “representing that Neo4j Sweden AB’s addition of the Commons Clause to the license governing Neo4j Enterprise Edition violated the terms of AGPL or that removal of the Commons Clause is lawful, and similar statements”. But they haven't enjoined us, and our view on the matter is as follows:

Clearly, Neo4j gave explicit permission, pursuant to the AGPLv3, for anyone who would like to to remove the “Commons Clause” from their LICENSE.txt file in version 3.4 and other versions of their Enterprise edition where it appears. We believe that you have full permission, pursuant to AGPLv3, to distribute that software under the terms of the AGPLv3 as written. In saying that, we also point out that we're not a law firm, our lawyers are not your lawyers, and this is not legal advice. However, after our decades of work in copyleft licensing, we know well the reason and motivations of this policy in the license (describe above), and given the error by the Court, it's our civic duty to inform the public that the licensing conclusions (upon which they based their temporary injunction) are incorrect.

Meanwhile, despite what you may have read last week, the key software licensing issues in this case have not been decided — even by the lower Court. For example, the DMCA issue is still before the trial court. Furthermore, if you do read the docket of this case, it will be obvious that neither party is perfect. We have not analyzed every action Suhy took, nor do we have any comment on any action by Suhy other than this: we believe that Suhy's removal of the “Commons Clause” was fully permitted by the terms of the AGPLv3, and that Neo4j gave him that permission in that license. Suhy also did a great service to the community by taking action that obviously risked litigation against him. Misappropriation and manipulation of the strongest and most freedom-protecting copyleft license ever written to bolster a proprietary relicensing business model is an affront to FOSS and its advancement. It's even worse when the Courts are on the side of the bad actor. Neo4j should not have done this.

Finally, we note that the Court was rather narrow on what it said regarding the question of “What Is Open Source?”. The Court ruled that one individual and his companies — when presented with ambiguous licensing information in one part of a document, who then finds another part of the document grants permission to repair and clarify the licensing information, and does so — is temporarily forbidden from telling others that the resulting software is, in fact, FOSS, after making such a change. The ruling does not set precedent, nor does it bind anyone other than the Defendants as to what they can or cannot say is FOSS, which is why we can say it is FOSS, because the AGPLv3 is an OSI-approved license and the AGPLv3 permits removal of the toxic “Commons Clause” in this situation.

We will continue to follow this case and write further when new events occur..

0 We were unable to find anywhere in the Court record that shows Neo4j used a Contributor Licensing Agreement (CLA) or Copyright Assignment Agreement (©AA) that sufficiently gave them exclusive rights as licensor of this software. We did however find evidence online that Neo4j accepted contributions from others. If Neo4j is, in fact, also a licensor of others' AGPLv3'd derivative works that have been incorporated into their upstream versions, then there are many other arguments (in addition to the one presented herein) that would permit removal of the “Commons Clause”. This issue remains an open question of fact in this case.

1 Fontana made these statements on a mailing list governed by an odd confidentiality rule called CHR (which was originally designed for in-person meetings with a beginning and an end, not a mailing list). Nevertheless, Fontana explicitly waived CHR (in writing) to allow me to quote his words publicly.

Posted Wed Mar 30 00:00:00 2022 Tags:

Last year the Bacting paper was published (doi:10.21105/joss.02558) and later Charles wrapped this in the pybacting package so Bioclipse scripts can be run in Python. We discussed how to update the A lot of Bioclipse Scripting Language examples ebook with Python examples. I already tested earlier if the scripts worked in Google Colab (it did), but I had not gotten to adding a link to the ebook to open a script in Colab. Until today. Here's an example code snippet from the book:

Clicking the Open in Google Colab link resulting in this page:

With thanks to this Python to Jupyter notebook convertor!

Posted Sun Mar 20 13:00:00 2022 Tags:

A quick reminder: libei is the library for emulated input. It comes as a pair of C libraries, libei for the client side and libeis for the server side.

libei has been sitting mostly untouched since the last status update. There are two use-cases we need to solve for input emulation in Wayland - the ability to emulate input (think xdotool, or Synergy/Barrier/InputLeap client) and the ability to capture input (think Synergy/Barrier/InputLeap server). The latter effectively blocked development in libei [1], until that use-case was sorted there wasn't much point investing too much into libei - after all it may get thrown out as a bad idea. And epiphanies were as elusive like toilet paper and RATs, so nothing much get done. This changed about a week or two ago when the required lightbulb finally arrived, pre-lit from the factory.

So, the solution to the input capturing use-case is going to be a so-called "passive context" for libei. In the traditional [2] "active context" approach for libei we have the EIS implementation in the compositor and a client using libei to connect to that. The compositor sets up a seat or more, then some devices within that seat that typically represent the available screens. libei then sends events through these devices, causing input to be appear in the compositor which moves the cursor around. In a typical and simple use-case you'd get a 1920x1080 absolute pointer device and a keyboard with a $layout keymap, libei then sends events to position the cursor and or happily type away on-screen.

In the "passive context" <deja-vu> approach for libei we have the EIS implementation in the compositor and a client using libei to connect to that. The compositor sets up a seat or more, then some devices within that seat </deja-vu> that typically represent the physical devices connected to the host computer. libei then receives events from these devices, causing input to be generated in the libei client. In a typical and simple use-case you'd get a relative pointer device and a keyboard device with a $layout keymap, the compositor then sends events matching the relative input of the connected mouse or touchpad.

The two notable differences are thus: events flow from EIS to libei and the devices don't represent the screen but rather the physical [3] input devices.

This changes libei from a library for emulated input to an input event transport layer between two processes. On a much higher level than e.g. evdev or HID and with more contextual information (seats, devices are logically abstracted, etc.). And of course, the EIS implementation is always in control of the events, regardless which direction they flow. A compositor can implement an event filter or designate key to break the connection to the libei client. In pseudocode, the compositor's input event processing function will look like this:

function handle_input_events():
real_events = libinput.get_events()
for e in real_events:
if input_capture_active:

emulated_events = eis.get_events_from_active_clients()
for e in emulated_events:
Not shown here are the various appropriate filters and conversions in between (e.g. all relative events from libinput devices would likely be sent through the single relative device exposed on the EIS context). Again, the compositor is in control so it would be trivial to implement e.g. capturing of the touchpad only but not the mouse.

In the current design, a libei context can only be active or passive, not both. The EIS context is both, it's up to the implementation to disconnect active or passive clients if it doesn't support those.

Notably, the above only caters for the transport of input events, it doesn't actually make any decision on when to capture events. This handled by the CaptureInput XDG Desktop Portal [4]. The idea here is that an application like Synergy/Barrier/InputLeap server connects to the CaptureInput portal and requests a CaptureInput session. In that session it can define pointer barriers (left edge, right edge, etc.) and, in the future, maybe other triggers. In return it gets a libei socket that it can initialize a libei context from. When the compositor decides that the pointer barrier has been crossed, it re-routes the input events through the EIS context so they pop out in the application. Synergy/Barrier/InputLeap then converts that to the global position, passes it to the right remote Synergy/Barrier/InputLeap client and replays it there through an active libei context where it feeds into the local compositor.

Because the management of when to capture input is handled by the portal and the respective backends, it can be natively integrated into the UI. Because the actual input events are a direct flow between compositor and application, the latency should be minimal. Because it's a high-level event library, you don't need to care about hardware-specific details (unlike, say, the inputfd proposal from 2017). Because the negotiation of when to capture input is through the portal, the application itself can run inside a sandbox. And because libei only handles the transport layer, compositors that don't want to support sandboxes can set up their own negotiation protocol.

So overall, right now this seems like a workable solution.

[1] "blocked" is probably overstating it a bit but no-one else tried to push it forward, so..
[2] "traditional" is probably overstating it for a project that's barely out of alpha development
[3] "physical" is probably overstating it since it's likely to be a logical representation of the types of inputs, e.g. one relative device for all mice/touchpads/trackpoints
[4] "handled by" is probably overstating it since at the time of writing the portal is merely a draft of an XML file

Posted Fri Mar 4 04:30:00 2022 Tags:


In late 2005 I joined Google. The interviews took a surprising long time, which is a tale for another time. Today I want to tell a story that happened in one of my first weeks on campus.

In the main building was an impressive staircase going up to the second floor. Somewhere near the top was a spacious office. A very important engineer worked there. I checked the name on the door and realized I knew him: he had been a grad student from the UK who had spent some time visiting our research group (the Amoeba project) at CWI in Amsterdam in the early '90s.

Happy to find someone I knew long ago, one day I knocked on the door and introduced myself. Yes, he remembered me too, but my delight was soon over. Not only was Python the bane of Mike's existence at Google (he detested everything that wasn't C++), but the one memory from his stay in Amsterdam that stood out was about a time I had given him a ride across town on the back of my bike: "Worst ride of my life."
Posted Tue Mar 1 06:19:00 2022 Tags:

After roughly 20 years and counting up to 0.40 in release numbers, I've decided to call the next version of the xf86-input-wacom driver the 1.0 release. [1] This cycle has seen a bulk of development (>180 patches) which is roughly as much as the last 12 releases together. None of these patches actually added user-visible features, so let's talk about technical dept and what turned out to be an interesting way of reducing it.

The wacom driver's git history goes back to 2002 and the current batch of maintainers (Ping, Jason and I) have all been working on it for one to two decades. It used to be a Wacom-only driver but with the improvements made to the kernel over the years the driver should work with most tablets that have a kernel driver, albeit some of the more quirky niche features will be more limited (but your non-Wacom devices probably don't have those features anyway).

The one constant was always: the driver was extremely difficult to test, something common to all X input drivers. Development is a cycle of restarting the X server a billion times, testing is mostly plugging hardware in and moving things around in the hope that you can spot the bugs. On a driver that doesn't move much, this isn't necessarily a problem. Until a bug comes along, that requires some core rework of the event handling - in the kernel, libinput and, yes, the wacom driver.

After years of libinput development, I wasn't really in the mood for the whole "plug every tablet in and test it, for every commit". In a rather caffeine-driven development cycle [2], the driver was separated into two logical entities: the core driver and the "frontend". The default frontend is the X11 one which is now a relatively thin layer around the core driver parts, primarily to translate events into the X Server's API. So, not unlike libinput + xf86-input-libinput in terms of architecture. In ascii-art:

+--------------------+ | big giant
/dev/input/event0->| core driver | x11 |->| X server
+--------------------+ | process

Now, that logical separation means we can have another frontend which I implemented as a relatively light GObject wrapper and is now a library creatively called libgwacom:

+-----------------------+ |
/dev/input/event0->| core driver | gwacom |--| tools or test suites
+-----------------------+ |

This isn't a public library or API and it's very much focused on the needs of the X driver so there are some peculiarities in there. What it allows us though is a new wacom-record tool that can hook onto event nodes and print the events as they come out of the driver. So instead of having to restart X and move and click things, you get this:

$ ./builddir/wacom-record
version: 0.99.2
git: xf86-input-wacom-0.99.2-17-g404dfd5a
path: /dev/input/event6
name: "Wacom Intuos Pro M Pen"
- source: 0
event: new-device
name: "Wacom Intuos Pro M Pen"
type: stylus
keys: true
is-absolute: true
is-direct-touch: false
ntouches: 0
naxes: 6
- {type: x , range: [ 0, 44800], resolution: 200000}
- {type: y , range: [ 0, 29600], resolution: 200000}
- {type: pressure , range: [ 0, 65536], resolution: 0}
- {type: tilt_x , range: [ -64, 63], resolution: 57}
- {type: tilt_y , range: [ -64, 63], resolution: 57}
- {type: wheel , range: [ -900, 899], resolution: 0}
- source: 0
mode: absolute
event: motion
mask: [ "x", "y", "pressure", "tilt-x", "tilt-y", "wheel" ]
axes: { x: 28066, y: 17643, pressure: 0, tilt: [ -4, 56], rotation: 0, throttle: 0, wheel: -108, rings: [ 0, 0]
This is YAML which means we can process the output for comparison or just to search for things.

A tool to quickly analyse data makes for faster development iterations but it's still a far cry from reliable regression testing (and writing a test suite is a daunting task at best). But one nice thing about GObject is that it's accessible from other languages, including Python. So our test suite can be in Python, using pytest and all its capabilities, plus all the advantages Python has over C. Most of driver testing comes down to: create a uinput device, set up the driver with some options, push events through that device and verify they come out of the driver in the right sequence and format. I don't need C for that. So there's pull request sitting out there doing exactly that - adding a pytest test suite for a 20-year old X driver written in C. That this is a) possible and b) a lot less work than expected got me quite unreasonably excited. If you do have to maintain an old C library, maybe consider whether's possible doing the same because there's nothing like the warm fuzzy feeling a green tick on a CI pipeline gives you.

[1] As scholars of version numbers know, they make as much sense as your stereotypical uncle's facebook opinion, so why not.
[2] The Colombian GDP probably went up a bit

Posted Tue Feb 15 05:24:00 2022 Tags:
Understanding the delayed rollout of post-quantum cryptography. #pqcrypto #patents #ntru #lpr #ding #peikert #newhope
Posted Sat Jan 29 15:47:53 2022 Tags:

source: Wikimedia.

My last blog was already two months ago. The reason for this was the log4j security risk. Since much of our software actually is written in Java, the question was indeed if the CDK (doi:10.1186/s13321-017-0220-4), BridgeDb (doi:10.1186/1471-2105-11-5), Bacting (doi:10.21105/joss.02558), etc were affected. 

Basically, the toolkit is that old, that everyone jumped on in: it was just good. Now, practically, the problems were minor. The Chemistry Development Kit dependency was a build dependency: it still has support for log4j, but the user decides what logging platform to use. This was the result of an abstraction for Bioclipse, allowing CDK log messages to be passed to the Eclipse logger, instead of log4j. Still, you want even that build dependency to be updated. CDK 2.7.1 has been released now.

BridgeDb had a similar situation, tho some BridgeDb modules do have a runtime dependency which may have impact. However, the core did not, and the webservice did not. But the same applies here: even the build dependency should have the latest version. BridgeDb 3.0.13 has been released.

Now, if read up on the Blue Obelisk movement (maybe the 2011 update paper needs an update, doi:10.1186/1758-2946-3-37), then you know all the dependencies between projects. So, besides multiple releases for multiple projects, it also required updates on other packages and additional releases were made for the Blue Obelisk core projects Euclid and CMLXOM. Euclid 2.0 and CMLXOM 4.0 were released.

On the bright side, many Java software projects generally worked on library updates, Java 17 support, etc. It totally messed up my schedule and generally a really relaxed xmas holiday.

Who payed for this? Mostly myself. Yes, you're welcome.

Posted Fri Jan 14 07:17:00 2022 Tags:

Recently a security hole in a certain open source Java library resulted in a worldwide emergency kerfuffle as, say, 40% of the possibly hundreds of millions of worldwide deployments of this library needed to be updated in a hurry. (The other 60% also needed to be updated in a hurry, but won't be until they facilitate some ransomware, which is pretty normal for these situations.)

I have a 20+ year history of poking fun at Java in this space, and it pains me to stop now. But the truth is: this could have happened to anyone.

What happened was:

  • Someone wrote a library they thought was neat
  • They decided to share it with the world for free
  • Millions of people liked it and used it everywhere
  • Some contributors contributed some good ideas and, in this case, at least one bad idea
  • Out of a sense of stewardship, they decided to maintain backward compatibility with the bad idea
  • The bad idea turned out to have one or more security flaws that affected all the users
  • The stewards responded quickly with a fix

From this, if you believe the Internet Consensus, we can conclude that open source doesn't work, people don't get paid enough, capitalism is a sham, billionaires are built on the backs of the proletariat, your $50 Patreon donation makes a real difference, and Blorkchain Would Have Solved This.

(Miraculously the Internet Consensus is always the same both before and after these kinds of events. In engineering we call this a "non-causal system" because the outputs are produced before the inputs.)

Nevertheless, I have been dared to take my take on the issue. It, too, was the same before and after, but the difference is I didn't write it down until now, which makes this journal a causal system. You could probably write an interesting philosophical paper about observations of a non-causal system nevertheless being causal, but mercifully, I will not.

Free Software is Communism

So anyway, meandering gently toward the point, let's go back in time to the original Free Software movement. Long ago, before the average reader of this sentence was born, a person whose name is now unpopular was at a university, where they had a printer, and the printer firmware was buggy. This person firmly believed they could quickly fix the printer firmware if only they had the source code. (In the spirit of every "I could do this better in a weekend" story, I'm not sure whether we ever confirmed if this was true. In any case printer firmware is still buggy.)

As a result, they started a nonprofit organization to rewrite all of Unix, which the printer did not run and which therefore would not solve any of the original problem, but was a pretty cool project nonetheless and was much more fun than the original problem, and the rest was history.

This story archetype is the Hero's Journey that inspires all software development:

  • I have a problem
  • I do not know how to solve that problem
  • But I strongly believe, without evidence, that I can solve a generalized version of that problem if I form a large enough team and work at it for 35 years(*)
  • We are now accepting donations

(*) initial estimate is often less than 35 years

Now, you have probably heard all this before, and if you're a software developer you have probably lived it. This part is not really in question. The burning question for us today, as we enjoy the (hopefully) peak of late-stage capitalism, is: ...but where will the donations come from?


Before we get back onto communism, let me draw an important distinction. Most communist governments in history ended up being authoritarian systems, which is to say, top-down control. Ironically, the people at the top seem to have more power than the people at the bottom, which at first seems like the antithesis of communism. This is not the place to claim an understanding of why that always seems to happen. But one has to acknowledge a pattern when one sees it.

On the other hand, it's easy to find examples of authoritarianism outside communism. Our world is filled with top-down control systems. Many corporations are in many ways, top-down controlled. The US system of government is increasingly top-down controlled (ie. authoritarian), despite the many safety measures introduced early to try to prevent that.

When politicians rail against communism it is because they don't want you to notice the ever-growing non-communist authoritarianism.

Authoritarianism is self-reinforcing. Once some people or groups start having more power, they tend to use that power to adjust or capture the rules of the system so they can accumulate more power, and so on. Sometimes this is peacefully reversible, and sometimes it eventually leads to uprisings and revolutions.

People like to write about facism and communism as if they are opposite ends of some spectrum, but that's not really true in the most important sense. Fascism blatantly, and communism accidentally but consistently, leads to authoritarianism. And authoritarianism is the problem.

Authoritarianism is about taking things from me. Communism, in its noncorporeal theoretical form, is about giving things away.

I read a book once which argued that the problem with modern political discourse is it pits the "I don't want things taken from me" (liberty!) people against the "XYZ is a human right" (entitlement!) people. And that a better way to frame the cultural argument is "XYZ is my responsibility to society."

As a simple example, "Internet access is a human right," is just a sneaky way of saying "someone should give people free Internet." Who is someone? It's left unspecified, which is skipping over the entire mechanism by which we deliver the Internet. It's much more revealing to write, "To live in a healthy society, it's our responsibility to make sure every person has Internet access." Suddenly, oh, crap. The someone is me!

Healthy society is created through constant effort, by all of us, as a gift to our fellow members. It's not extracted from us as a mandatory payment to our overlords who will do all the work.

If there's one thing we know for sure about overlords, it's that they never do all the work.

Free software is a gift.

I would like to inquire about the return policy

Here's the thing about gifts: the sender chooses them, not the recipient. We can have norms around what gifts are appropriate, and agreements to not over-spend, and wishlists, and so on. But I won't always get the exact gift I want. Sometimes I didn't even want a gift. Sometimes the gift interprets JNDI strings in my log messages and executes random code from my LDAP server. This is the nature of gifts.

On the other hand, the best gifts are the things I never would have bought for myself, because they seemed too expensive or I didn't even realize I would like them or they were too much work to obtain, or because someone hand-made them just for me. These feel like luxuries of the sort capitalism cannot produce, because deciding, going out, and buying something for myself isn't luxury, it's everyday. It's lonely. It's a negotiation. It's limited by my own lack of creativity.

The best part of free software is it sometimes produces stuff you never would have been willing to pay to develop (Linux), and sometimes at quality levels too high to be rational for the market to provide (sqlite).

The worst part of free software is you get what you get, and the developers don't have to listen to you. (And as a developer, the gift recipients aren't always so grateful either.)

Paying for gifts

...does not work.

You don't say to someone, "here's $100, maybe this time get me a gift worth $100 more than you'd regularly spend." It's kind of insulting. It still probably won't get you exactly the thing you wanted. Actually, the other person might just pocket the $100 and run off with it.

We already have a way for you to spend $100 to get the thing you want. It's a market. A market works fine for that. It's not very inspiring, but most of the time it's quite efficient. Even gift-givers will often buy things on the same market, but with a different selection criteria, thus adding value of their own.

When you try to pay for gifts, it turns the whole gift process into a transaction. It stops being a gift. It becomes an inefficient, misdesigned, awkward market.

There's research showing that, for example, financial compensation in a job is more likely a demotivator than a motivator (ie. if you pay me too little, I'll work less hard or quit, but if you double my pay, it won't double my output). If you tie cash compensation to specific metrics, people will game the metrics and usually do an overall worse job. If you pay someone for doing you a favour, they are less likely to repeat the favour. Gifts are inherently socially and emotionally meaningful. Ruin the giftiness, and you ruin the intangible rewards.

So it is with free software. You literally cannot pay for it. If you do, it becomes something else.

This is why we have things like the Linux Foundation, where the idea is you can give a gift because you appreciate and want to support Linux (and ideally you are a rich megacorporation so your gift is very big), but it dilutes the influence of that money through an organization that supposedly will not try to influence the gift of Linux that was already happening. You end up with multiple gift flows in different directions. Money goes here, code goes there. They are interdependent - maybe if one flow slows down the other flow will also slow down - but not directly tied. It's a delicate balance. People who keep receiving Christmas gifts but never give any might eventually stop receiving them. But might not.

Anyway, gifts will not get you 24-hour guaranteed response times to security incidents.

Gifts won't get you guaranteed high quality code reviews.

Gifts will not, for heaven's sake, prevent developers from implementing bad ideas occasionally that turn into security holes. Nothing will. Have you met developers?

Open source

I've avoided the term "open source" so far because it means something different from the original idea of Free Software.

Open source was, as I understand it, coined to explain what happened when Netscape originally opened their Mozilla source code, back at the end of the 1990s. That was not a gift. That was a transaction. Or at least, it was intended to be.

The promise of open source was:

  • You, the company, can still mostly control your project
  • Customers will still pay you to add new features
  • Actually customers might pay other people to add new features, but you can still capitalize on it because you get their code too
  • Linux distributions only package open source code so you'll onboard more customers more easily this way
  • You can distance yourself from this anti-capitalist gift-giving philosophical stuff that makes investors nervous
  • Plus a bunch of people will look at the code and find bugs for you for free!

Maybe this sounds cynical, but capitalists are cynical, and you know what? It worked! Okay, not for Netscape Corporation (sorry), but for a lot of other people since then.

It also failed a lot of people. Many developers and companies have been disappointed to learn that just uploading your code to github doesn't make a community of developers appear. (It does make it more likely that AWS will fork your product and make more money from it than you do.) Code reviews are famously rare even in security-critical projects. Supply chain issues are rampant.

In fact, we've now gotten to the point where some people hesitate to give away their source code, mainly because of this confusion of gifts and customers. If I spend some spare time hacking something together on a weekend and give it away, that's a gift. If you yell at me for making it, that makes giving less fun, and I will spend fewer weekends making gifts.

Whereas when a company has a product and open sources it and you complain, that's customers giving valuable feedback and it's worth money to learn from them and service them, because you eventually earn money in exchange (through whatever business model they've established). No gift necessary.

Call it cynical or call it a win/win relationship. But it's not a gift.

The startup ecosystem

Since the creation of the open source designation 20+ years ago, software startups have taken off more than ever. I attribute this to a combination of factors:

  • Cloud computing has made it vastly cheaper to get started
  • Incubators like YCombinator have industrialized the process of assembling and running a small software company
  • Megacorps have become exponentially richer but no more creative, so they need to acquire or acqui-hire those startups faster and faster in order to grow.

Although a lot of startups open source their code, and they all depend heavily on open source ecosystems, the startup world's motivations are amazingly different from the free software and open source worlds.

Gifts exist in the startup world. They are things like "we were both in YCombinator so I will intro you to this investor I like" or "I got extremely rich so let me invest in your startup and incidentally I get a lottery ticket for becoming even more rich." These absolutely are still gifts. They each strengthen social ties. The startup world is a society, and the society is built up from these gifts. It's a society that largely ignores the trials and tribulations of anyone who isn't a rich software engineer insider, but history has hosted many past societies of that sort and it takes a long time to build and deploy enough guillotines, and anyway they are having fun and producing a lot and surely that counts for something.

If free software gifts are communism and open source is cynically capitalist exploitation, then startups may be, weirdly, the democratization of capitalism.

Hear me out. Big companies don't care what you think; you can't pay them enough to care. Gift givers care only a little what you think; if they gave you what you wanted, it wouldn't be a gift. But startups, well, there are a lot of them and their mantras are "do things that don't scale" and "focus on the customer" and "build rapid feedback loops." What that spells for you is a whole bunch of people who want to give you what you want, in exchange for money, and who are excited to amortize the costs of that over all the other customers who want the same thing.

It's kind of exciting, conceptually, and more self-optimizing than untuned gift giving, and so it's not super surprising to me that it has started to eclipse the earlier concepts of free software and open source. More and more "open" projects are backed by small companies, who have financial incentives to make their users happy because some of the users turn into paying customers. They'll even provide the uptime SLAs and security fix turnaround guarantees you wanted so much. Our company, Tailscale, is unabashedly one of those. Nothing to be ashamed of there. The system works.

What doesn't work is assuming those startup mechanics apply to everyone out there who gives you a software gift. Not every project on github is the same.

Not everyone has the same motivations.

Giving them money won't change their motivations.

Trying to pay them or regulate them taints the gift.

If you wanted to pay someone to fix some software, you didn't want a gift. You wanted a company.

But if there is no company and someone gave you something anyway? Say thanks.


This isn't where evolution stops. There's a lot more to say about how SaaS taints the unwritten agreement of open source (because you don't have to give back your changes to the code), and how startups tend to go bankrupt and their tech dies with them, and how the best developers are not good at starting companies (no matter how much easier it has become), and how acquiring a startup usually destroys all the stuff they innovated, and how open source is often used as a way to exfiltrate past those kinds of disasters, and how simultaneously, whole promising branches of the "gift economy" structure have never been explored. But that's enough for today. Maybe another time.

Posted Thu Dec 30 12:43:51 2021 Tags: