The lower-post-volume people behind the software in Debian. (List of feeds.)

The latest feature release Git v2.5.0 is now available at the usual places. It is comprised of 583 non-merge commits since v2.4.0, contributed by 70 people, 21 of which are new faces.

One interesting change is to git help. We now list commands, grouped by the situation in which you would want to use them. This came from discussion on usability, inspired by one of the talks at GitMerge conference we had in spring.

Among notable new features, some of my favourites are:
  • A new short-hand branch@{push} denotes the remote-tracking branch that tracks the branch at the remote the branch would be pushed to.
  • git send-email learned the alias file format used by the sendmail program.
  • Traditionally, external low-level 3-way merge drivers are expected to produce their results based solely on the contents of the three variants given in temporary files named by %O, %A and %B placeholders on their command line. They are now additionally told about the final path (given by %P).
  • A heuristic we use to catch mistyped paths on the command line git cmd revs pathspec is to make sure that all the non-rev parameters in the later part of the command line are names of the files in the working tree, but that means git grep string -- \*.c must always be disambiguated with --, because nobody sane will create a file whose name literally is asterisk-dot-see.  We loosen the heuristic to declare that with a wildcard string the user likely meant to give us a pathspec. So you can now simply say git grep string \*.c without --.
  • Filter scripts were run with SIGPIPE disabled on the Git side, expecting that they may not read what Git feeds them to filter. We however treated a filter that does not read its input fully before exiting as an error.  We no longer do and ignore EPIPE when writing to feed the filter scripts.

    This changes semantics, but arguably in a good way.  If a filter can produce its output without fully consuming its input using whatever magic, we now let it do so, instead of diagnosing it as a programming error.
  • Whitespace breakages in deleted and context lines can also be painted in the output of git diff and friends with the new --ws-error-highlight option.
  • git merge FETCH_HEAD learned that the previous "git fetch" could be to create an Octopus merge, i.e. recording multiple branches that are not marked as "not-for-merge"; this allows us to lose an old style invocation git merge msg HEAD commits... in the implementation of git pull script; the old style syntax can now be deprecated (but not removed yet).
There are a few "experimental" new features, too. They are still incomplete and/or buggy around the edges and likely to change in the future, but nevertheless interesting.
  • git cat-file --batch learned the --follow-symlinks option that follows an in-tree symbolic link when asked about an object via extended SHA-1 syntax.  For example, HEAD:RelNotes may be a symbolic link that points at Documentation/RelNotes/2.5.0.txt.  With the new option, the command behaves as if HEAD:Documentation/RelNotes/2.5.0.txt was given as input instead.

    This is incomplete in at least a few ways.
    (1) A symbolic link in the index, e.g. :RelNotes, should also be treated the same way, but isn't. (2) Non-batch mode, e.g. git cat-file --follow-symlinks blob HEAD:RelNotes, may also want to behave the same way, but it doesn't.
  • A replacement mechanism for contrib/workdir/git-new-workdir that does not rely on symbolic links and make sharing of objects and refs safer by making the borrowee and borrowers aware of each other has been introduced and accessible via git worktree add. This is accumulating more and more known bugs but may prove useful once they are fixed.

Posted Mon Jul 27 21:18:00 2015 Tags:

Disclaimer: I have almost no experience cooking Vietnamese, this is just a distillation of various recipes I’ve only read.

Lamb steak (or other meat: just did this with rib-eye steak, was delicious)
Onion
Sugar
Garlic
Crushed chilli (or other form of chilli)
Vinegar
Soy sauce
Fish sauce

First cook the lamb (or other) steak until browned outside but rare inside. Set aside and slice into thin slices during the remaining cooking.

Slice onions then fry gently in oil until transparent. Add sugar (quite a lot – for two I use a couple of tablespoons at least), garlic, chilli and fry until the sugar is lightly caramelised. Throw in equal parts of vinegar, soy and fish sauce (roughly speaking, you want a little more total liquid than you had sugar), stir in and bring to the boil. Cook until slightly thickened. Add the sliced meat and stir-fry until cooked to your liking.

Serve with boiled rice and something green.

Posted Thu Jul 23 06:48:28 2015 Tags:

Hello Internet! I wanted to share some updates of Roslyn and Mono.

We have been working towards using Roslyn in two scenarios. As the compiler you get when you use Mono, and as the engine that powers code completion and refactoring in the IDE.

This post is a status update on the work that we have been doing here.

Roslyn on MonoDevelop/XamarinStudio

For the past year, we have been working on replacing the IDE's engine that gives us code completion, refactoring capabilities and formatting capabilities with one powered by Roslyn.

The current engine is powered by a combination of NRefactory and the Mono C# compiler. It is not as powerful, comprehensive or reliable as Roslyn.

Feature-wise, we completed the effort, and we now have a Roslyn-powered branch that uses Roslyn for code completion, refactoring, suggestions and code formatting.

In addition, we ported most of the refactoring capabilities from NRefactory to work on top of Roslyn. These were quite significant. Visual Studio users can try them out by installing the Refactoring Essentials for Visual Studio extension.

While our Roslyn branch is working great and is a pleasure to use, it also consumes more memory and by extension, runs a little slower. This is not Roslyn's fault, but the side effects of leaks and limitations in our code.

Our original plan was to release this for our September release (what we internally call "Cycle 6"), but we decided to pull the feature out from the release to give us time to fix the leaks that affected the Roslyn engine and tune the performance of Roslyn running on Mono.

Our revisited plan is to ship an update to our tooling in Cycle 6 (the regular feature update) but without Roslyn. In parallel, we will ship a Roslyn-enabled preview of MonoDevelop/XamarinStudio. This will give us time to collect your feedback on performance and memory usage regressions, and time to fix the issues before we make Roslyn the default.

Roslyn as a Compiler in Mono

One of the major roadblocks for the adoption of Roslyn in Mono was the requirement to generate debugging information that Mono could consume on Unix (the other one is that our C# batch compiler is still faster than Roslyn).

The initial Roslyn release only had support for generating debug information through a proprietary/native library on Windows, which meant that while Roslyn could be used to compile code on Unix, the result would not contain any debug information - this prevented Roslyn from being useful for most compilation uses.

Recently, Roslyn got support for Portable Program Database (PPDB) files. This is a fully documented, open, compact and efficient format for storing debug information.

Mono's master release contains now support for using PPDB files as its debug information. This means that Roslyn can produce debug information that Mono can consume.

That said, we still need more work in the Mono ecosystem to fully support PPDB files. The Cecil library is used extensively to manipulate IL images as well as their associated debug information. Our Reflection.Emit implementation will need to get a backend to generate PPDBs (for third party compilers, dynamic code generators) and support in IKVM to produce PPDB files (this is used by Mono's C# compiler and other third party compilers).

Additionally, many features in Roslyn surfaced bloat and bugs in Mono's class libraries. We have been fixing those bugs (and in many cases, the bugs have gone away by replacing Mono's implementation with implementations from Microsoft's Reference Source).

Posted Tue Jul 21 16:00:37 2015 Tags:

Below is an outline of the various types of touchpads that can be found in the wild. Touchpads aren't simply categorised into a single type, instead they have a set of properties, a combination of number of physical buttons, touch support and physical properties.

Number of buttons

Physically separate buttons

For years this was the default type of touchpads: a touchpad with a separate set of physical buttons below the touch surface. Such touchpads are still around, but most newer models are Clickpads now.

Touchpads with physical buttons usually provide two buttons, left and right. A few touchpads with three buttons exist, and Apple used to have touchpads with a single physical buttons back in the PPC days. Touchpads with only two buttons require the software stack to emulate a middle button. libinput does this when both buttons are pressed simultaneously.


A two-button touchpad, with a two-button pointing stick above.

Note that many Lenovo laptops provide a pointing stick above the touchpad. This pointing stick has a set of physical buttons just above the touchpad. While many users use those as substitute touchpad buttons, they logically belong to the pointing stick. The *40 and *50 series are an exception here, the former had no physical buttons on the touchpad and required the top section of the pad to emulate pointing stick buttons, the *50 series has physical buttons but they are wired to the touchpads. The kernel re-routes those buttons through the trackstick device.

Clickpads

Clickpads are the most common type of touchpads these days. A Clickpad has no separate physical buttons, instead the touchpad itself is clickable as a whole, i.e. a user presses down on the touch area and triggers a physical click. Clickpads thus only provide a single button, everything else needs to be software-emulated.


A clickpad on a Lenovo x220t. Just above the touchpad are the three buttons associated with the pointing stick. Faint markings on the bottom of the touchpad hint at where the software buttons should be.

Right and middle clicks are generated either via software buttons or "clickfinger" behaviour. Software buttons define an area on the touchpad that is a virtual right button. If a finger is in that area when the click happens, the left button event is changed to a right button event. A middle click is either a separate area or emulated when both the left and right virtual buttons are pressed simultaneously.

When the software stack uses the clickfinger method, the number of fingers decide the type of click: a one-finger is a left button, a two-finger click is a right button, a three-finger click is a middle button. The location of the fingers doesn't matter, though there are usually some limits in how the fingers can be distributed (e.g. some implementations try to detect a thumb at the bottom of the touchpad to avoid accidental two-finger clicks when the user intends a thumb click).

The libinput documentation has a section on Clickpad software button behaviour with more detailed illustrations


The touchpad on a T440s has no physical buttons for the pointing stick. The marks on the top of the touchpad hint at the software button position for the pointing stick. Note that there are no markings at the bottom of the touchpad anymore.

Clickpads are labelled by the kernel with the INPUT_PROP_BUTTONPAD input property.

Forcepads

One step further down the touchpad evolution, Forcepads are Clickpads without a physical button. They provide pressure and (at least in Apple's case) have a vibration element that is software-controlled. Instead of the satisfying click of a physical button, you instead get a buzz of happiness. Which apparently feels the same as a click, judging by the reviews I've read so far. A software-controlled click feel has some advantages, it can be disabled for some gestures, modified for others, etc. I suspect that over time Forcepads will become the main touchpad category, but that's a few years away.

Not much to say on the implementation here. The kernel has some ForcePad support but everything else is spotty.


Note how Apple's Clickpads have no markings whatsoever, Apple uses the clickfinger method by default.

Touch capabilities

Single-touch touchpads

In the beginning, there was the single-finger touchpad. This touchpad would simply provide x/y coordinates for a single finger and get mightily confused when more than one finger was present. These touchpads are now fighting with dodos for exhibition space in museums, few of those are still out in the wild.

Pure multi-touch touchpads

Pure multi-touch touchpads are those that can track, i.e. identify the location of all fingers on the touchpad. Apple's touchpads support 16 touches (iirc), others support 5 touches like the Synaptics touchpads when using SMBus.

Pure multi-touch touchpads are the easiest to support, we can rely on the finger locations and use them for scrolling, gestures, etc. These touchpads usually also provide extra information. In the case of the Apple touchpads we get an ellipsis and the orientation of the ellipsis for each touch point. Other touchpads provide a pressure value for each touch point. Though pressure is a bit of a misnomer, pressure is usually directly related to contact area. Since our puny human fingers flatten out as the pressure on the pad increases, the contact area increases and the firmware then calculates that back into a (mostly rather arbitrary) pressure reading.

Because pressure is really contact area size, we can use it to detect accidental palm contact or thumbs though it's fairly unreliable. A light palm touch or a touch at the very edge of a touchpad will have a low pressure reading simply because the palm is mostly next to the touchpad and thus the contact area itself remains small.

Partial multi-touch touchpads

The vast majority of touchpads fall into this category. It's the half-way point between single-touch and pure multi-touch. These devices can track N fingers, but detect more than N. The current Synaptics touchpads fall into that category when they're using the serial protocol. Most touchpads that fall into this category can track two fingers and detect up to four or five. So a typical three-finger interaction would give you the location of two fingers and a separate value telling you that a third finger is down.

The lack of finger location doesn't matter for some interactions (tapping, three-finger click) but it can cause issues in some cases. For example, a user may have a thumb resting on a touchpad while scrolling with two fingers. Which touch locations you get depends on the order of the fingers being set down, i.e. this may look like thumb + finger + third touch somewhere (lucky!) or two fingers scrolling + third touch somewhere (unlucky, this looks like a three-finger swipe). So far we've mostly avoided having anything complex enough that requires the exact location of more than two fingers, these pads are so prevalent that any complex feature would exclude the majority of users.

Semi-mt touchpads

A sub-class of partial multi-touch touchpads. These touchpads can technically detect two fingers but the location of both is limited to the bounding box, i.e. the first touch is always the top-left one and the second touch is the bottom-right one. Coordinates jump around as fingers move past each other. Most semi-mt touchpads also have a lower resolution for two touches than for one, so even things like two-finger scrolling can be very jumpy.

Semi-mt are labelled by the kernel with the INPUT_PROP_SEMI_MT input property.

Physical properties

External touchpads

USB or Bluetooth touchpads not in a laptop chassis. Think the Apple Magic Trackpad, the Logitech T650, etc. These are usually clickpads, the biggest difference is that they can be removed or added at runtime. One interaction method that is only possible on external touchpads is a thumb resting on the very edge/immediately next to the touchpad. On the far edge, touchpads don't always detect the finger location so clicking with a thumb barely touching the edge makes it hard or impossible to figure out which software button area the finger is on.

These touchpads also don't need palm detection - since they're not located underneath the keyboard, accidental palm touches are a non-issue.


A Logitech T650 external touchpad. Note the thumb position, it is possible to click the touchpad without triggering a touch.

Circular touchpads

Yes, used to be a thing. Touchpad shaped in an ellipsis or circle. Luckily for us they have gone full dodo. The X.Org synaptics driver had to be aware of these touchpads to calculate the right distance for edge scrolling - unsurprisingly an edge scroll motion on a circular touchpad isn't very straight.

Graphics tablets

Touch-capable graphics tablets are effectively external touchpads, with two differentiators: they are huge compared to normal touchpads and they have no touchpad buttons whatsoever. This means they can either work like a Forcepad, or rely on interaction methods that don't require buttons (like tap-to-click). Since the physical device is shared with the pen input, some touch arbitration is required to avoid touch input interfering when the pen is in use.

Dedicated edge scroll area

Mostly on older touchpads before two-finger scrolling became the default method. These touchpads have a marking on the touch area that designates the edge to be used for scrolling. A finger movement in that edge zone should trigger vertical motions. Some touchpads have markers for a horizontal scroll area too at the bottom of the touchpad.


A touchpad with a marked edge scroll area on the right.
Posted Tue Jul 21 04:40:00 2015 Tags:

Just a quick note that we’ve released the latest stable NetworkManager, version 1.0.4.  This is mainly a bugfix release, though it does have a couple new features and larger changes.  Some of those are:

  • Some configuration options can now be changed without restarting the NM daemon.  Those include the ‘dns’, ‘connectivity’, and ‘ignore-carrier’ settings.
  • Devices that have only an IPv6 link-local address are no longer assumed to be connected; by default whenever an interface is set “up” the kernel will assign an IPv6 link-local address which could theoretically be used for communication.  NetworkManager used to interpret this as the interface being available for network communication, while this is rarely what users want or expect.
  • Correct routing is now maintained when two interfaces of the same priority are connected to the same network.
  • udev rules can now be used to tell NetworkManager to manage or unmanage specific devices.
  • Connections with children (bridge, bond, team, etc) can now optionally bring up their slave interfaces when the master is started.
  • Many, many bugs and crashers have also been fixed in the core daemon, the libraries, and nmcli.

Grab the 1.0.4 release here!

We’re well into the development cycle of NetworkManager 1.2 as well, where among other things, we’ll finally be moving to GDBus instead of dbus-glib.  We’ll also have support for structured logging with journald, indicating that certain connections are metered, advertising LLDP-discovered information, built-in IPv4 link-local addressing to get rid of avahi-autoipd, improvements to Wi-Fi Access Point scanning, less verbose logging by default, improvements to DNS handling, and more.  Look for it later this year!

 

Posted Thu Jul 16 16:20:47 2015 Tags:

In a perfect world, any device that advertises absolute x/y axes also advertises the resolution for those axes. Alas, not all of them do. For libinput, this problem is two-fold: parts of the touchscreen API provide data in mm - without knowing the resolution this is a guess at best. But it also matters for touchpads, where a lack of resolution is a lot more common (though the newest generations of major touchpad manufacturers tend to advertise resolutions now).

We have a number of features that rely on the touchpad resolution: from the size of the software button to deciding which type of palm detection we need, it all is calculated based on physical measurements. Until recently, we had code to differentiate between touchpads with resolution and most of the special handling was a matter of magic numbers, usually divided by the diagonal of the touchpad in device units. This made code maintenance more difficult - without testing each device, behaviour could not be guaranteed.

With libinput 0.20, we now got rid of this special handling and instead require the touchpads to advertise resolutions. This requires manual intervention, so we're trying to fix this in multiple places, depending on the confidence of the data. We have hwdb entries for the bcm5974 (Apple) touchpads and the Chromebook Pixel. For Elantech touchpads, a kernel patch is currently waiting for merging. For ALPS touchpads, we ship size hints with libinput's hwdb. If all that fails, we fall back to a default touchpad size of 69x55mm. [1]

All this affects users in two ways: one is that you may notice a slightly different behaviour of your touchpad now. The software-buttons may be bigger or smaller than before, pointer acceleration may be slightly different, etc. Shouldn't be too bad, but you may just notice it. The second noticeable change is that libinput will now log when it falls back to the default size. If you notice a message like that in your log, please file a bug and attach the output of evemu-describe and the physical dimensions of your touchpad. Once we have that information, we can add it to the right place and make sure that everyone else with that touchpad gets the right settings out of the box.

[1] The default size was chosen because it's close enough to what old touchpads used to be, and those are most likely to lack resolution values. This size may change over time as we get better data.

Posted Thu Jul 16 11:28:00 2015 Tags:

Most of you by now have probably seen Conservancy's and FSF's statements regarding the today's update to Canonical, Ltd.'s Ubuntu IP Policy. I have a few personal comments, speaking only for myself, that I want to add that don't appear in the FSF's nor Conservancy's analysis. (I wrote nearly all of Conservancy's analysis and did some editing on FSF's analysis, but the statements here I add are my personal opinions and don't necessarily reflect the views of the FSF nor Conservancy, notwithstanding that I have affiliations with both orgs.)

First of all, I think it's important to note the timeline: it took two years of work by two charities to get this change done. The scary thing is that compared to their peers who have also violated the GPL, Canonical, Ltd. acted rather quickly. As Conservancy pointed out regarding the VMware lawsuit, it's not uncommon for these negotiations to take even four years before we all give up and have to file a lawsuit. So, Canonical, Ltd. resolved the matter at least twice as fast as VMware, and they deserve some credit for that — even if other GPL violators have set the bar quite low.

Second, I have to express my sympathy for the positions on this matter taken by Matthew Garrett and Jonathan Riddell. Their positions show clearly that, while the GPL violation is now fully resolved, the community is very concerned about what the happens regarding non-copylefted software in Ubuntu, and thus Ubuntu as a whole.

Realize, though, that these trump clauses are widely used throughout the software industry. For example, electronics manufacturers who ship an Android/Linux system with standard, disgustingly worded, forbid-everything EULA usually include a trump clause not unlike Ubuntu's. In such systems, usually, the only copylefted program is the kernel named Linux. The rest of the distribution includes tons of (now proprietarized) non-copylefted code from Android (as well as a bunch of born-proprietary applications too). The trump clause assures the software freedom rights for that one copylefted work present, but all the non-copylefted ones are subject to the strict EULA (which often includes “no reverse engineer clauses”, etc.). That means if the electronics company did change the Android Java code in some way, you can't even legally reverse engineer it — even though it was Apache-licensed by upstream.

Trump clauses are thus less than ideal because they achieve compliance only by allowing a copyleft to prevail when the overarching license contradicts specific requirements, permissions, or rights under copyleft. That's acceptable because copyleft licenses have many important clauses that assure and uphold software freedom. By contrast, most non-copyleft licenses have very few requirements, and thus they lack adequate terms to triumph over any anti-software-freedom terms of the overarching license. For example, if I take a 100% ISC-licensed program and build a binary from it, nothing in the ISC license prohibits me from imposing this license on you: “you may not redistribute this binary commercially”. Thus, even if I also say to you: “but also, if the ISC license grants rights, my aforementioned license does not modify or reduce those rights”, nothing has changed for you. You still have a binary that you can't distribute commercially, and there was no text in the ISC license to force the trump clause to save you.

Therefore, this whole situation is a simple and clear argument for why copyleft matters. Copyleft can and does (when someone like me actually enforces it) prevent such situations. But copyleft is not infinitely expansive. Nearly every full operating system distribution available includes an aggregated mix of copylefted, non-copyleft, and often fully-proprietary userspace applications. Nearly every company that distributes them wraps the whole thing with some agreement that restricts some rights that copyleft defends, and then adds a trump clause that gives an exception just for FLOSS license compliance. Sadly, I have yet to see a company trailblaze adoption of a “software freedom preservation” clause that guarantees copyleft-like compliance for non-copylefted programs and packages. Thus, the problem with Ubuntu is just a particularly bad example of what has become a standard industry practice by nearly every “open source” company.

How badly these practices impact software freedom depends on the strictness and detailed terms of the overarching license (and not the contents of the trump clause itself; they are generally isomorphic0). The task of analyzing and rating “relative badness” of each overarching licensing document is monumental; there are probably thousands of different ones in use today. Matthew Garrett points out why Canonical, Ltd.'s is particularly bad, but that doesn't mean there aren't worse (and better) situations of a similar ilk. Perhaps our next best move is to use copyleft licenses more often, so that the trump clauses actually do more.

In other words, as long as there is non-copylefted software aggregated in a given distribution of an otherwise Free Software system, companies will seek to put non-Free terms on top of the non-copylefted parts, To my knowledge, every distribution-shipping company (except for extremely rare, Free-Software-focused companies like ThinkPenguin) place some kind of restrictions in their business terms for their enterprise distribution products. Everyone seems to be asking me today to build the “worst to almost-benign” ranking of these terms, but I've resisted the urge to try. I think the safe bet is to assume that if you're looking at one of these trump clauses, there is some sort of software-freedom-unfriendly restriction floating around in the broader agreement, and you should thus just avoid that product entirely. Or, if you really want to use it, fork it from source and relicense the non-copylefted stuff under copyleft licenses (which is permitted by nearly all non-copyleft licenses), to prevent future downstream actors from adding more restrictive terms. I'd even suggest this as a potential solution to the current Ubuntu problem (or, better yet, just go back upstream to Debian and do the same :).

Finally, IMO the biggest problem with these “overarching licenses with a trump clause” is their use by companies who herald “open source” friendliness. I suspect the community ire comes from a sense of betrayal. Yet, I feel only my usual anger at proprietary software here; I don't feel betrayed. Rather, this is just another situation that proves that saying you are an “open source company” isn't enough; only the company's actions and “fine print” terms matter. Now that open source has really succeeded at coopting software freedom, enormous effort is now required to ascertain if any company respects your software freedom. We must ignore the ballyhoo of “community managers” and look closely at the real story.


0Despite Canonical, Ltd.'s use of a trump clause, I don't think these various trump clauses are canonically isomorphic. There is no natural mapping between these various trump clauses, but they all do have the same effect: they assure that when the overarching terms conflict with the a FLOSS license, the FLOSS license triumphs over the overarching terms, no matter what they are. However, the potential relevance of the phrase “canonical isomorphism” here is yet another example why it's confusing and insidious that Canonical, Ltd. insisted so strongly on using canonical in a non-canonical way.

Posted Wed Jul 15 22:45:10 2015 Tags:

The libinput test suite takes somewhere around 35 minutes now for a full run. That's annoying, especially as I'm running it for every commit before pushing. I've tried optimising things, but attempts at making it parallel have mostly failed so far (almost all tests need a uinput device created) and too many tests rely on specific timeouts to check for behaviours. Containers aren't an option when you have to create uinput devices so I started out farming out into VMs.

Ideally, the test suite should run against multiple commits (on multiple VMs) at the same time while I'm working on some other branch and then accumulate the results. And that's where git notes come in. They're a bit odd to use and quite the opposite of what I expected. But in short: a git note is an object that can be associated with a commit, without changing the commit itself. Sort-of like a post-it note attached to the commit. But there are plenty of limitations, for example you can only have one note (per namespace) and merge conflicts are quite easy to trigger. Look at any git notes tutorial to find out more, there's plenty out there.

Anyway, dealing with merge conflicts is a no-go for me here. So after a bit of playing around, I found something that seems to work out well. A script to run make check and add notes to the commit, combined with a repository setup to fetch those notes and display them automatically. The core of the script is this:


make check
rc=$?
if [ $? -eq 0 ]; then
status="SUCCESS"
else
status="FAIL"
fi

if [ -n "$sha" ]; then
git notes --ref "test-$HOSTNAME" append \
-m "$status: $HOSTNAME: make check `date`" HEAD
fi
exit $rc
Then in my main repository, I add each VM as a remote, adding a fetch path for the notes:

[remote "f22-libinput1"]
url = f22-libinput1.local:/home/whot/code/libinput
fetch = +refs/heads/*:refs/remotes/f22-libinput1/*
fetch = +refs/notes/*:refs/notes/f22-libinput1/*
Finally, in the main repository, I extended the glob that displays notes to 'everything':

$ git config notes.displayRef "*"
Now git log (and by extension tig) displays all notes attached to a commit automatically. All that's needed is a git fetch --all to fetch everything and it's clear in the logs which commit fails and which one succeeded.

:: whot@jelly:~/code/libinput (master)> git log
commit 6896bfd3f5c3791e249a0573d089b7a897c0dd9f
Author: Peter Hutterer
Date: Tue Jul 14 14:19:25 2015 +1000

test: check for fcntl() return value

Mostly to silence coverity complaints.

Signed-off-by: Peter Hutterer

Notes (f22-jelly/test-f22-jelly):
SUCCESS: f22-jelly: make check Tue Jul 14 00:20:14 EDT 2015

Whenever I look at the log now, I immediately see which commits passed the test suite and which ones didn't (or haven't had it run yet). The only annoyance is that since a note is attached to a commit, amending the commit message or rebasing makes the note "go away". I've copied notes manually after this, but it'd be nice to find a solution to that.

Everything else has been working great so far, but it's quite new so there'll be a bit of polishing happening over the next few weeks. Any suggestions to improve this are welcome.

Posted Tue Jul 14 20:40:00 2015 Tags:

I think it’s weird that I have to write this post in 2015, but earlier today I had to explain to someone with the technical skills to submit a good patch that he was doing the process wrong in some basic and extremely annoying ways.

Googling revealed that most explanations of patch etiquette are rather project-specific in their advice. So I’m going to explain the basics of patch submission that apply to just about any open-source project, with a focus on how to do it right when you aren’t a regular committer (that is, it’s what’s often called a drive-by patch). Here we go…

Let’s start with the blunder that motivated this post. It was email that inquired “Have you had a chance to review my previously submitted patch?” What was infuriating was that said inquiry did not identify or include a copy of that drive-by patch, didn’t identify what revision of the project it was made against, and for that matter didn’t even name the project.

This violated the first rule of getting your patch accepted, which is to minimize the cognitive load on the person processing it. That person may see dozens of patches a week on multiple projects (and frequently does, if it’s me). Expecting him to remember lots of context and match it to your name when you’re not already a regular committer that he recognizes is a good way to make yourself disliked.

So make it easy. When you mail a drive-by patch, always specify the target project it applies to, and what base revision of the project it applies to (by commit ID or date). The hapless n00b I am now excoriating replied, when asked what revision his patch was against, replied “HEAD” and it was all I could do not to leap through the screen and throttle him. Because of course HEAD now is not necessarily the same revision it was when he composed the patch.

Good etiquette is different if you have a well-established working relationship with the person you are submitting to. If I get a git-format patch submission from a regular contributor whom I recognize to one of my projects. he doesn’t have to repeat that context; I can generally assume it was made from a recently-synced repository clone and will apply against head. But I can only be relaxed about this if I am sure he will be explicit about the target project and base revision if it’s for a different project.

Another way of minimizing cognitive load is to make the patch self-explaining. Always include an explanation of the patch, its motivation, and its risks (in particular, if the patch is untested you should say so).

Hapless n00b failed to do this. I had to Google for the name of the principal function in his patch to learn that it’s a code-hardening move derived from OpenBSD security work. Even though it’s a good patch, and I merged it, that’s more work than he should have made me do. Through not being explicit, he actually slowed down the merge of a fix for a potential security issue by a couple of weeks.

Even if you are not sending a git-format patch, the git conventions for change comments are good to follow. That is, a summary line in imperative form (“Fix bug 2317.”; “Implement feature foo.”) should be optionally followed by a blank line and explanatory paragraphs.

Really good bug-fix patches include a recipe for reproducing the problem that motivated them. Because I can test those instantly, I can merge them instantly. On projects with a regression-test suite, you scale the heights of best practice by including a patch bands that enhance the test suite to demonstrate the correctness of the patched code. When you do that, I love you and want to see more patches from you.

Here’s a mistake the n00b thankfully did not make: his patch was small and simple, easily reviewed. When in doubt, send a patch series of simple changes rather than a huge rollup patch that does multiple things – that sort of mess is very likely to be rejected because reviewing it hurts the maintainer’s brain.

At least he shipped in diff -u format; it’d been years since I saw even a n00b use the horrible default -e format, which is very brittle in the face of any change to the target code at all (never do this!). diff -u -p format, which tries to put the name of the affected function in the header before each patch band, is better (this what git format-patch does).

Maintainers vary in whether they like patches to be plain text inline in the message or a MIME attachment. I myself am relatively indifferent to this, but others have strong preferences. If you are not sure, include an offer to resend in the form the maintainer prefers in your cover note.

On the other hand, unless the patch is large enough to blow an email size limit (which suggests it’s much too large) do not ever send it as an http/ftp link to be chased to the patch text. Your maintainer might be on the road with spotty network access when he tries to process it. Even if he isn’t, every step of hand-work he has to do (including trivial steps like typing a wget command) increases his process friction, which is unkind and reduces the odds that your patch will be merged.

It is irritating to the maintainer to get a patch against an old release, it is very irritating to get a patch for a problem he/she has already fixed, and it is extremely irritating to get a patch for a problem that the maintainer has to do extra work to find out he has already fixed. Find the project repository, clone it, test your issue, and make the patch against head. This will, among other things, greatly reduce the odds of your patch failing to apply.

Many maintainers (including me) dislike unnecessary merge bubbles. If you write a patch series using git that takes a while to complete, rebase it against current head before you ship. Again, this maximizes the chances that it will apply cleanly.

I’m going to finish with a preference of mine which while not necessarily universal practice, at least won’t offend anyone. If you have to refer to a previous commit in your change comment, please do not use commit IDs like git hashes or Subversion revision numbers to do it. Instead, quote the summary line of the commit and if in doubt cite the author and date.

There are two good reasons for this practice. One is that it’s human-friendly; a patch summary line is a much better cue to memory than a numeric or hex cookie. The other is that at some time in the future the repository might undergo surgery or conversion that will turn that cookie into a turd. Be kind to future maintainers by using a reference format that won’t become invalid and require hand translation.

Posted Mon Jul 13 12:31:42 2015 Tags:
Visualization of functional groups.
Public domain from Wikipedia.
In the past 50 years we have been trying to understand why certain classes of compounds show the same behavior. Quantum chemical calculations are still getting cheaper and easier (though, I cannot point you to a review of recent advances), but it has not replaced other approaches, as is visible in the number of QSAR/descriptor applications of the CDK.

Functional Group Ontology
Sankar et al. have developed an ontology for functional groups (doi:10.1016/j.jmgm.2013.04.003). One popular thought is that subgroups of atoms are more important than the molecule as a whole. Much of our cheminformatics is based on this idea. And it matches what we experimentally observe. If we add a hydroxyl or an acid group, the molecule becomes more hydrophylic. Semantically encoding this clearly important information seems important, though intuitively I would have left this to the cheminformatics tools. This paper and a few cited papers, however, show far you can take this. It organizes more than 200 functional groups, but I am not sure where the ontology can be downloaded.

Sankar, P., Krief, A., Vijayasarathi, D., Jun. 2013. A conceptual basis to encode and detect organic functional groups in XML. Journal of Molecular Graphics and Modelling 43, 1-10. URL http://dx.doi.org/10.1016/j.jmgm.2013.04.003

Linking biological to chemical similarities
If we step aside from our concept of "functional group", we can also just look at whatever is similar between molecules. Michael Kuhn et al. (of STITCH and SIDER) looked into the role of individual proteins in side effect (doi:10.1038/msb.2013.10). They find that many drug side effects are mediated by a selection of individual proteins. The study uses a drug-target interaction data set, and to reduce the change of bias due to some compound classes more extensively studies (more data), they removed too similar compounds from the data set, using the CDK's Tanimoto stack.

Kuhn, M., Al Banchaabouchi, M., Campillos, M., Jensen, L. J., Gross, C., Gavin, A. C., Bork, P., Apr. 2014. Systematic identification of proteins that elicit drug side effects. Molecular Systems Biology 9 (1), 663. URL http://dx.doi.org/10.1038/msb.2013.10

Drug-induced liver injury
These approaches can also be used to study if there are structural reasons why Drug-induced liver injury (DILI) occurs. This was studied in this paper Zhu et al. where the CDK is used to calculate topological descriptors (doi:10.1002/jat.2879). They compared explanatory models that correlate descriptors with the measured endpoint and a combination with hepatocyte imaging assay technology (HIAT) descriptors. These descriptors capture phenotypes such as nuclei count, nuclei area, intensities of reactive oxygen species intensity, tetramethyl rhodamine methyl ester, lipid intensity, and glutathione. It doesn't cite any of the CDK papers, so I left a comment with PubMed Commons.

Zhu, X.-W., Sedykh, A., Liu, S.-S., Mar. 2014. Hybrid in silico models for drug-induced liver injury using chemical descriptors and in vitro cell-imaging information. Journal of Applied Toxicology 34 (3), 281-288. URL http://dx.doi.org/10.1002/jat.2879
Posted Sun Jul 12 09:39:00 2015 Tags:
Tool validation
The first paper this week is a QSAR paper. In fact, it does some interesting benchmarking of a few tools with a data set of about 6000 compounds. It includes looking into the applicability domain, and studies the error of prediction for compounds inside and outside the chemical space defined by the training set. The paper indirectly uses the CDK descriptor calculation corner, by using EPA's T.E.S.T. toolkit (at least one author, Todd Martin, contributed to the CDK).

Callahan, A., Cruz-Toledo, J., Dumontier, M., Apr. 2013. Ontology-Based querying with Bio2RDF's linked open data. Journal of Biomedical Semantics 4 (Suppl 1), S1+. URL http://dx.doi.org/10.1186/2041-1480-4-s1-s1

Tetranortriterpenoid
Arvind et al. study tetranortriterpenoids using a QSAR approach involving COMFA and the CPSA descriptor (green OA PDF). The latter CDK descriptor is calculated using Bioclipse. The study finds that using compound classes can improve the regression.

Arvind, K., Anand Solomon, K., Rajan, S. S., Apr. 2013. QSAR studies of tetranortriterpenoid: An analysis through CoMFA and CPSA parameters. Letters in Drug Design & Discovery 10 (5), 427-436. URL http://dx.doi.org/10.2174/1570180811310050010

Accurate monoisotopic masses
Another useful application of the CDK is the Java wrapping of the isotope data in the Blue Obelisk Data Repository (BODR). Mareile Niesser et al. use Rajarshi's rcdk package for R to calculate the differences in accurate monoisotopic masses. They do not cite the CDK directly, but do mention it by name in the text.

Niesser, M., Harder, U., Koletzko, B., Peissner, W., Jun. 2013. Quantification of urinary folate catabolites using liquid chromatography–tandem mass spectrometry. Journal of Chromatography B 929, 116-124. URL http://dx.doi.org/10.1016/j.jchromb.2013.04.008
Posted Sat Jul 11 13:22:00 2015 Tags:

Last night f2pool mined a 1MB block containing a single 1MB transaction.  This scooped up some of the spam which has been going to various weakly-passworded “brainwallets”, gaining them 0.5569 bitcoins (on top of the normal 25 BTC subsidy).  You can see the megatransaction on blockchain.info.

It was widely reported to take about 25 seconds for bitcoin core to process this block: this is far worse than my “2 seconds per MB” result in my last post, which was considered a pretty bad case.  Let’s look at why.

How Signatures Are Verified

The algorithm to check a transaction input (of this form) looks like this:

  1. Strip the other inputs from the transaction.
  2. Replace the input script we’re checking with the script of the output it’s trying to spend.
  3. Hash the resulting transaction with SHA256, then hash the result with SHA256 again.
  4. Check the signature correctly signed that hash result.

Now, for a transaction with 5570 inputs, we have to do this 5570 times.  And the bitcoin core code does this by making a copy of the transaction each time, and using the marshalling code to hash that; it’s not a huge surprise that we end up spending 20 seconds on it.

How Fast Could Bitcoin Core Be If Optimized?

Once we strip the inputs, the result is only about 6k long; hashing 6k 5570 times takes about 265 milliseconds (on my modern i3 laptop).  We have to do some work to change the transaction each time, but we should end up under half a second without any major backflips.

Problem solved?  Not quite….

This Block Isn’t The Worst Case (For An Optimized Implementation)

As I said above, the amount we have to hash is about 6k; if a transaction has larger outputs, that number changes.  We can fit in fewer inputs though.  A simple simulation shows the worst case for 1MB transaction has 3300 inputs, and 406000 byte output(s): simply doing the hashing for input signatures takes about 10.9 seconds.  That’s only about two or three times faster than the bitcoind naive implementation.

This problem is far worse if blocks were 8MB: an 8MB transaction with 22,500 inputs and 3.95MB of outputs takes over 11 minutes to hash.  If you can mine one of those, you can keep competitors off your heels forever, and own the bitcoin network… Well, probably not.  But there’d be a lot of emergency patching, forking and screaming…

Short Term Steps

An optimized implementation in bitcoind is a good idea anyway, and there are three obvious paths:

  1. Optimize the signature hash path to avoid the copy, and hash in place as much as possible.
  2. Use the Intel and ARM optimized SHA256 routines, which increase SHA256 speed by about 80%.
  3. Parallelize the input checking for large numbers of inputs.

Longer Term Steps

A soft fork could introduce an OP_CHECKSIG2, which hashes the transaction in a different order.  In particular, it should hash the input script replacement at the end, so the “midstate” of the hash can be trivially reused.  This doesn’t entirely eliminate the problem, since the sighash flags can require other permutations of the transaction; these would have to be carefully explored (or only allowed with OP_CHECKSIG).

This soft fork could also place limits on how big an OP_CHECKSIG-using transaction could be.

Such a change will take a while: there are other things which would be nice to change for OP_CHECKSIG2, such as new sighash flags for the Lightning Network, and removing the silly DER encoding of signatures.

Posted Wed Jul 8 03:09:59 2015 Tags:

Since I was creating large blocks (41662 transactions), I added a little code to time how long they take once received (on my laptop, which is only an i3).

The obvious place to look is CheckBlock: a simple 1MB block takes a consistent 10 milliseconds to validate, and an 8MB block took 79 to 80 milliseconds, which is nice and linear.  (A 17MB block took 171 milliseconds).

Weirdly, that’s not the slow part: promoting the block to the best block (ActivateBestChain) takes 1.9-2.0 seconds for a 1MB block, and 15.3-15.7 seconds for an 8MB block.  At least it’s scaling linearly, but it’s just slow.

So, 16 Seconds Per 8MB Block?

I did some digging.  Just invalidating and revalidating the 8MB block only took 1 second, so something about receiving a fresh block makes it worse. I spent a day or so wrestling with benchmarking[1]…

Indeed, ConnectTip does the actual script evaluation: CheckBlock() only does a cursory examination of each transaction.  I’m guessing bitcoin core is not smart enough to parallelize a chain of transactions like mine, hence the 2 seconds per MB.  On normal transaction patterns even my laptop should be about 4 times faster than that (but I haven’t actually tested it yet!).

So, 4 Seconds Per 8MB Block?

But things are going to get better: I hacked in the currently-disabled libsecp256k1, and the time for the 8MB ConnectTip dropped from 18.6 seconds to 6.5 seconds.

So, 1.6 Seconds Per 8MB Block?

I re-enabled optimization after my benchmarking, and the result was 4.4 seconds; that’s libsecp256k1, and an 8MB block.

Let’s Say 1.1 Seconds for an 8MB Block

This is with some assumptions about parallelism; and remember this is on my laptop which has a fairly low-end CPU.  While you may not be able to run a competitive mining operation on a Raspberry Pi, you can pretty much ignore normal verification times in the blocksize debate.


 

[1] I turned on -debug=bench, which produced impenetrable and seemingly useless results in the log.

So I added a print with a sleep, so I could run perf.  Then I disabled optimization, so I’d get understandable backtraces with perf.  Then I rebuilt perf because Ubuntu’s perf doesn’t demangle C++ symbols, which is part of the kernel source package. (Are we having fun yet?).  I even hacked up a small program to help run perf on just that part of bitcoind.   Finally, after perf failed me (it doesn’t show 100% CPU, no idea why; I’d expect to see main in there somewhere…) I added stderr prints and ran strace on the thing to get timings.

Posted Mon Jul 6 21:58:18 2015 Tags:

I'm seeing plenty of people, including some non-profit organizations along with the usual punditocracy, opining on the USA Supreme Court's denial for a writ of certiorari in the Oracle v. Google copyright infringement case. And, it's not that I expect everyone in the world to read my blog, but I'm amazed that people who should know better haven't bothered to even read the lower Court's decision, which is de-facto upheld upon denial by the Supreme Court to hear the appeal.

I wrote at great length about why the decision isn't actually a decision about whether APIs are copyrightable, and that the decision actually gives us some good clarity with regard to the issue of combined work distribution (i.e., when you distribute your own works with the copyrighted material of others combined into a single program). The basic summary of the blog post I linked to above is simply: The lower Court seemed genially confused about whether Google copy-and-pasted code, as the original trial seems to have inappropriately conflated API reimplemenation with code cut-and-paste.

No one else has addressed this nuance of the lower Court's decision in the year since the decision came down, and I suspect that's because in our TL;DR 24-hour-news cycle, it's much easier for the pundits and organizations tangentially involved with this issue to get a bunch of press over giving confusing information.

So, I'm mainly making this blog post to encourage people to go back and read the decision and my blog post about it. I'd be delighted to debate people if they think I misread the decision, but I won't debate you unless you assure me you read the lower Court's decision in its entirety. I think that leaves virtually no one who will. :-/

Posted Sat Jul 4 20:30:00 2015 Tags:

I'm seeing plenty of people, including some non-profit organizations along with the usual punditocracy, opining on the USA Supreme Court's denial for a writ of certiorari in the Oracle v. Google copyright infringement case. And, it's not that I expect everyone in the world to read my blog, but I'm amazed that people who should know better haven't bothered to even read the lower Court's decision, which is de-facto upheld upon denial by the Supreme Court to hear the appeal.

I wrote at great length about why the decision isn't actually a decision about whether APIs are copyrightable, and that the decision actually gives us some good clarity with regard to the issue of combined work distribution (i.e., when you distribute your own works with the copyrighted material of others combined into a single program). The basic summary of the blog post I linked to above is simply: The lower Court seemed genially confused about whether Google copy-and-pasted code, as the original trial seems to have inappropriately conflated API reimplemenation with code cut-and-paste.

No one else has addressed this nuance of the lower Court's decision in the year since the decision came down, and I suspect that's because in our TL;DR 24-hour-news cycle, it's much easier for the pundits and organizations tangentially involved with this issue to get a bunch of press over giving confusing information.

So, I'm mainly making this blog post to encourage people to go back and read the decision and my blog post about it. I'd be delighted to debate people if they think I misread the decision, but I won't debate you unless you assure me you read the lower Court's decision in its entirety. I think that leaves virtually no one who will. :-/

Posted Sat Jul 4 20:15:09 2015 Tags:

Welcome to my first new-project release of the year, git-weave. It’s a polished and documented version of the script I used to reconstruct the early history of INTERCAL five years ago – see Risk, Verification, and the INTERCAL Reconstruction Massacree for the details on that one.

git-weave can be used to explode a git repository into a sequence of per-commit directory trees accompanied by a metadata file describing parent-child linkage, holding committer/author/timestamps/comment metadata, and carrying tags.

Going in the other direction, it can take the same sequence of trees plus metadata file and reconstruct the live repository. Round-tripping is lossless.

What it’s really useful for is reconstructing a partial but useful ancient history of a project from before it was put under version control. Find its release archives, synthesize a metadata file, apply this tool, and you get a repository that can easily be glued to the modern, more continuous history.

Yes, you only get a commit for each release tree or patch you can dig up, but this is better than nothing and often quite interesting.

Nifty detail: the project logo is the ancient Egyptian hieroglyph for a weaver’s shuttle.

Posted Sat Jul 4 05:10:03 2015 Tags:

Linux’s perf competes with early git for title of least-friendly Linux tool.  Because it’s tied to kernel versions, and the interfaces changes fairly randomly, you can never figure out how to use the version you need to use (hint: always use -g).

But when it works, it’s very useful.  Recently I wanted to figure out where bitcoind was spending its time processing a block; because I’m a cool kid, I didn’t use gprof, I used perf.  The problem is that I only want information on that part of bitcoind.  To start with, I put a sleep(30) and a big printf in the source, but that got old fast.

Thus, I wrote “perfme.c“.  Compile it (requires some trivial CCAN headers) and link perfme-start and perfme-stop to the binary.  By default it runs/stops perf record on its parent, but an optional pid arg can be used for other things (eg. if your program is calling it via system(), the shell will be the parent).

Posted Fri Jul 3 03:19:12 2015 Tags:
After applying a patch that moves a bulk of code that was placed in a wrong file to its correct place, a quick way to sanity-check that the patch does not introduce anything unexpected is to run "git blame -C -M" between HEAD^ and HEAD, like this:

  $ git blame -C -M HEAD^..HEAD -- new-location.c

This should show that the lines moved from the old location in the output as coming from there; lines blamed for the new commit (i.e. not coming from the old location) can then be inspected more carefully to see if it makes sense.

One problem I had while doing exactly that today was that most of the screen real-estate on my 92-column wide terminal was taken by the author name and the timestamp, and I found myself pressing right and left arrow in my pager to scroll horizontally a lot, which was both frustrating and suboptimal.

  $ git blame -h

told me that there is "git blame -s" to omit that information. I thought that I didn't know about the option. Running "git blame" on its source itself revealed that the option was added by me 8 years ago, and it wasn't that I didn't know but I simply forgot ;-)
Posted Mon Jun 29 16:23:00 2015 Tags:
One of the questions I had in the hackathon today is about how to use the CDK to convert SMILES string into InChIs and InChIKeys (see doi:10.1186/1758-2946-5-14). So, here goes. This is the Groovy variant, though you can access the CDK just as well in other programming languages (Python, Java, JavaScript). We'll use the binary jar for CDK 1.5.10.  We can then run code, say test.groovy, using the CDK with:

groovy -cp cdk-1.5.10.jar test.groovy

With that out of the way, let's look at the code. Let's assume we start with a text file with one SMILES string on each line, say test.smi, then we parse this file with:

new File("test.smi").eachLine { line ->
  mol = parser.parseSmiles(line)
}

This already parses the SMILES string into a chemical graph. If we pass this to the generator to create an InChIKey, we may get an error, so we do an extra check:

gen = factory.getInChIGenerator(mol)
if (gen.getReturnStatus() == INCHI_RET.OKAY) {
  println gen.inchiKey;
} else {
  println "# error: " + gen.message
}

If we combine these two bits, we get a full test.groovy program:

import org.openscience.cdk.silent.*
import org.openscience.cdk.smiles.*
import org.openscience.cdk.inchi.*
import net.sf.jniinchi.INCHI_RET

parser = new SmilesParser(
  SilentChemObjectBuilder.instance
)
factory = InChIGeneratorFactory.instance

new File("test.smi").eachLine { line ->
  mol = parser.parseSmiles(line)
  gen = factory.getInChIGenerator(mol)
  if (gen.getReturnStatus() == INCHI_RET.OKAY) {
    println gen.inchiKey;
  } else {
    println "# error: " + gen.message
  }
}

Update: John May suggested an update, which I quite like. If the result is not 100% okay, but the InChI library gave a warning, it still yields an InChIKey which we can output, along with the warning message. For this, replace the if-else statement with this code:

if (gen.returnStatus == INCHI_RET.OKAY) {
  println gen.inchiKey;
} else if (gen.returnStatus == INCHI_RET.WARNING) {
  println gen.inchiKey + " # warning: " + gen.message;
} else {
  println "# error: " + gen.message

}

Posted Sun Jun 28 22:36:00 2015 Tags:

I've been otherwise impressed with John Oliver and his ability on Last Week Tonight to find key issues that don't have enough attention and give reasonably good information about them in an entertaining way — I even lauded Oliver's discussion of non-profit organizational corruption last year. I suppose that's why I'm particularly sad (as I caught up last weekend on an old episode) to find that John Oliver basically fell for the large patent holders' pro-software-patent rhetoric on so-called “software patents”.

In short, Oliver mimics the trade association and for-profit software industry rhetoric of software patent reform rather than abolition — because trolls are the only problem. I hope the worlds' largest software patent holders send Oliver's writing staff a nice gift basket, as such might be the only thing that would signal to them that they fell into this PR trap. Although, it's admittedly slightly unfair to blame Oliver and his writers; the situation is subtle.

Indeed, someone not particularly versed in the situation can easily fall for this manipulation. It's just so easy to criticize non-practicing entities. Plus, the idea that the sole inventor might get funded on Shark Tank has a certain appeal, and fits a USAmerican sensibility of personal capitalistic success. Thus, the first-order conclusion is often, as Oliver's piece concludes, maybe if we got rid of trolls, things wouldn't be so bad.

And then there's also the focus on the patent quality issue; it's easy to convince the public that higher quality patents will make it ok to restrict software sharing and improvement with patents. It's great rhetoric for a pro-patent entities to generate outrage among the technology-using public by pointing to, say, an example of a patent that reads on every Android application and telling a few jokes about patent quality. In fact, at nearly every FLOSS conference I've gone to in the last year, OIN has sponsored a speaker to talk about that very issue. The jokes at such talks aren't as good as John Oliver's, but they still get laughs and technologists upset about patent quality and trolls — but through carefully cultural engineering, not about software patents themselves.

In fact, I don't think I've seen a for-profit industry and its trade associations do so well at public outrage distraction since the “tort reform” battles of the 1980s and 1990s, which were produced in part by George H. W. Bush's beloved M.C. Rove himself. I really encourage those who want to understand of how the anti-troll messaging manipulation works to study how and why the tort reform issue played out the way it did. (As I mentioned on the Free as in Freedom audcast, Episode 0x13, the documentary film Hot Coffee is a good resource for that.)

I've literally been laughed at publicly by OIN representatives when I point out that IBM, Microsoft, and other practicing entities do software patent shake-downs, too — just like the trolls. They're part of a well-trained and well-funded (by trade associations and companies) PR machine out there in our community to convince us that trolls and so-called “poor patent quality” are the only problems. Yet, nary a year has gone in my adult life where I don't see a some incident where a so-called legitimate, non-obvious software patent causes serious trouble for a Free Software project. From RSA, to the codec patents, to Microsoft FAT patent shakedowns, to IBM's shakedown of the Hercules open source project, to exfat — and that's just a few choice examples from the public tip of the practicing entity shakedown iceberg. IMO, the practicing entities are just trolls with more expensive suits and proprietary software licenses for sale. We should politically oppose the companies and trade associations that bolster them — and call for an end to software patents.

Posted Fri Jun 26 19:25:00 2015 Tags: