Build a Package from Source on Ubuntu

Thursday, November 11, 2021

A Chance Find

Today I came across a blog post that someone named Lawrence Kesteloot wrote in 2010 called Package Managers Hurt Open Source. He writes:

In the good old days, before fancy modern package managers like apt-get and yum, if you wanted an open-source program you had to download the source tarball, compile it, and install it.

I am, barely, old enough to remember the tail end of those days (Wikipedia says the initial release of APT was in 1998 and YUM followed in 2002). When I first started using Linux in 1996, if memory serves, we did have packages, but you just had to install them with rpm or dpkg (or whatever Slackware used and probably still uses). Aside from the fact that dependency resolution was manual, so you just had to figure it out the best you could, there were plenty of programs nobody had gotten around to packaging yet. So even with packages, there was a fair amount of trying to get things to compile and run.

Kesteloot’s point is that back when you were “forced to do the work” anyway, if the software didn’t behave the way you liked, you were already working with the source code and might be more amenable to taking the time to change that behavior. I don’t know whether that’s supportable or not, but anyway, it’s not what I want to talk about.

Instead, what I want to talk about is a sentence in his last paragraph:

I wish Ubuntu had a flag to apt-get that would install from source Gentoo-style and keep the whole thing in your home directory where you could muck with it.

Wish no more! Debian’s, and therefore Ubuntu’s, apt-get can indeed do this, and has been able to for a long time. Actually, it’s been able to do so for a very long time, since well before Kesteloot wrote in 2010. I figured writing a blog post about it was a better use of my time and his than sending him an e-mail letting him know I can help with a problem he briefly had eleven years ago.

A Historical Digression

But first, some personal history. Twenty-three years ago, I was in my freshman year of college at the University of Mississippi. Although the Internet certainly wasn’t new in 1998—and the juggernaut was clearly coming—in some ways it was still early days yet. I came to campus with a computer—a dull grey HP Pavilion which had been a high school graduation present—but I had to buy an Ethernet card for it when I got there. The school IT department then gave me a static, publicly routable IP address complete with A and PTR records in DNS. As I say, it was a different time.

To check their university e-mail, most people telnetted (SSH had been invented in 1995, but wasn’t yet widespread) into the mailserver, and ran the e-mail client. I’ve heard of other mail clients from the time, but at Ole Miss we used Pine. Pine had been released by the University of Washington with source code available under a BSD-style license beginning in 1992, but at some point thereafter had taken the position that it would consider legal action against anyone distributing a modified version. Wikipedia claims this was in 1996 with version 3.92, but their citation for it is a link to a post on the debian-legal mailing list from 2002, which references a different post to the same list from 2000, neither of which specify a date or a version number. (I stopped digging at this point, since the exact date doesn’t really matter.) The latter is a reply to yet another e-mail, this one from Richard Stallman, and quotes him as saying:

[The University of Washington’s] position was that the words “permission to copy, distribute and modify” [in the license] do not grant permission to distribute a modified version. In other words, they say you can distribute the software, and you can modify the software, but you can’t modify it and then distribute the result.

This strikes me now, as it did then, as not particularly logical, but as my father (a lawyer himself) once told me, you have to remember that legal language doesn’t say what it means, it says what it says. In any case, it was UW’s position.

I don’t remember the details if I ever knew them, but my recollection is that Pine needed some sort of patch—i.e., a modification—to operate properly on Debian. And since none of the Debian packagers wanted to get sued, whether or not they thought they could prevail, they didn’t package it. But lots of Debian users still wanted to use Pine.

By that point, “Debian users” included me; and I had discovered that because the mailserver supported POP3, instead of telnetting to it to use Pine, I could use Fetchmail to bring my e-mail to my computer and read it there with any mail client I wanted. And I could get a notification when I had new mail! (Another way things have changed: these days I try to get as few notifications as possible.) So I read my mail locally first with Elm (briefly) and then with Mutt, but they were different, and I was used to Pine.

So the way to do it was:

  1. Download the Pine source code.
  2. Download needed patches and apply them.
  3. Compile your own “modified” Pine binary and package it up.

All above board and comforming with UW’s idiosyncratic interpretation of the license text. And Debian (and, by inheritance, Ubuntu) gave you the tools, whether that’s to get around a licensing issue or simply because you want to scratch an itch and add a flag to netcat (as Kesteloot did).

The Meat

All this is basically a long-winded, roundabout way of referring you to section 6.1.1 of the Debian Handbook. APT supports not just binary repositories, but source repositories, and has built-in commands to create your own installable packages therefrom, after fiddling with the source.

For purposes of demonstration, I’ll use GNU hello.

Edit sources.list

In /etc/apt/sources.list there will be at least one line, and probably several, starting with deb. Here is the first one from mine:

deb http://us.archive.ubuntu.com/ubuntu/ focal main restricted

Create a new line that’s just the same, but replace deb with deb-src. That is:

deb http://us.archive.ubuntu.com/ubuntu/ focal main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ focal main restricted

You’ll then need to run sudo apt-get update.

Download the Prerequisites

First, install your tools. Fortunately, there is a convenient metapackage: sudo apt-get install build-essential.

Then install the dependencies for the specific package: sudo apt-get build-dep hello.

Download, Build, and Install

Note that unlike downloading and installing binary packages, this step does not require administrative privileges: apt-get source hello. This does three things:

  • Downloads the files:
    • hello_2.10-2ubuntu2.dsc (the description file)
    • hello_2.10.orig.tar.gz (the original source from upstream)
    • hello_2.10-2ubuntu2.debian.tar.xz (Debian/Ubuntu-specific customizations).
  • Unpacks the upstream source, in this case into ./hello-2.10
  • Unpacks the customizations into ./hello-2.10/debian

You can then do whatever you want with the source. Once you’re ready, in ./hello-2.10 execute dpkg-buildpackage --no-sign. (The flag here is because official packages are signed by the maintainer’s key, which you obviously don’t have.) This will build the source and create the package in your original directory. (If you don’t want to change anything you can combine the download and build steps with apt-get --compile source hello.) You can then install with dpkg -i.

At a very basic level that’s essentially it, but as should go without saying, of course this is really the tip of the iceberg. For more details, see the Debian New Maintainers’ Guide.

Epilogue

If this is barely the tip of the iceberg, I hear you ask, why write about it at all? No reason, really, except that I find a lot of my conversations with my colleagues go something like “Did you know that you can do x with y?” “No, really!?” (I’m on each side about half the time.) And it occurred to me when I read the blog post above that I’d never had a conversation about this particular usage. Maybe it’ll help someone else, maybe not. But it was fun to brush up on things I haven’t thought about in twenty years.