JPEG 2000 on Ubuntu — without anyone getting stabbed

Aargh! Ubuntu 16.10 has decided that ImageMagick doesn’t need JPEG 2000 support, and will quietly (and very very wrongly) write JP2s as JPEG.

(NB: JPEG 2000 images still maybe crash Ubuntu’s file browser in 14.10. My old installation didn’t like them, but my reinstall seems quite happy. Go figure.)

JPEG 2000 is a great image file format: well-defined, and able to store high quality photographic data in a very small space. It truly is the JPEG of the 2000s — except for its dismal support under Ubuntu.

The problem is the patents. An open library has been a long time coming, and lots of Linux software is built without JP2 support. This helped keep it away from my desktop.

Under Ubuntu 14.04, here’s what does and doesn’t support JP2 files:

  • Gimp — not supported. It appears to have a non-functioning plugin that tries to read the file, then gives up. This is annoying, as Gimp is defined as the system default viewer for JPEG 2000.
  • Image Viewer — does support JP2, but occasionally mis-renders pages. To make this the default, right-click on a JP2 file, and select Open with → Other Application …, then choose Image Viewer. It should work from then onwards.
  • Document Viewer — a bit rough when looking at JPEG 2000-encoded PDFs. Very slow, too.
  • GraphicsMagick — seems to be the most painless way of converting graphics files to JPEG 2000. My preferred method of invoking it is:
    gm convert -define 'jp2:rate=0.008' in.png out.jp2
    The rate option should be a small number; the smaller, the greater the compression, and the worse the image quality.
  • OpenJPEG — provides the image_to_j2k and j2k_to_image tools. Far more picky about input formats than it should be, and often fails on seemingly perfect input.
  • img2pdf — (built from source)  is a tiny gem of a package. All it does is wrap various image formats into a PDF file. It doesn’t modify the image data in any way, so with a bit of ingenuity (and pdftk) you can use PDF as a true metafile archive. You can view the content on any platform, but get the source images out bit-for-bit perfect. We used to call files which could contain files metafiles, but that stopped being popular when TIFF started to be a baroque travesty of an image container back in the mid-1990s.
  • poppler — (for full features, build from source) has a tool, pdfimages, which can extract embedded image files from PDFs. Some of the metadata might get lost, but all of the image bits come through.

Since JPEG 2000 isn’t included in web browsers (grar), I’ve embedded a sample scanned JPEG into a PDF, and added a series of progressively more compressed JPEG 2000 versions: JPEG-2000Booklet [PDF].  The booklet has notes showing the byte size of each page. The image still looks pretty good at 8% of the original file size!

Creating secure digital QSL cards with your LoTW certificate

Hams have sent out QSL cards since the hobby began. It may be less popular than it used to be, because mail seems slow now (especially when QSL bureaux are used), and there are online alternatives like eQSL, QRZ.com, and the ARRL‘s Logbook of the World (LoTW).

LoTW has been criticized for being too complex. It uses an X.509 public key signature to verify your submitted log entries, and a QSL is confirmed when the other party sends in a matching entry. In a way, it’s rather elegant, as the LoTW server does the work of matching the entries, so QSLs “magically” appear in your log. You don’t get a fancy QSL card in the mail or on your screen, and I think a lot of users miss that.

X.509 might be a bit unweildy, but props to ARRL for setting up a industry standard, robust (-ish; X.509 has its flaws), general purpose signing infrastructure. Since other file formats — notably PDF — support X.509 signing, you can use your LoTW certificate to make other data tamper-evident.

Here I document a method of creating a digital QSL card that can be e-mailed, and subsequently verified by the recipient as being legitimate. Any alterations to the file will break the signature, and the file will just appear as a regular PDF (or not display properly). The process can be used to sign any other Acrobat file. There are probably more streamlined ways to do this, but I only came up with it last night as the beginning of a scriptable solution.

You will need:

  • Your ARRL LoTW electronic certificate file. This is called callsign.p12; mine is VA3PID.P12
  • A means of making a QSL card as a PDF fill-in form. I used Scribus; it’s a free but powerful DTP system.
  • Pdftk, “the” PDF toolkit, to apply the QSL details to the PDF form. If you’re on Windows, you might want the Pdftk GUI.
  • PortableSigner, a java application for signing Acrobat files using an X.509 certificate.

(Quite a bit of what follows was learnt from the two pages Your first PDF form with Scribus – Scribus Wiki and filling in forms with pdftk, so thanks to the authors of those for the guidance.)

First, make your QSL card. Since you’re not going to print this, it can be any size you want, but postcard size is standard. At the very least, create spaces for the recipient callsign, the date, the time, frequency, mode used, and signal report.

Under these headings, I’ve made six PDF text form fields. Scribus creates form fields like text frames/boxes. I used plain text fields (which are selected by this icon: ), centred text, and with the name of the field set from the PDF Options → Field Properties context menu. Each field needs a different name. I used callsign, date, utc, mhz, mode, and report.

Save your QSL card as PDF. It might be an idea to check it to see if the form fields are really there and editable:

Now it starts to get really nerdy. Adobe specified the Forms Data Format (FDF) to allow PDF form data to be slung around. FDF looks a bit like PostScript or raw PDF:

%FDF-1.2

1 0 obj
<<
/FDF << /Fields 2 0 R>>
>>
endobj
2 0 obj
[<< /T (callsign) /V (VE3KPK) >>
<< /T (date) /V (2011-10-02) >>
<< /T (mhz) /V (7.03581) >>
<< /T (mode) /V (CW) >>
<< /T (report) /V (499) >>
<< /T (utc) /V (2341) >>
]
endobj
trailer
<< /Root 1 0 R >>
%%EOF

If you think of the T & V values above as Tag and Value, you can see that the file defines callsign=VE3KPK, date=2011-10-02, mode=CW, and so on. This step can be easily scripted. If you’re not sure what the fields are called, pdftk has a “dump_data_fields” option that spits out the field names as plain text.

pdftk is also used to put the data from the FDF file into the PDF template. It’s a slightly hairy command line:

pdftk QSL-blank.pdf fill_form VE3KPK.fdf output QSL-VE3KPK.pdf flatten

Here the source file is QSL-blank.pdf, the FDF data is VE3KPK.fdf, and the output is to QSL-VE3KPK.pdf. The flatten option turns the pdf form into regular, uneditable PDF. (These details are from a real QSO, by the way, and by utter coincidence I’m writing this in a hotel in Ken VE3KPK’s home town in Northern Ontario.)

Checking that this worked in Adobe Reader:

For many people this is probably enough (or perhaps, too much already!) but I really want to have a digital QSL card that will stand up to some scrutiny. This is where your LoTW certificate file comes in.

PortableSigner is a java application for signing PDF files. It seems quite happy signing the files made in this workflow. It can run from the command line, or as a windowed application:

You use your .p12 signature file and its password to sign the PDF. Once the file is signed, you can send it to your contact, and they can prove (and hopefully, any certification/contest agency will agree) that the contact was confirmed.

Viewing the signed document is deeply unimpressive:

It’s not such a big deal that Reader says that “the validity of the certification is unknown”. It’s just that Adobe doesn’t have the ARRL’s certificate loaded into everyone’s reader (what!? mock outrage!) and so it doesn’t match a certificate it knows about. You have to dig a bit deeper into the signature panel to check out who is responsible for this.

Well, that’s a start; at least it was signed by someone with my e-mail address. There’s more under Show Signature Properties:

There’s the ARRL signature in there. Buried deep in another properties tree is my callsign; can’t find it today, but I saw it in there last night. Either way, the digital QSL PDF is now signed and certified that it came from me, as an ARRL LoTW user.