I’m on a major decluttering toot. When I realised that the filing cabinet I bought three years ago would no longer close with all the papers stuffed in it, I knew something had to change. I’ve been shredding like it’s Houston in 2001. I have the duplex scanner to suck in the stuff I need to keep. I’m moving to paperless wherever possible to stop it building up again.
My bank provides PDF statements. Of this I approve. PDF is almost perfect for this: it provides an electronic version of the page, but with searchable text and the potential for some level of security. Except, this is not the way that my bank does it. At first glance, the text looks pretty harmless:
Zoom in, and it gets a bit blocky:
Zoom right in:
Aargh! Blockarama! Did they really store text as bitmaps? Sure enough, pdftotext output from the files contains no text. Running pdfimages produces hundreds of tiny images; here’s just a few:
Dear oh dear. This format is the worst of electronic, combined with paper’s lack of computer indexability. The producer claims to be Xenos D2eVision. Smooth work there, Xenos.
So, how can I fix this? It’s a bit of a pain to set this workflow up, but what I’ve done is:
Convert the PDF to individual TIFF files at 300 dpi. Ghostscript is good for this: gs -SDEVICE=tiffg4 -r300x300 -sOutputFile=file%03d.tif -dNOPAUSE -dBATCH -- file.pdf
Run Tesseract OCR on the TIFF files to make hOCR output: for f in file*tif
do
tesseract $f `basename $f` hocr
done Update: Cuneiform seems to work better than Tesseract when feeding pdfbeads: for f in file*tif
do
cuneiform -f hocr -o `basename $f .tif`.html $f
done
Move the images and the hOCR files to a new folder; the filenames must correspond, so file001.tif needs file001.html, file002.tif file002.html, etc.
In the new folder, run pdfbeads * > ../Output.pdf
The files are really small, and the text is recognized pretty well. It still looks pretty bad:
The files are downloadable as PDF, too. They look pretty decent on my e-reader:
(and yes, that is really an article about making a contact over 121km using a 5mW laser)
I don’t think any of the editions before 2012 will be going online. It would be nice, but RAC is severely limited in resources. The almost total lack of fanfare is a contrast to the ARRL’s digital QST, which is much announced but not actually available yet …
I like epost. I’d like it even more if they hurried up and processed my direct payment ability — which required a form and a void cheque mailed to an address in Toronto — but it’s a pretty good service. I get my bills, viewable and payable online, on the day of issue. No paper. This is good.
This is good because every single filing container I buy eventually ends up full of (paid) bills and financial administrivia. Less paper = less messy Stewart = happy Stewart. Some messes, like my electronics table, could be classed as glorious, however, and therefore joyous in their creation and use. Not all tidiness is good.
So I got my first Visa bill by e-post. Yay! Reviewed it, paid it. No hassle. But since this a PDF facsimile of my bill, something mighty odd has happened to my address:
It’s a perfect substitution cypher of my name and address. I’ve been out of the prepress industry for long enough not to immediately recognize it as a font encoding error. I’m confused why it might have A, T, E & N, but no M. Odd indeed.
Following on from Creating secure digital QSL cards with your LoTW certificate, here’s a Bash script to generate encrypted signed PDF QSLs. You will need to edit the certificate file name, the QSL blank file name, your call sign, your LoTW password and the PDF encryption password. After doing so, please keep the script safe, as whoever has your LoTW password can pretend to be you.
The only checks that this script doesn’t do (and probably should) are if you have pdftk and PortableSigner executables in your path. PortableSigner is rather weird the way it runs; you need to specify full paths for all files, or it dies.
Hams have sent out QSL cards since the hobby began. It may be less popular than it used to be, because mail seems slow now (especially when QSL bureaux are used), and there are online alternatives like eQSL, QRZ.com, and the ARRL‘s Logbook of the World (LoTW).
LoTW has been criticized for being too complex. It uses an X.509 public key signature to verify your submitted log entries, and a QSL is confirmed when the other party sends in a matching entry. In a way, it’s rather elegant, as the LoTW server does the work of matching the entries, so QSLs “magically” appear in your log. You don’t get a fancy QSL card in the mail or on your screen, and I think a lot of users miss that.
X.509 might be a bit unweildy, but props to ARRL for setting up a industry standard, robust (-ish; X.509 has its flaws), general purpose signing infrastructure. Since other file formats — notably PDF — support X.509 signing, you can use your LoTW certificate to make other data tamper-evident.
Here I document a method of creating a digital QSL card that can be e-mailed, and subsequently verified by the recipient as being legitimate. Any alterations to the file will break the signature, and the file will just appear as a regular PDF (or not display properly). The process can be used to sign any other Acrobat file. There are probably more streamlined ways to do this, but I only came up with it last night as the beginning of a scriptable solution.
You will need:
Your ARRL LoTW electronic certificate file. This is called callsign.p12; mine is VA3PID.P12
A means of making a QSL card as a PDF fill-in form. I used Scribus; it’s a free but powerful DTP system.
Pdftk, “the” PDF toolkit, to apply the QSL details to the PDF form. If you’re on Windows, you might want the Pdftk GUI.
PortableSigner, a java application for signing Acrobat files using an X.509 certificate.
First, make your QSL card. Since you’re not going to print this, it can be any size you want, but postcard size is standard. At the very least, create spaces for the recipient callsign, the date, the time, frequency, mode used, and signal report.
Under these headings, I’ve made six PDF text form fields. Scribus creates form fields like text frames/boxes. I used plain text fields (which are selected by this icon: ), centred text, and with the name of the field set from the PDF Options →Field Properties context menu. Each field needs a different name. I used callsign, date, utc, mhz, mode, and report.
Save your QSL card as PDF. It might be an idea to check it to see if the form fields are really there and editable:
Now it starts to get really nerdy. Adobe specified the Forms Data Format (FDF) to allow PDF form data to be slung around. FDF looks a bit like PostScript or raw PDF:
If you think of the T & V values above as Tag and Value, you can see that the file defines callsign=VE3KPK, date=2011-10-02, mode=CW, and so on. This step can be easily scripted. If you’re not sure what the fields are called, pdftk has a “dump_data_fields” option that spits out the field names as plain text.
pdftk is also used to put the data from the FDF file into the PDF template. It’s a slightly hairy command line:
Here the source file is QSL-blank.pdf, the FDF data is VE3KPK.fdf, and the output is to QSL-VE3KPK.pdf. The flatten option turns the pdf form into regular, uneditable PDF. (These details are from a real QSO, by the way, and by utter coincidence I’m writing this in a hotel in Ken VE3KPK’s home town in Northern Ontario.)
Checking that this worked in Adobe Reader:
For many people this is probably enough (or perhaps, too much already!) but I really want to have a digital QSL card that will stand up to some scrutiny. This is where your LoTW certificate file comes in.
PortableSigner is a java application for signing PDF files. It seems quite happy signing the files made in this workflow. It can run from the command line, or as a windowed application:
You use your .p12 signature file and its password to sign the PDF. Once the file is signed, you can send it to your contact, and they can prove (and hopefully, any certification/contest agency will agree) that the contact was confirmed.
Viewing the signed document is deeply unimpressive:
It’s not such a big deal that Reader says that “the validity of the certification is unknown”. It’s just that Adobe doesn’t have the ARRL’s certificate loaded into everyone’s reader (what!? mock outrage!) and so it doesn’t match a certificate it knows about. You have to dig a bit deeper into the signature panel to check out who is responsible for this.
Well, that’s a start; at least it was signed by someone with my e-mail address. There’s more under Show Signature Properties:
There’s the ARRL signature in there. Buried deep in another properties tree is my callsign; can’t find it today, but I saw it in there last night. Either way, the digital QSL PDF is now signed and certified that it came from me, as an ARRL LoTW user.
**** This file had errors that were repaired or ignored.
**** The file was produced by:
**** >>>> Adobe PDF Library 7.0 <<<<
**** Please notify the author of the software that produced this
**** file that it does not conform to Adobe's published PDF
**** specification.
I’d totally put these up on my walls, but I may end up climbing them:
I made them by taking 32×32 pixel tiles of random grey noise, Atkinson dithering them (using pamditherbw) then vectorizing them using potrace. If you click on the tiles, you can download/view the PDF source of each.
(pgmnoise, the source of the grey noise, relies heavily on the system time as its seed. Before I introduced a delay between image generation, several images appeared almost identical.)
I’ve been trying to learn banjo chords for a while, and the books I have keep flopping closed. So I resolved to make a blank chord form that I could fill in, like this:
You might wonder why it goes to the 7th fret. If you’re in Double C tuning, you’ll need that if you’re drawing a tuning chart.
So for G tuning, the F chord would look like:
There are 12 fretboard images to a page – that’s enough for four whole folk songs!
I got a bit carried away with doing this. Instead of just smacking together all the 360 dpi TIFFs I scanned seven years ago, I had to scan a new set at a higher resolution, then crop them, then fix the page numbers, add chapter marks, and make the table of contents a set of live links.
I’ve got out of the way of thinking in PostScript, so I spent some time looking for tools that would do things graphically. Bah! These things’d cost a fortune, so armed only with netpbm, libtiff, ghostscript, the pdfmark reference, Aquamacs, awk to add content based on the DSC, and gimp to work out the link zones on the contents page, I made it all go. Even I’m impressed.
One thing that didn’t impress me, though:
I used to edit multi-gigabyte files with emacs on Suns. They never used to complain like this. They just loaded (admittedly fairly slowly) and let me do my thing. Real emacs don’t give warning messages.
Lots of people are drooling over the book Rule the Web. I’m not, particularly. It’s good in parts, but reminds me so much of those mid-late 1990s “Best Web Directory Ever” tomes that are currently propping up shelves in bargain bookstore, and propping up houses built on landfills in Arizona.
My biggest complaint is its US-centric approach. Pretty much everything related to buying, selling or finding people or things mentioned in the book only applies to the USA.
As is the way when web meets paper, some things are out of date already. It happens, but it’s a shame when the book’s pretty new in the shops.
I did find a couple of things I genuinely didn’t know about, but might find useful:
Combine PDFs, for slicing and dicing PDFs under OS X. (I could do this with pdftk, but Combine PDFs is purty).
The Freesound Project is a collaborative database of Creative Commons licensed sounds. When I next need a comic boing, I’ll know where to look.
It also gave links to OnyX and HandBrake, both of which I already use. But that’s about it. I’d have been peeved if I bought the book (yay, Toronto Public Library!), as this is more of a basic manual than a compendium of coolness.
I’ve always thought that Adobe missed a great opportunity when they didn’t make their basic PDF writer freely available for Windows. Other OSs now have transparent print-to-PDF options. If you’re lucky, a corporate PC might have MS Office Document Image Writer installed, but a 300dpi monochrome TIFF can’t compare to a PDF.
Still, one can always install PDFCreator (if you have admin rights to the PC, of course). It’s a shame they decide to bundle a marginally dodgy toolbar/spyware package with it, but you don’t get that if you use the MSI installer package.
So, what would you think would be “The clean air choice of Earth Day Canada“? A bicycle, perhaps? Some kind of renewable energy? Some really brilliant Canadian enviro-social development, like a biodegradeable donut?
Nope, a car; the Toyota Prius. Last time I checked, it still used petroleum (with its high environmental and geopolitical toxicity). It still causes gridlock; I see Priuses (Prii? Your moon-pie eye!) inching along the Gardiner from the GO train with all the other wretched junkers. The way I see it, it’s not looking like part of the solution. It’s a bit like having an official assault rifle for the the International Day of Peace.
Toyota also give out $5000 Toyota Earth Day Scholarships. I mean, that’s nice and all, but it’s hardly giving back. If you look at all the scholarship materials, it’s carefully arranged so it looks like the event is called Toyota Earth Day, with the ‘scholarship’ on the next line. Nice cooption. Good greenwash.
The coffee place I get my morning fix got rid of its unreliable Dexit machine when it changed hands. So I’ve got nearly $70 sitting on the useless Dexit tag, doing nothing.
Only in the event of your death (and upon receipt of such documents as Dexit reasonably requires in such circumstances as to whom is entitled to your estate funds), or Dexit closing your Dexit Account without cause, will the remaining funds in your Dexit Account be repaid to you. “Cause” will include any violation of this Agreement, any fraud or attempted fraud, any other operation of the Dexit Account or use of a Dexit Tag in an unsatisfactory manner, or non-use of your Dexit Account for over three (3) years.
So, do I hafta kill myself to get my money back?
Oh, and Dexit’s phone support staff are untruthful. I needed to speak to a supervisor. They promised one would call before 8pm this evening. It’s 9:55 now, and I’ve heard nothing.
but at least the text can be copied and indexed.
It’s a perfect 
