Category Archives: computers suck

JPEG 2000 on Ubuntu — without anyone getting stabbed

JPEG 2000 is a great image file format: well-defined, and able to store high quality photographic data in a very small space. It truly is the JPEG of the 2000s — except for its dismal support under Ubuntu.

The problem is the patents. An open library has been a long time coming, and lots of Linux software is built without JP2 support. This helped keep it away from my desktop.

Under Ubuntu 14.04, here’s what does and doesn’t support JP2 files:

  • Gimp — not supported. It appears to have a non-functioning plugin that tries to read the file, then gives up. This is annoying, as Gimp is defined as the system default viewer for JPEG 2000.
  • Image Viewer — does support JP2, but occasionally mis-renders pages. To make this the default, right-click on a JP2 file, and select Open with → Other Application …, then choose Image Viewer. It should work from then onwards.
  • Document Viewer — a bit rough when looking at JPEG 2000-encoded PDFs. Very slow, too.
  • GraphicsMagick — seems to be the most painless way of converting graphics files to JPEG 2000. My preferred method of invoking it is:
    gm convert -define 'jp2:rate=0.008' in.png out.jp2
    The rate option should be a small number; the smaller, the greater the compression, and the worse the image quality.
  • OpenJPEG — provides the image_to_j2k and j2k_to_image tools. Far more picky about input formats than it should be, and often fails on seemingly perfect input.
  • img2pdf — (built from source)  is a tiny gem of a package. All it does is wrap various image formats into a PDF file. It doesn’t modify the image data in any way, so with a bit of ingenuity (and pdftk) you can use PDF as a true metafile archive. You can view the content on any platform, but get the source images out bit-for-bit perfect. We used to call files which could contain files metafiles, but that stopped being popular when TIFF started to be a baroque travesty of an image container back in the mid-1990s.
  • poppler — (for full features, build from source) has a tool, pdfimages, which can extract embedded image files from PDFs. Some of the metadata might get lost, but all of the image bits come through.

Since JPEG 2000 isn’t included in web browsers (grar), I’ve embedded a sample scanned JPEG into a PDF, and added a series of progressively more compressed JPEG 2000 versions: JPEG-2000Booklet [PDF].  The booklet has notes showing the byte size of each page. The image still looks pretty good at 8% of the original file size!

The cuddliest microfiche reader of 1968

wooster-cuddly_microfiche_reader

Remind you of anything?

(from Wooster, Harold. Towards a Uniform Federal Report Numbering System and a Cuddly Microfiche Reader, Two Modest Proposals, 1968, cited in Stevens, Mary Elizabeth. “Research and Development in the Computer and Information Sciences. Volume 2, Processing, Storage, and Output Requirements in Information Processing Systems: A Selective Literature Review.” (1970).)

Ⓗⓞⓦ ⓣⓞ ⓑⓔ ⓐⓝⓝⓞⓨⓘⓝⓖ ⓦⓘⓣⓗ Ⓟⓔⓡⓛ ⓐⓝⓓ Ⓤⓝⓘⓒⓞⓓⓔ

It’s been so long since I’ve programmed in Perl. Twelve years ago, it was my life, but what with the Raspberry Pi intervening, I hadn’t used it in a while. It’s been so long, in fact, that I wasn’t aware of the new language structures available since version 5.14. Perl’s Unicode support has got a lot more robust, and I’m sick of Python’s whining about codecs when processing anything other than ASCII anyway. So I thought I’d combine re-learning some modern Perl with some childish amusement.

What I came up with was a routine to convert ASCII alphanumerics ([0-9A-Za-z]) to Unicode Enclosed Alphanumerics ([⓪-⑨Ⓐ-Ⓩⓐ-ⓩ]) for advanced lulz purposes. Ⓘ ⓣⓗⓘⓝⓚ ⓘⓣ ⓦⓞⓡⓚⓢ ⓡⓐⓣⓗⓔⓡ ⓦⓔⓛⓛ:

#!/usr/bin/perl
# annoying.pl - ⓑⓔ ⓐⓝⓝⓞⓨⓘⓝⓖ ⓦⓘⓣⓗ ⓤⓝⓘⓒⓞⓓⓔ
# created by scruss on 2014-05-18

use v5.14;
# fun UTF8 tricks from http://stackoverflow.com/questions/6162484/
use strict;
use utf8;
use warnings;
use charnames qw( :full :short );
sub annoyify;

die "usage: $0 ", annoyify('string to print like this'), "\n" if ( $#ARGV < 0 );
say annoyify( join( ' ', @ARGV ) );
exit;

# 💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩

sub annoyify() {
    # convert ascii to chars in circles
    my $str = shift;
    my @out;
    foreach ( split( '', $str ) ) {
        my $c = ord($_);             # remember, can be > 127 for UTF8
        if ( $c == charnames::vianame("DIGIT ZERO") )
	{
            # 💩💩💩 sigh; this one's real special ... 💩💩💩
            $c = charnames::vianame("CIRCLED DIGIT ZERO");
        }
        elsif ($c >= charnames::vianame("DIGIT ONE")
            && $c <= charnames::vianame("DIGIT NINE") )
        {
            # numerals, 1-9 only (grr)
            $c =
              charnames::vianame("CIRCLED DIGIT ONE") +
              $c -
              charnames::vianame("DIGIT ONE");
        }
        elsif ($c >= charnames::vianame("LATIN CAPITAL LETTER A")
            && $c <= charnames::vianame("LATIN CAPITAL LETTER Z") )
        {
            # upper case
            $c =
              charnames::vianame("CIRCLED LATIN CAPITAL LETTER A") +
              $c -
              charnames::vianame("LATIN CAPITAL LETTER A");
        }
        elsif ($c >= charnames::vianame("LATIN SMALL LETTER A")
            && $c <= charnames::vianame("LATIN SMALL LETTER Z") )
        {
            # lower case
            $c =
              charnames::vianame("CIRCLED LATIN SMALL LETTER A") +
              $c -
              charnames::vianame("LATIN SMALL LETTER A");
        }
        else {
            # pass thru non-ascii chars
        }
        push @out, chr($c);
    }
    return join( '', @out );
}

Yes, I really did have to do that special case for ⓪; ⓪…⑨ are not contiguous like ASCII 0…9. ⓑⓞⓞ!

Precisely what nobody wanted — the Hershey fonts as a huge great PDF

HersheyNumberedOccidentalGrid-section

It’s impractically huge, but under the image link lives a table of all of the Hershey fonts (well, the Western ones, at least). It’s interesting to note Dr Hershey’s preferences in this pre-ASCII table: almost every variant has degree, minute and second symbols, but none of them have ‘\’. Many of them don’t have ‘@’, either, so no e-mail addresses in Hershey Fraktur for you …

Forgive me, A V Hershey ...

Forgive me, A V Hershey …

I’ve been playing with the Hershey Fonts, vector data digitised by the US government in 1967. It’s in a bit of a bear of a format.

I hope to do more fun stuff with the data. For now, here’s an 120 page sample book showing all the characters:

hershey_sample_page

Making cheap HP plotter pens + yet another HP-GL viewer

If you’re running an old plotter, getting pens can be a worry. While there are some companies that might still make them (Graphic Controls/DIA-Nielsen, for one) they are expensive and limited in range. They’re also felt-tip, which means they’ll dry out if not carefully re-capped.

While eBay might supply all things (like these Roland DG plotter pens I scored a couple of days back; fine, black, new old-stock, or these German plotter pens), I also found this:

$_3It’s described as “11.5*28MM cutting plotter vinyl cutter pen holder 50mm for Roland holder Pcut”. I bought two, and eventually the slow boat from China came …

pens

The one on the left is an unmodified pen holder. Well, it’s really a ballpoint-refill holder, as it comes with a (random colour of blue) refill.  To modify these to fit into an HP desktop plotter, you will need to:

  1. Cut ~10 mm from the end of the holder. A Dremel + cutting disk is a satisfying way of doing this. The gap between the knurled bit and  the thread seems to be a decent place. Clean up the sharp edges
  2. As the knurled lock ring will stop the pen engaging in the carriage (my HP-7470A does a lovely little hesitant try… nope; try… nope; try… give up sequence), you’ll have to do without it. Find another way of jamming the threads of the threaded collar in the right place. I used electrical tape, and it’s held so far. Wiser users will use different colours of electrical tape for different pens, ahem …
  3. Stick the pen refill in and tighten down the collet lightly with pliers. ¡¡¡ Do not try to pull the refill out while it is in the collet !!! (The ballpoint insert will likely pop out, and viscous ink will start to blort out everywhere. Ask me how I know!)
  4. Snip the end of the refill flush with the end of the pen holder using diagonal cutters. Best to do this directly over a rubbish bin, as pen ink is nasty. Dab off excess ink from the end of the refill, and clean your cutters, too.
  5. The base of the threaded collar should be around 29 mm from the pen tip, otherwise nothing will plot (if it’s too short) or you’ll poke holes in the paper (if too long). This measurement doesn’t seem to be extremely critical: my Roland pens have it at 28.5 mm, the DIA-Nielsen pens are 28.9 mm. One of my homebrew pens is working at 30½ mm, but then, my basic plotter has no force control, so it may be more forgiving than more elegant beasts.

modified holders and cut/not-so-cut refills

My modified pens look like the above.

The dollar store is a good source of cheap ballpoint pens. I managed to snag 8 retractable red pens for $1.25, and 4 black pens for $1.

one of the donor red pens, plus the disassembled pen holder

(These retractable pens more often than not eject the whole internals across the room when you retract ‘em.)

It’s probably a good idea to scribble with the pens a bit before and after modifying them, as they take a while to flow freely. They plot very lightly; the black ink looks more like a faint pencil line.

Double-plotted nested bézier curves
Double-plotted nested bézier curves

If you look close up, not merely are the lines very faint, but something else important shows up:

double-plotted detail, showing off-centre effects (actual size 17 × 17 mm)
double-plotted detail, showing off-centre effects (actual size 17 × 17 mm)

The lines — which should be a constant(ish) distance apart, if the paper has stayed in registration — are showing a varying distance from each other. It looks like the pen points are a little off-centre, so when the pen is swapped out, it gets turned to a slightly different position. This would really only matter for precise work, and I find the effect interesting.

As for the HP-GL viewer? GhostPDL, by the makers of Ghostscript. You’ll have to build it from source, and its documentation isn’t quite where one might want it to be, but it implements a full HP PCL6  / HP-GL/2 interpreter than can output bitmaps, PostScript or PDF. The SVG graphic below was made using the tools/plot2pdf.sh script to convert HP-GL to PDF, then I used ghostscript to convert that to SVG. Nifty!

weave.pltAs a bonus, GhostPDL comes with one of the prettiest plotter fonts ever:
testlb

More fun with plotters: graphing, simplifying, and random noises …

  1. If you go to the Incompetech Multi-Color Graph Paper PDF Generator and set the grid spacing to 0.4, 0.8 and 4 lines/cm, you get a nifty HP-GL unit grid with major lines at 1000 units (25 mm), intermediate lines at 500 units (12½ mm) and minor lines at 100 units (2½ mm):
    hpgl-grid-web
  2. By a very roundabout route, I used GDAL’s simplify function to half the line count of my hypotrochoid plots with little visible degradation. The optimised page took far less time to plot:
    hypo1
    hypo2
    Next time, I’d definitely use Shapely instead of messing about with GIS formats. Who knows, I might even integrate it into Chiplotle. Eventually. (Update: though I have some code that does something …)
  3. Plotters are resolutely electromechanical devices. They turn digital command into analogue output, but my one never sounds too happy doing it. It grumbles, sighs and even moos as it plots. A recent plot sounded a bit like this: .

Pleasantly futile: Dogecoin digging on the Raspberry Pi

I don’t recommend this in any way, but cpuminer will run on the Raspberry Pi. It’s pretty easy to build:

sudo apt-get install build-essential libcurl4-openssl-dev automake git
git clone https://github.com/pooler/cpuminer.git
cd cpuminer/
./autogen.sh
./configure CFLAGS="-march=armv6 -mtune=arm1176jzf-s -mfloat-abi=hard -mfpu=vfp -ffast-math -ffast-math -O3"
make
sudo make install

I’m not convinced that the l33t funroll-l00pz CFLAGS are strictly necessary. Yes, you still only get 0.34 khash/s on a stock Raspberry Pi — which means it would take several days to earn 1Đ (or roughly 0.1¢).

I’m trying to find the most futile computer on which to dig Đ. My BeagleBone Black is (somewhat surprisingly) more than 2× as fast, once you replace the very limited Ångström distribution with Debian. I’m really disappointed that I can’t build cpuminer on my Intel Galileo. Its Yocto distribution is extremely small yet confusing. As the board runs burny hot under no load, I wonder how quickly it would glow white-hot under 100% CPU load.

For all your HP 7470a plotter manual needs

On the off chance you need to control a 30 year old graphics plotter, have I got something for you:

HP-7470A_Graphics_Plotter-Interfacing_and_Programming_Manual

The image links to a scanned copy of the HP 7470A Graphics Plotter: Interfacing and Programming Manual which I found on the web, and cleaned up. The pages have been OCR’d, so it should be searchable.

So that's how the Raspberry Pi camera fits in the CamdenBoss housing ...

So that’s how the Raspberry Pi camera fits in the CamdenBoss housing …

 

The CamdenBoss Raspberry Pi Camera Enclosure (data sheet, suppliers: RPI CAMERA BOARD – RASPBERRY-PI – ADD-ON BRD, CAMERA MODULE, RASPBERRY PI | Newark element14 Canada, MCM Electronics Carbon Raspberry Pi Camera Case 83-15493 – Micro Center) comes with no instructions. Maybe the lighter coloured ones are easier to work out, but on the faux-carbon one I bought, the little guide slots the board has to slide into are very hard to see.

Given all the warnings about static, I was a little too careful trying to install the camera into the housing. Slip open the camera case, then put the board in at an angle with one side in one slot, then (with a bit more force than I’d like) spring or flex the housing so the other side of the board can click into place. You have to make sure that both sides are fully engaged in the slots before the cover will slide back on.

So here it is, all set up:

raspberry_pi_and_camera_agh_my_retinas

Oh, sorry;  should’ve warned you about the bright pink case and the awesome/appalling Lisa Frank sticker. The sticker is in no way to cover up where I cut the wrong place for the camera connector, nope nope nope …

Wonderful blank Terms & Conditions

I’d just like to go on the record that Intel made me accept these Terms & Conditions when logging on their Space: Makers | Intel Communities site:intel-tcs-2014-02-08-15:43:04Yup, completely blank. I guess I can do anything I want now, eh?

elementary OS: could be worse

Imagine there’s a really nicely arranged screenshot of elementary OS here. You know, browser arranged just so, dock showing shiny icons, and a coy little dropdown showing that I’m playing music that’s faaaar hipper than you’ll ever be. Got that image? Good. ‘Cos I just spent a ½-hour setting it up, then deleted it in a second of unthought when I cleaned up the elementary OS VM from VirtualBox. Derp!

elementary OS is a very pretty Ubuntu/Debian distro. It has a very strong visual identity, and is designed and managed by a very small group. This rigidity may annoy the seasoned Linux user,  but will likely work in a more logical way if you’re used other OSs. You won’t face jarringly mismatched user interface elements, as still happens with Ubunty/Unity. Linux UX allows too much choice, so you’re never sure which options do what at any given time.
(F’rinstance: Ctrl+Q used to quit programs. Now, Ubuntu wants us to use Ctrl+W, the old close-the-window command. Some programs no longer quit with Ctrl+Q, so you’re left with an awksmash of Ctrl+Q_uh-I-meant_W. Don’t make me think!)

A couple of things put me off elementary OS:

  1. You can’t put files on the desktop. In an effort to be tidy, eOS forbids you putting the stuff you’re working on in the place you’ll see it. This is a major annoyance, but worse things are coming.
  2. It expects you to pay for it. No, really. They want $10 from you, right from the get-go. While you can get around this (click the “Download” button; it only looks like it’s linked to the payment button), they do something far, far worse:
    They use a PayPal pop-under.
    Srsly. Gahh. Eww. Pop-unders are the web equivalent of taking a 💩 on an art gallery floor. If they’re low enough to pull that kind of stunt, who knows what leakage lurks under their pretty graphics?

Linking ≠ Abuse.

Got this e-mail today.  Addresses redacted, but otherwise verbatim:


Subject: Link removal request
From: Rodger Lodge <entrust_____@gmail.com>
Date: 14-01-27 08:46 AM
To: abuse@my hosting company.com, me, hostmaster@my hosting company.com

Hi,

We have recently received a notification from Google stating that our website has unnatural links pointing towards it. This has really damaged our rankings on Google and as a result, we’re trying to clear things up.
Our website url is www.entrust.net.

We noticed the following links are pointing to our website from your site:
http://scruss.com/blog/2006/12/
http://scruss.com/blog/2006/12/14/when-you-really-havent-chosen-not-to-trust-citrix-mac-os-x-and-entrust-certificates/
http://scruss.com/blog/tag/citrix/

I appreciate this is inconvenient and isn’t a reflection on your website at all, but if you’re able to remove the links, we would really appreciate it and would be very grateful.

I look forward to hearing from you.

Rodger Lodge

Entrust, Inc.


So, one of the prime security companies (I’m strongly resisting the urge to link to them here, but I’m above that) is using gmail to send out official notices? Why not a securely signed e-mail from their .net domain? That might be more legit. Also, sending me a copypasta response lifted  from the “Ultimate Guide to Google Penalty Removal” seems a little … gauche.

If this is for real, then I’m annoyed. I wrote the original article in 2006 in an attempt to help Mac users get work done with a popular remote-access package. I’m not dissing the company in any way: I’m helping people use their products better. Linking is not abuse.

It’s probably just spam, though.

Update: Nope, not spam. Got an apology by e-mail from Nate Plaunt of Online Performance Marketing, saying: “This was an automated email generated by this digital agency and your sites were mistakenly targeted”.  Still no apology for the e-mail to abuse@…, tho’. That’s never an acceptable first communication.

The HP48: the best calculator ever

We had an unscheduled overnight stop in East Lansing last week, and I took the chance to visit the MSU Surplus Store.  For $15, they had HP48G calculators, seemingly unused:

hp48gThey still have a bunch of them: HP 48G Graphic Calculator.

They’re maybe not the quickest (the 4 MHz Saturn processor chugs sometimes, and wanders off to clean up memory when it feels like it), the display is downright chunky these days, but they have everything that a scientific calculator should have. The keys have a good action. It just works. Yes, your smartphone has hundreds of times the processing power, but it’s not specialized to the task. The HP48 is.

If you’re feeling really nerdy, you can run an HP48 (a GX, not the G which I have) under Linux using x48. Jeff has some useful tips on installing x48 on newer ubuntu versions (though you don’t have to do the font thing under Ubuntu 13.10).

x48Building it is a small matter of ./autogen.sh ; ./configure ; make ; sudo make install.  To run it, you’ll need to install the GX ROM image to ~/.hp48.  The first time you run it, I’d recommend running it from the terminal with:

x48 -reset -initialize -rom ~/.hp48/rom

as the ROM format has an outdated config file which causes it to complain weakly every time you start the emulator.

Scanned manuals are available from HP here: Manuals for HP 48gx Graphing Calculator.

Processing 2.1 + Oracle Java + Raspberry Pi + Serial + Arduino = ☺

With Sun Oracle hardfloat Java now available, Processing now runs at a decent clip on the Raspberry Pi. My old instructions are now very obsolete. Here are current, tested instructions for installing it under Raspbian.

[This is a particular solution to installing a Serial/Firmata-enabled Processing 2.1 distribution on a Raspberry Pi. Processing still has issues with other aspects of visual programming (particularly video) that I'm not addressing here.]

A lot of software is installed here, and much of it depends on previous steps. Don’t jump in mid-way and expect it to work.

Update the system

Always a good plan if you’re doing major upgrades:

sudo apt-get update
sudo apt-get dist-upgrade

Install Sun Oracle Java

sudo apt-get install oracle-java7-jdk

Check if the right version is installed as default: java -version should give

java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) Client VM (build 24.0-b56, mixed mode)

If you get anything else, you need to make Sun Oracle’s version the default:

sudo update-alternatives --config java

Download & Install Processing

Go to Download \ Processing.org and get the Linux 32-bit version.  It’s big; about 100 MB. I’m going to install it in my home directory, so the base path will be ~/processing-2.1. Extract it:

tar xvzf processing-2.1-linux32.tgz

Now you have to remove the included x86 Java runtime, and replace it with the Raspberry Pi’s armhf one:

rm -rf ~/processing-2.1/java 
ln -s /usr/lib/jvm/jdk-7-oracle-armhf ~/processing-2.1/java

You should now have a Processing installation that will run, but there’s some more we need to get serial and Arduino support.

Install the  java Simple Serial connector

Download jSSC-2.6.0-Release.zip and extract it:

unzip jSSC-2.6.0-Release.zip

Now overwrite the jssc.jar that ships with Processing with the one you just downloaded:

mv jSSC-2.6.0-Release/jssc.jar ~/processing-2.1/modes/java/libraries/serial/library/

(You can remove the jSSC folder now: rm -r jSSC-2.6.0-Release)

Test Processing’s serial support

You’re almost there! Fire up Processing:

~/processing-2.1/processing

and try Perhaps the World’s Most Boring Processing Sketch:

// Example by Tom Igoe

import processing.serial.*;

// The serial port
Serial myPort;

// List all the available serial ports
println(Serial.list());

Screenshot from 2014-01-07 20:08:32When this runs (it’s a little slow), you should get a single line of output, which should start /dev/tty___:

/dev/ttyACM0

(I have an Arduino Leonardo attached, which usually appears as an ACM device.)

Installing Arduino/Firmata support

(I’m not going to go into uploading Firmata onto your Arduino here. All I can recommend is that you use the newest version at firmata/arduino, rather than the old code bundled with your Arduino distribution.)

Exit Processing, and download processing-arduino.zip from firmata/processing. Extract it into your Processing sketchbook:

unzip processing-arduino.zip -d ~/sketchbook/libraries/

For tedious reasons, you also have to rename one of the files:

mv  ~/sketchbook/libraries/arduino/library/Arduino.jar  ~/sketchbook/libraries/arduino/library/arduino.jar

Start up Processing again, and  save Most Probably the World’s Second Least Interesting Processing Program™:

import processing.serial.*;
import cc.arduino.*;
Arduino arduino;
int ledPin = 13;

void setup()
{
  println(Arduino.list());
  arduino = new Arduino(this, Arduino.list()[0], 57600);
  arduino.pinMode(ledPin, Arduino.OUTPUT);
}

void draw()
{
  arduino.digitalWrite(ledPin, Arduino.HIGH);
  delay(1000);
  arduino.digitalWrite(ledPin, Arduino.LOW);
  delay(1000);
}

Screenshot from 2014-01-07 21:13:54
What this sketch does is emulate the µC’s “Hello World” program, Blink. It flashes the board’s LED once per second. Boring? Yes. But if it worked, you have a working Processing 2.1 installation on your Raspberry Pi. Go forth and make more interesting things.
(Props to bitcraftlab/wolfing for the basic outline for installing Processing, and for samaygoenka for the prodding needed to update and test the Processing installation process. If you’re still stuck, the Processing 2.0 Forum and the Raspberry Pi Forum are good places to ask.)

Introducing RAFTP — the Really Annoying File Transfer Protocol

I would like to describe a new and highly impractical method of transferring data between computers. Modern networks are getting more efficient every year. This protocol aims to reverse this trend, as RAFTP features:

  1. Slow file transfers
  2. A stubborn lack of error correction
  3. The ability to irritate neighbours while ensuring inaccurate transmission through playing the data over the air using Bell 202 tones.
doge-small-tx
Figure 1

Figure 1 shows a test image before it was converted into PGM format. This was then converted into an audio file using minimodem:

minimodem --tx -v 0.90 -f doge-small-1200.wav 1200 < doge-small-tx.pgm

This file was then transferred to an audio player. To ensure maximal palaver, the audio player was connected to a computer via a USB audio interface and a long, minimally-shielded audio cable. The output was captured as mp3 by Audacity as this file: RAFTP-demo

The mp3 file was then decoded back to an image:

madplay -o wav:- RAFTP-demo.mp3   | minimodem --rx -q -f - 1200 | rawtopgm 90 120 | pnmtopng > doge-small-rx.png

Figure 2 shows the received and decoded file:

Figure 2
Figure 2

My Raspberry Pi talks to my Oscilloscope

… it complains that the oscilloscope is always making waves.

DS1EB134907266_0

Ahem. Anyway. I have a Rigol DS1102E 100 MHz Digital Oscilloscope. For such a cheap device, it’s remarkable that you can control it using USB Test & Measurement Class commands. I’d been wanting to use a Raspberry Pi as a headless data acquisition box with the oscilloscope for a while, but Raspbian doesn’t ship with the usbtmc kernel module. I thought I was stuck.

Alex Forencich turned up in the forum with an all-Python solution: Python USBTMC (source: alexforencich / python-usbtmc). I got this working quite nicely today on both the Raspberry Pi and my Ubuntu laptop. Here’s how I installed it:

  1. Check your device’s USB code with lsusb:
    $ lsusb
    Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
    ….
    Bus 001 Device 004: ID 1ab1:0588 Rigol Technologies DS1000 SERIES
  2. Ensure that libusb-1.0 is installed:
    sudo apt-get install libusb-1.0-0
  3. Create a new group, usbtmc:
    sudo groupadd usbtmc
  4. Add yourself to this group:
    sudo usermod -a -G usbtmc pi
  5. As root, create a file /etc/udev/rules.d/usbtmc.rules. You’ll need to put in your device’s ID values:
    # USBTMC instruments
    # Rigol DS1100 – ID 1ab1:0588 Rigol Technologies DS1000 SERIES
    SUBSYSTEMS==”usb”, ACTION==”add”, ATTRS{idVendor}==”1ab1″, ATTRS{idProduct}==”0588″, GROUP=”usbtmc”, MODE=”0660″
    (all of the SUBSYSTEMS to MODE= should be one one line)
  6. Download and install the latest pyusb (Raspbian version is rather old):
    git clone https://github.com/walac/pyusb.git
    cd pyusb
    python setup.py build
    sudo python setup.py install
  7. Now get python-usbtmc:
    git clone https://github.com/alexforencich/python-usbtmc.git
    cd python-usbtmc
    python setup.py build
    sudo python setup.py install
  8. For this simple demo, you’ll need to convert the USB vendor IDs to decimal:
    0x1ab1 = 6833
    0×0588 = 1416
  9. Now, start python as root (sudo python) then type:
    import usbtmc
    instr =  usbtmc.Instrument(6833, 1416)
    print(instr.ask(“*IDN?”))
  10. This should return something like:
    Rigol Technologies,DS1102E,DS1EB13490xxxx,00.02.06.00.01

If you get the status line, congratulations! You now have a fully working usbtmc link. I haven’t had much time to play with this, but I know I can make really nice screenshots to an attached USB drive using the command: instr.write(“:HARDcopy”). Many more commands can be found in the DS1000D/E Programming Guide, available on Rigol‘s site.

I had a couple of problems, though:

  1. The library seems to need root privileges, despite the udev rule thing. After creating the udev rule, you will need to reboot. This is the simplest way of getting it to work without being root.
  2. Reading from the ‘scope’s memory  chokes on non-UTF8 characters. If I do:
    rawdata = instr.ask(“:WAV:DATA? CHAN1″)[10:]
    I get a lengthy Python error which ends:

    File “/usr/lib/python2.7/encodings/utf_8.py”, line 16, in decode
        return codecs.utf_8_decode(input, errors, True)
    UnicodeDecodeError: ‘utf8′ codec can’t decode byte 0×99 in position 10: invalid start byte
    I have no idea what that means, or how to fix it. Alex suggested using ask_raw instead of ask, and the data comes through with no complaints.

I’ve still got to work my way through the Rigol’s data format, but other people have done that before:

  1. Controlling a Rigol oscilloscope using Linux and Python | C i b o M a h t o . c o m
  2. Ken Shirriff’s blog: Four Rigol oscilloscope hacks with Python

I’ll post any updates here, along with the Raspberry Pi forum topic: USB Test & Measurement class (usbtmc) driver?

Incidentally, if you’re working with WFM data dumps from the Rigol ‘scopes (and you should, because they make storing data to USB drives quick), mabl/pyRigolWFM is basically magic. Not merely can it describe and decode those binary files, it can do pretty graphics with no thought required:

made by pyRigolWFMHat tip for the mention: MP3 Options & Oscilloscope Interfacing For Raspberry Pi @Raspberry_Pi #piday #raspberrypi « adafruit industries blog

Update, 2013-12-20: I’ve successfully managed to run most of Ken’s examples with Alex’s code. The major modification you have to do is use ask_raw instead of ask. Example code shown below:

#!/usr/bin/python
# -*- coding: utf-8 -*-

"""
Download data from a Rigol DS1102E oscilloscope and graph with matplotlib
         using  Alex Forencich's python-usbtmc pure python driver

https://github.com/alexforencich/python-usbtmc

scruss - 2013-12-20

based on
Download data from a Rigol DS1052E oscilloscope and graph with matplotlib.
By Ken Shirriff, http://righto.com/rigol

which in turn was
Based on http://www.cibomahto.com/2010/04/controlling-a-rigol-oscilloscope-using-linux-and-python/
by Cibo Mahto.
"""

import usbtmc
import time
import numpy
import matplotlib.pyplot as plot

# initialise device
instr =  usbtmc.Instrument(0x1ab1, 0x0588) # Rigol DS1102E

# read data
instr.write(":STOP")
instr.write(":WAV:POIN:MODE RAW")
# first ten bytes are header, so skip
rawdata = instr.ask_raw(":WAV:DATA? CHAN1")[10:]
data_size = len(rawdata)

# get metadata
sample_rate = float(instr.ask_raw(':ACQ:SAMP?'))
timescale = float(instr.ask_raw(":TIM:SCAL?"))
timeoffset = float(instr.ask_raw(":TIM:OFFS?"))
voltscale = float(instr.ask_raw(':CHAN1:SCAL?'))
voltoffset = float(instr.ask_raw(":CHAN1:OFFS?"))

# show metadata
print "Data size:      ", data_size
print "Sample rate:    ", sample_rate
print "Time scale:     ", timescale
print "Time offset:    ", timeoffset
print "Voltage offset: ", voltoffset
print "Voltage scale:  ", voltscale

# convert data from (inverted) bytes to an array of scaled floats
# this magic from Matthew Mets
data = numpy.frombuffer(rawdata, 'B')
data = data * -1 + 255
data = (data - 130.0 - voltoffset/voltscale*25) / 25 * voltscale

# creat array of matching timestamps
time = numpy.linspace(timeoffset - 6 * timescale, timeoffset + 6 * timescale,
                      num=len(data))

# scale time series and label accordingly
if (time[-1] < 1e-3):
    time = time * 1e6
    tUnit = "µS"
elif (time[-1] < 1):
    time = time * 1e3
    tUnit = "mS"
else:
    tUnit = "S"

# Plot the data
plot.plot(time, data)
plot.title("Oscilloscope Channel 1")
plot.ylabel("Voltage (V)")
plot.xlabel("Time (" + tUnit + ")")
plot.xlim(time[0], time[-1])
plot.show()