bench64: a new BASIC benchmark index for 8-bit computers

Nobody asked for this. Nobody needs this. But here we are …

commodore 64 screen shot showing benchmark results:
basic bench index
>i good. ntsc c64=100

1/8 - for:
 60 s; 674.5 /s; i= 100
2/8 - goto:
 60 s; 442.3 /s; i= 100
3/8 - gosub:
 60 s; 350.8 /s; i= 100
4/8 - if:
 60 s; 242.9 /s; i= 100
5/8 - fn:
 60 s; 60.7 /s; i= 100
6/8 - maths:
 60 s; 6.4 /s; i= 100
7/8 - string:
 60 s; 82.2 /s; i= 100
8/8 - array:
 60 s; 27.9 /s; i= 100

overall index= 100


ready.
bench64 running on the reference system, an NTSC Commodore 64c

Inspired by J. G. Harston’s clever but domain-specific ClockSp benchmark, I set out to write a BASIC benchmark suite that was:

  1. more portable;
  2. based on a benchmark system that more people might own;
  3. and a bunch of other less important ideas.

Since I already had a Commodore 64, and seemingly several million other people did too, it seemed like a fair choice to use as the reference system. But the details, so many details …

basic bench index
>i good. ntsc c64=100

1/8 - for:
 309.5 s; 130.8 /s; i= 19 
2/8 - goto:
 367.8 s; 72.1 /s; i= 16 
3/8 - gosub:
 340.9 s; 61.7 /s; i= 18 
4/8 - if:
 181.8 s; 80.1 /s; i= 33 
5/8 - fn:
 135.3 s; 26.9 /s; i= 44 
6/8 - maths:
 110.1 s; 3.5 /s; i= 54 
7/8 - string:
 125.8 s; 39.2 /s; i= 48 
8/8 - array:
 103 s; 16.3 /s; i= 58 

overall index= 29
It was entirely painful running the same code on a real ZX Spectrum at under ⅓ the speed of a C64

(I mean: who knew that Commodore PET BASIC could run faster or slower depending on how your numbered your lines? Not me — until today, that is.)

While the benchmark doesn’t scale well for BASIC running on modern computers — the comparisons between a simple 8-bit processor at a few MHz and a multi-core wildly complex modern CPU at many GHz just aren’t applicable — it turns out I may have one of the fastest 8-bit BASIC computers around in the matchbox-sized shape of the MinZ v1.1 (36.864 Z180, CP/M 2.2, BBC BASIC [Z80] v3):

BASIC BENCH INDEX
>I GOOD. NTSC C64=100

1/8 - FOR:
 3.2 S; 12778 /S; I= 1895 
2/8 - GOTO:
 6.1 S; 4324.5 /S; I= 978 
3/8 - GOSUB:
 3.1 S; 6789 /S; I= 1935 
4/8 - IF:
 2.9 S; 4966.9 /S; I= 2046 
5/8 - FN:
 3.5 S; 1030.6 /S; I= 1698 
6/8 - MATHS:
 1.5 S; 255.3 /S; I= 4000 
7/8 - STRING:
 2.6 S; 1871.6 /S; I= 2279 
8/8 - ARRAY:
 3.1 S; 540.3 /S; I= 1935 

OVERALL INDEX= 1839 

That’s more than 9× the speed of a BBC Micro Model B.

Github link: bench64 – a new BASIC benchmark index for 8-bit computers.

Archive download:

BeagleBone Black: slow as a dog

All benchmarks are artificial, but this one had me scratching my head. One hears  that the BeagleBone Black is screamingly fast compared to the Raspberry Pi; faster, newer processor, blahdeblah, mcbtyc, etc. I found the opposite is true.

So I buy one at the exceptionally soggy Toronto Mini Maker Faire. Props to the CircuitCo folks, they are easy to set up: just a mini-USB cable provides power and virtual network shell. And BoneScript — an Arduino-like JavaScript library — is very clever indeed. But I need to see if this thing has any grunt, and so I need a benchmark.

After hearing about the business-card raytracer, I thought it would be perfect. I compiled it on both machines with:

g++  -Ofast   card.cpp   -o card

and then ran it with:

time ./card > /dev/null

The results are … surprising:

  • Raspberry Pi: 4′ 15″
  • BeagleBone Black: 12′ 39″ → 3× slower

(In comparison, my i7 quad-core laptop runs it in 8½ seconds.)

I don’t have any explanation why the BBB is so much slower. It’s almost as if the compiler isn’t fully optimizing under Ångström Linux.

Raspberry Pi: system info

$ uname -a
Linux rpi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l GNU/Linux

$ cat /proc/cpuinfo 
Processor    : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS    : 697.95
Features    : swp half thumb fastmult vfp edsp java tls 
CPU implementer    : 0x41
CPU architecture: 7
CPU variant    : 0x0
CPU part    : 0xb76
CPU revision    : 7

Hardware    : BCM2708
Revision    : 000f

BeagleBone Black: system info

# uname -a
Linux beaglebone 3.8.13 #1 SMP Tue Jun 18 02:11:09 EDT 2013 armv7l GNU/Linux
# cat /proc/cpuinfo 
processor    : 0
model name    : ARMv7 Processor rev 2 (v7l)
BogoMIPS    : 297.40
Features    : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls 
CPU implementer    : 0x41
CPU architecture: 7
CPU variant    : 0x3
CPU part    : 0xc08
CPU revision    : 2

Hardware    : Generic AM33XX (Flattened Device Tree)
Revision    : 0000

Both boards are running at stock speed.

Update: I’ve tried with an external power supply, and checked that the processor was running at full speed. It made no difference. I suspect that Raspbian enables armhf floating point by default, while Ångström needs to be told to use it.