Tag: unicode

  • I (U+1F494, BROKEN HEART) UTF-8

    Something has gone very wrong with the database encoding on this blog after a recent update, so all my lovely UTF-8 characters have gone mojibake.

    Trying to find ways to fix it. It may have to be manual. Remember, kids: have backups before letting WordPress upgrade!

    Here’s the Python equivalent of what I think the database has done:

    bytes("I ???? UTF-8", encoding='utf-8').decode(encoding='cp1252')
    'I 💔 UTF-8'

    Quite why my hosting thought a character encoding from last century was appropriate, I’ll never know.

    Update, November 2023: kinda-sort fixed the backend, but the encoding is still weird — can we…?

  • 𒐳 / ༳ == ( ⑽ – 𐹭 ) * ( 𒐲 / 𐅉 ), of course

    I just got brian d. foy’s Learning Perl 6 from the library. It’s a pretty good book, though it’ll take a good few readings for some of Perl 6’s features to stick.

    Since Perl 6 is built using Unicode from the ground up, it does two rather wonderful things when dealing with numbers:

    1. regular expressions match numerals beyond 0–9: ٤ is as much four as 4
    2. numeric constants can (pretty much) be expressed in terms of Unicode values in your Perl 6 source code. Assigning π to a variable does what you think it does. Dividing by ¼ is the same as multiplying by, well, ٤.

    So herewith a table (probably incomplete, and very unlikely to render properly for you) of Unicode glyphs accepted by Perl 6 as numeric values:

    Value Glyphs
    -0.5 ༳
    0 0 ٠ ۰ ߀ ० ০ ੦ ૦ ୦ ௦ ౦ ౸ ೦ ൦ ๐ ໐ ༠ ၀ ႐ ០ ៰ ᠐ ᥆ ᧐ ᪀ ᪐ ᭐ ᮰ ᱀ ᱐ ⁰ ₀ ↉ ⓪ ⓿ 〇 ꘠ ꛯ ꣐ ꤀ ꧐ ꩐ ꯰ 0 𐆊 𐒠 𑁦 𑃰 𑄶 𑇐 𑛀 𝟎 𝟘 𝟢 𝟬 𝟶 🄀 🄁
    0.0625 à§´ à­µ ê ³
    0.1 â…’
    0.111111 â…‘
    0.125 à§µ à­¶ â…› ê ´ ð’‘Ÿ
    0.142857 ⅐
    0.166667 â…™ ð’‘¡
    0.1875 à§¶ à­· ê µ
    0.2 â…•
    0.25 ¼ ৷ ୲ ൳ ꠰ 𐅀 𐹼 𒑠 𒑢
    0.333333 ⅓ 𐹽 𒑚 𒑝
    0.375 ⅜
    0.4 â…–
    0.5 ½ ୳ ൴ ༪ ⳽ ꠱ 𐅁 𐅵 𐅶 𐹻
    0.6 â…—
    0.625 ⅝
    0.666667 ⅔ 𐅷 𐹾 𒑛 𒑞
    0.75 ¾ ৸ ୴ ൵ ꠲ 𐅸
    0.8 â…˜
    0.833333 ⅚ 𒑜
    0.875 â…ž
    1 1 ¹ ١ ۱ ߁ १ ১ ੧ ૧ ୧ ௧ ౧ ౹ ౼ ೧ ൧ ๑ ໑ ༡ ၁ ႑ ፩ ១ ៱ ᠑ ᥇ ᧑ ᧚ ᪁ ᪑ ᭑ ᮱ ᱁ ᱑ ₁ ⅟ Ⅰ ⅰ ① ⑴ ⒈ ⓵ ❶ ➀ ➊ 〡 ㆒ ㈠ ㊀ ꘡ ꛦ ꣑ ꤁ ꧑ ꩑ ꯱ 1 𐄇 𐅂 𐅘 𐅙 𐅚 𐌠 𐏑 𐒡 𐡘 𐤖 𐩀 𐩽 𐭘 𐭸 𐹠 𑁒 𑁧 𑃱 𑄷 𑇑 𑛁 𒐕 𒐞 𒐬 𒐴 𒑏 𒑘 𝍠 𝟏 𝟙 𝟣 𝟭 𝟷 🄂
    1.5 ༫
    2 2 ² ٢ ۲ ߂ २ ২ ੨ ૨ ୨ ௨ ౨ ౺ ౽ ೨ ൨ ๒ ໒ ༢ ၂ ႒ ፪ ២ ៲ ᠒ ᥈ ᧒ ᪂ ᪒ ᭒ ᮲ ᱂ ᱒ ₂ Ⅱ ⅱ ② ⑵ ⒉ ⓶ ❷ ➁ ➋ 〢 ㆓ ㈡ ㊁ ꘢ ꛧ ꣒ ꤂ ꧒ ꩒ ꯲ 2 𐄈 𐅛 𐅜 𐅝 𐅞 𐏒 𐒢 𐡙 𐤚 𐩁 𐭙 𐭹 𐹡 𑁓 𑁨 𑃲 𑄸 𑇒 𑛂 𒐀 𒐖 𒐟 𒐣 𒐭 𒐵 𒑊 𒑐 𒑖 𒑙 𝍡 𝟐 𝟚 𝟤 𝟮 𝟸 🄃
    2.5 ༬
    3 3 ³ ٣ ۳ ߃ ३ ৩ ੩ ૩ ୩ ௩ ౩ ౻ ౾ ೩ ൩ ๓ ໓ ༣ ၃ ႓ ፫ ៣ ៳ ᠓ ᥉ ᧓ ᪃ ᪓ ᭓ ᮳ ᱃ ᱓ ₃ Ⅲ ⅲ ③ ⑶ ⒊ ⓷ ❸ ➂ ➌ 〣 ㆔ ㈢ ㊂ ꘣ ꛨ ꣓ ꤃ ꧓ ꩓ ꯳ 3 𐄉 𐒣 𐡚 𐤛 𐩂 𐭚 𐭺 𐹢 𑁔 𑁩 𑃳 𑄹 𑇓 𑛃 𒐁 𒐈 𒐗 𒐠 𒐤 𒐥 𒐮 𒐯 𒐶 𒐷 𒐺 𒐻 𒑋 𒑑 𒑗 𝍢 𝟑 𝟛 𝟥 𝟯 𝟹 🄄
    3.141592653589793 π
    3.5 ༭
    4 4 ٤ ۴ ߄ ४ ৪ ੪ ૪ ୪ ௪ ౪ ೪ ൪ ๔ ໔ ༤ ၄ ႔ ፬ ៤ ៴ ᠔ ᥊ ᧔ ᪄ ᪔ ᭔ ᮴ ᱄ ᱔ ⁴ ₄ Ⅳ ⅳ ④ ⑷ ⒋ ⓸ ❹ ➃ ➍ 〤 ㆕ ㈣ ㊃ ꘤ ꛩ ꣔ ꤄ ꧔ ꩔ ꯴ 4 𐄊 𐒤 𐩃 𐭛 𐭻 𐹣 𑁕 𑁪 𑃴 𑄺 𑇔 𑛄 𒐂 𒐉 𒐏 𒐘 𒐡 𒐦 𒐰 𒐸 𒐼 𒐽 𒐾 𒐿 𒑌 𒑒 𒑓 𝍣 𝟒 𝟜 𝟦 𝟰 𝟺 🄅
    4.5 ༮
    5 5 ٥ ۵ ߅ ५ ৫ ੫ ૫ ୫ ௫ ౫ ೫ ൫ ๕ ໕ ༥ ၅ ႕ ፭ ៥ ៵ ᠕ ᥋ ᧕ ᪅ ᪕ ᭕ ᮵ ᱅ ᱕ ⁵ ₅ Ⅴ ⅴ ⑤ ⑸ ⒌ ⓹ ❺ ➄ ➎ 〥 ㈤ ㊄ ꘥ ꛪ ꣕ ꤅ ꧕ ꩕ ꯵ 5 𐄋 𐅃 𐅈 𐅏 𐅟 𐅳 𐌡 𐒥 𐹤 𑁖 𑁫 𑃵 𑄻 𑇕 𑛅 𒐃 𒐊 𒐐 𒐙 𒐢 𒐧 𒐱 𒐹 𒑍 𒑔 𒑕 𝍤 𝟓 𝟝 𝟧 𝟱 𝟻 🄆
    5.5 ༯
    6 6 ٦ ۶ ߆ ६ ৬ ੬ ૬ ୬ ௬ ౬ ೬ ൬ ๖ ໖ ༦ ၆ ႖ ፮ ៦ ៶ ᠖ ᥌ ᧖ ᪆ ᪖ ᭖ ᮶ ᱆ ᱖ ⁶ ₆ Ⅵ ⅵ ↅ ⑥ ⑹ ⒍ ⓺ ❻ ➅ ➏ 〦 ㈥ ㊅ ꘦ ꛫ ꣖ ꤆ ꧖ ꩖ ꯶ 6 𐄌 𐒦 𐹥 𑁗 𑁬 𑃶 𑄼 𑇖 𑛆 𒐄 𒐋 𒐑 𒐚 𒐨 𒑀 𒑎 𝍥 𝟔 𝟞 𝟨 𝟲 𝟼 🄇
    6.5 ༰
    7 7 ٧ ۷ ߇ ७ ৭ ੭ ૭ ୭ ௭ ౭ ೭ ൭ ๗ ໗ ༧ ၇ ႗ ፯ ៧ ៷ ᠗ ᥍ ᧗ ᪇ ᪗ ᭗ ᮷ ᱇ ᱗ ⁷ ₇ Ⅶ ⅶ ⑦ ⑺ ⒎ ⓻ ❼ ➆ ➐ 〧 ㈦ ㊆ ꘧ ꛬ ꣗ ꤇ ꧗ ꩗ ꯷ 7 𐄍 𐒧 𐹦 𑁘 𑁭 𑃷 𑄽 𑇗 𑛇 𒐅 𒐌 𒐒 𒐛 𒐩 𒑁 𒑂 𒑃 𝍦 𝟕 𝟟 𝟩 𝟳 𝟽 🄈
    7.5 ༱
    8 8 ٨ ۸ ߈ ८ ৮ ੮ ૮ ୮ ௮ ౮ ೮ ൮ ๘ ໘ ༨ ၈ ႘ ፰ ៨ ៸ ᠘ ᥎ ᧘ ᪈ ᪘ ᭘ ᮸ ᱈ ᱘ ⁸ ₈ Ⅷ ⅷ ⑧ ⑻ ⒏ ⓼ ❽ ➇ ➑ 〨 ㈧ ㊇ ꘨ ꛭ ꣘ ꤈ ꧘ ꩘ ꯸ 8 𐄎 𐒨 𐹧 𑁙 𑁮 𑃸 𑄾 𑇘 𑛈 𒐆 𒐍 𒐓 𒐜 𒐪 𒑄 𒑅 𝍧 𝟖 𝟠 𝟪 𝟴 𝟾 🄉
    8.5 ༲
    9 9 ٩ ۹ ߉ ९ ৯ ੯ ૯ ୯ ௯ ౯ ೯ ൯ ๙ ໙ ༩ ၉ ႙ ፱ ៩ ៹ ᠙ ᥏ ᧙ ᪉ ᪙ ᭙ ᮹ ᱉ ᱙ ⁹ ₉ Ⅸ ⅸ ⑨ ⑼ ⒐ ⓽ ❾ ➈ ➒ 〩 ㈨ ㊈ ꘩ ꛮ ꣙ ꤉ ꧙ ꩙ ꯹ 9 𐄏 𐒩 𐹨 𑁚 𑁯 𑃹 𑄿 𑇙 𑛉 𒐇 𒐎 𒐔 𒐝 𒐫 𒑆 𒑇 𒑈 𒑉 𝍨 𝟗 𝟡 𝟫 𝟵 𝟿 🄊
    10 ௰ ൰ ፲ Ⅹ ⅹ ⑩ ⑽ ⒑ ⓾ ❿ ➉ ➓ 〸 ㈩ ㉈ ㊉ 𐄐 𐅉 𐅐 𐅗 𐅠 𐅡 𐅢 𐅣 𐅤 𐌢 𐏓 𐡛 𐤗 𐩄 𐭜 𐭼 𐹩 𑁛 𝍩
    11 Ⅺ ⅺ ⑪ ⑾ ⒒ ⓫
    12 Ⅻ ⅻ ⑫ ⑿ ⒓ ⓬
    13 ⑬ ⒀ ⒔ ⓭
    14 ⑭ ⒁ ⒕ ⓮
    15 ⑮ ⒂ ⒖ ⓯
    16 ৹ ⑯ ⒃ ⒗ ⓰
    17 ᛮ ⑰ ⒄ ⒘ ⓱
    18 ᛯ ⑱ ⒅ ⒙ ⓲
    19 ᛰ ⑲ ⒆ ⒚ ⓳
    20 ፳ ⑳ ⒇ ⒛ ⓴ 〹 ㉉ 𐄑 𐏔 𐡜 𐤘 𐩅 𐭝 𐭽 𐹪 𑁜 𝍪
    21 ㉑
    22 ㉒
    23 ㉓
    24 ㉔
    25 ㉕
    26 ㉖
    27 ㉗
    28 ㉘
    29 ㉙
    30 ፴ 〺 ㉊ ㉚ 𐄒 𐅥 𐹫 𑁝 𝍫
    31 ㉛
    32 ㉜
    33 ㉝
    34 ㉞
    35 ㉟
    36 ㊱
    37 ㊲
    38 ㊳
    39 ㊴
    40 ፵ ㉋ ㊵ 𐄓 𐹬 𑁞 𝍬
    41 ㊶
    42 ㊷
    43 ㊸
    44 ㊹
    45 ㊺
    46 ㊻
    47 ㊼
    48 ㊽
    49 ㊾
    50 ፶ Ⅼ ⅼ ↆ ㉌ ㊿ 𐄔 𐅄 𐅊 𐅑 𐅦 𐅧 𐅨 𐅩 𐅴 𐌣 𐩾 𐹭 𑁟 𝍭
    60 ፷ ㉍ 𐄕 𐹮 𑁠 𝍮
    70 ፸ ㉎ 𐄖 𐹯 𑁡 𝍯
    80 ፹ ㉏ 𐄗 𐹰 𑁢 𝍰
    90 ፺ 𐄘 𐍁 𐹱 𑁣 𝍱
    100 ௱ ൱ ፻ Ⅽ ⅽ 𐄙 𐅋 𐅒 𐅪 𐏕 𐡝 𐤙 𐩆 𐭞 𐭾 𐹲 𑁤
    200 𐄚 𐹳
    300 𐄛 𐅫 𐹴
    400 𐄜 𐹵
    500 Ⅾ ⅾ 𐄝 𐅅 𐅌 𐅓 𐅬 𐅭 𐅮 𐅯 𐅰 𐹶
    600 𐄞 𐹷
    700 𐄟 𐹸
    800 𐄠 𐹹
    900 𐄡 𐍊 𐹺
    1000 ௲ ൲ Ⅿ ⅿ ↀ 𐄢 𐅍 𐅔 𐅱 𐡞 𐩇 𐭟 𐭿 𑁥
    2000 𐄣
    3000 𐄤
    4000 𐄥
    5000 ↁ 𐄦 𐅆 𐅎 𐅲
    6000 𐄧
    7000 𐄨
    8000 𐄩
    9000 𐄪
    10000 ፼ ↂ 𐄫 𐅕 𐡟
    20000 𐄬
    30000 𐄭
    40000 𐄮
    50000 ↇ 𐄯 𐅇 𐅖
    60000 𐄰
    70000 𐄱
    80000 𐄲
    90000 𐄳
    100000 ↈ
    216000 𒐲
    432000 𒐳
    Inf ∞

    So the title of this post really is accepted as a valid Perl 6 expression in the REPL:

    $ perl6
    To exit type 'exit' or '^D'
    > 𒐳 / ༳ == ( ⑽ - 𐹭 ) * ( 𒐲 / 𐅉 )
    True

    What does it evaluate to? Well:

    • 𒐳 ‘CUNEIFORM NUMERIC SIGN SHAR2 TIMES GAL PLUS MIN’ represents 432000
    • ༳ ‘TIBETAN DIGIT HALF ZERO’ represents -½
    • ⑽ ‘PARENTHESIZED NUMBER TEN’ represents 10
    • 𐹭 ‘RUMI NUMBER FIFTY’ represents 50
    • 𒐲 ‘CUNEIFORM NUMERIC SIGN SHAR2 TIMES GAL PLUS DISH’ represents 216000
    • 𐅉 ‘GREEK ACROPHONIC ATTIC TEN TALENTS’ represents 10.

    Definitely into just because you can doesn’t mean you should territory, and a feature to make the Pythonistas reach for the Zantac again, poor dears.

  • ayaburnie!

    ayaburnie!

    ayaburnie!

    Instagram filter used: Normal

    View in Instagram ⇒

  • nerdy spreadsheet tick/cross formatting

    Screenshot from 2014-11-22 08:13:57

    The magic custom format string for this is:

    [Red][=0]✗;[Black][<>0]

    Works with LibreOffice and Excel on every platform I’ve tried.

  • Ⓗⓞⓦ ⓣⓞ â“‘â“” ⓐⓝⓝⓞⓨⓘⓝⓖ ⓦⓘⓣⓗ Ⓟⓔⓡⓛ ⓐⓝⓓ Ⓤⓝⓘⓒⓞⓓⓔ

    It’s been so long since I’ve programmed in Perl. Twelve years ago, it was my life, but what with the Raspberry Pi intervening, I hadn’t used it in a while. It’s been so long, in fact, that I wasn’t aware of the new language structures available since version 5.14. Perl’s Unicode support has got a lot more robust, and I’m sick of Python’s whining about codecs when processing anything other than ASCII anyway. So I thought I’d combine re-learning some modern Perl with some childish amusement.

    What I came up with was a routine to convert ASCII alphanumerics ([0-9A-Za-z]) to Unicode Enclosed Alphanumerics ([⓪-⑨Ⓐ-Ⓩⓐ-ⓩ]) for advanced lulz purposes. Ⓘ ⓣⓗⓘⓝⓚ ⓘⓣ ⓦⓞⓡⓚⓢ ⓡⓐⓣⓗⓔⓡ ⓦⓔⓛⓛ:

    #!/usr/bin/perl
    # annoying.pl - ⓑⓔ ⓐⓝⓝⓞⓨⓘⓝⓖ ⓦⓘⓣⓗ ⓤⓝⓘⓒⓞⓓⓔ
    # created by scruss on 2014-05-18
    
    use v5.14;
    # fun UTF8 tricks from http://stackoverflow.com/questions/6162484/
    use strict;
    use utf8;
    use warnings;
    use charnames qw( :full :short );
    sub annoyify;
    
    die "usage: $0 ", annoyify('string to print like this'), "\n" if ( $#ARGV < 0 );
    say annoyify( join( ' ', @ARGV ) );
    exit;
    
    # 💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩💩
    
    sub annoyify() {
        # convert ascii to chars in circles
        my $str = shift;
        my @out;
        foreach ( split( '', $str ) ) {
            my $c = ord($_);             # remember, can be > 127 for UTF8
            if ( $c == charnames::vianame("DIGIT ZERO") )
    	{
                # 💩💩💩 sigh; this one's real special ... 💩💩💩
                $c = charnames::vianame("CIRCLED DIGIT ZERO");
            }
            elsif ($c >= charnames::vianame("DIGIT ONE")
                && $c <= charnames::vianame("DIGIT NINE") )
            {
                # numerals, 1-9 only (grr)
                $c =
                  charnames::vianame("CIRCLED DIGIT ONE") +
                  $c -
                  charnames::vianame("DIGIT ONE");
            }
            elsif ($c >= charnames::vianame("LATIN CAPITAL LETTER A")
                && $c <= charnames::vianame("LATIN CAPITAL LETTER Z") )
            {
                # upper case
                $c =
                  charnames::vianame("CIRCLED LATIN CAPITAL LETTER A") +
                  $c -
                  charnames::vianame("LATIN CAPITAL LETTER A");
            }
            elsif ($c >= charnames::vianame("LATIN SMALL LETTER A")
                && $c <= charnames::vianame("LATIN SMALL LETTER Z") )
            {
                # lower case
                $c =
                  charnames::vianame("CIRCLED LATIN SMALL LETTER A") +
                  $c -
                  charnames::vianame("LATIN SMALL LETTER A");
            }
            else {
                # pass thru non-ascii chars
            }
            push @out, chr($c);
        }
        return join( '', @out );
    }
    

    Yes, I really did have to do that special case for ⓪; ⓪…⑨ are not contiguous like ASCII 0…9. ⓑⓞⓞ!

  • Compose yourself, Raspberry Pi!

    Years ago, I worked in multilingual dictionary publishing. I was on the computing team, so we had to support the entry and storage of text in many different languages. Computers could display accented and special characters, but we were stuck with 8-bit character sets. This meant that we could only have a little over 200 distinct characters display in the same font at the same time. We’d be pretty much okay doing French & English together, but French & Norwegian started to get a little trying, and Italian & Greek couldn’t really be together at all.

    We were very fortunate to be using Sun workstations in the editorial office. These were quite powerful Unix machines, which means that they were a fraction of the speed and capabilities of a Raspberry Pi. Suns had one particularly neat feature:

    Compose_key_on_Sun_Type_5c_keyboard

    (source: Compose key, Wikipedia.)

    That little key marked “Compose” (to the right of the space bar) acted as a semi-smart typewriter backspace key: if you hit Compose, then the right key combination, an accented character or symbol would appear. Some of the straightforward compose key sequences are:

     Compose +  
    AccentFirst keySecond keyResultExample
    Acuteeécafé
    Grave`aàdéjà
    Cedilla,cçsoupçon
    Circumflex^oôhôtel
    Umlautuüküche
    RingoaåHåkon
    Slash/LŁŁukasiewicz
    Tilde~nñmañana

    Like every (non-embedded) Linux system I’ve used, the Raspberry Pi running Raspbian can use the compose key method for entering extra characters. I’m annoyed, though, that almost every setup tutorial either says to disable it, or doesn’t explain what it’s for. Let me fix that for you …

    Setup

    Run raspi-config

    sudo raspi-config

    and go to the configure_keyboard “4 Internationalisation Options” → “I3 Change Keyboard Layout” section. Your keyboard’s probably mostly set up the way you want it, so hit the Tab key and select <Ok> until you get to the Compose key section:

    raspi-config: Compose key selection

    Choose whatever is convenient. The combined keyboard and trackpad I use (a SolidTek KB-3910) with my Raspberry Pi has a couple of “Windows® Logo” keys, and the one on the right works for me. Keep the rest of the keyboard options the same, and exit raspi-config. After the message

    Reloading keymap. This may take a short while
    [ ok ] Setting preliminary keymap...done.

    appears, you now have a working Compose key.

    Using the Compose key

    raspi-config hints (‘On the text console the Compose key does not work in Unicode mode …’) that Compose might not work everywhere with every piece of software. I’ve tested it across quite a few pieces of software — both on the text console and under LXDE — and support seems to be almost universal. The only differences I can find are:

    • Text Console — (a. k. a. the texty bit you see after booting) Despite raspi-config’s warning, accented alphabetical characters do seem to work (é è ñ ö ø å, etc). Most symbols, however, don’t (like ± × ÷ …). The currency symbol for your country is a special case. In Canada, I need to use Compose for  and £, but you’ve probably got a key for that.
    • LXDE — (a. k. a. the mousey bit you see after typing ‘startx’) All characters and symbols I’ve tried work everywhere, in LXTerminal, Leafpad, Midori, Dillo (browser), IDLE, and FocusWriter (a very minimal word processor).
    Special characters in Python's IDLE
    Special characters in Python’s IDLE
    Some Compose key sequences — Leafpad
    Some Compose key sequences — Leafpad

    To find out which key sequences do what, the Compose key – Wikipedia page is a decent start. I prefer the slightly friendlier Ubuntu references GtkComposeTable and Compose Key, or the almost unreadable but frighteningly comprehensive UTF-8 (Unicode) compose sequence reference (which is essentially mirrored on your Raspberry Pi as the file /usr/share/X11/locale/en_US.UTF-8/Compose). Now go forth and work that Compose key like a boß.

    (If you’re on a Mac and feeling a bit left out, you can do something similar with the Option key. Here’s how: Extended Keyboard Accent Codes for the Macintosh. On Windows®? Out of luck, I’m afraid WinCompose!)

  • disnaeland

    Conclusive proof (if any were needed) that Scotland invented Unicode:

    didnae

    isnae

    wasnae

    If you try to display a UTF-8 apostrophe on an ISO 8859-15 system, you get a reasonable representation of didnae, isnae and wasnae.