Mac to Linux: 1Password to KeePassX

I have too many passwords to remember, so I’ve been using a password manager for years. First there was Keyring for Palm OS, then 1Password on the Mac. 1Password’s a very polished commercial program, but it only has Mac and Windows desktop clients. Sadly, it had to go.

Finding a replacement was tough. It needed to be free, and yet cross-platform. It needed to work on iOS and Android. It also needed to integrate with a cloud service like Dropbox so I could keep my passwords in sync. The only program that met all of these requirements was KeePassX. I’ve stuck with the stable (v 0.4.3) branch rather than the flashy 2.0 version, as the older database format does all I need and is fully portable. MiniKeePass on iOS and KeePassDroid on Android look after my mobile needs. But first, I needed to get my password data out of 1Password.

1Password offers two export formats: a delimited text format (which seemed to drop some of the more obscure fields), and the 1Password Interchange Format (1PIF). The latter is a JSONish format (ಠ_ಠ) containing a dump of all of the internal data structures. There is, of course, no documentation for this file format, because no-one would ever move away from this lovely commercial software, no …

So armed with my favourite swiss army chainsaw, I set about picking the file apart. JSON::XS and Data::Dumper::Simple were invaluable for this process, and pretty soon I had all the fields picked apart that I cared about. I decided to write a converter that wrote KeePassX 1.x XML, since it was readily imported into KeePassX, would could then write a database readable by all of the KeePass variants.

To run this converter you’ll need Perl, the JSON::XS and Data::Dumper::Simple modules, and if your Perl is older than about 5.12, the Time::Piece module (it’s a core module for newer Perls, so you don’t have to install it). Here’s the code:

#!/usr/bin/perl -w
# 1pw2kpxxml.pl - convert 1Password Exchange file to KeePassX XML
# created by scruss on 02013/04/21

use strict;
use JSON::XS;
use HTML::Entities;
use Time::Piece;

# print xml header
print <<HEADER;
<!DOCTYPE KEEPASSX_DATABASE>
<database>
 <group>
  <title>General</title>
  <icon>2</icon>
HEADER

##############################################################
# Field Map
#
# 1Password			KeePassX
# ============================  ==============================
# title        			title
# username			username
# password			password
# location			url
# notesPlain			comment
#    -				icon
# createdAt			creation
#    -				lastaccess	(use updatedAt)
# updatedAt			lastmod
#    -				expire		('Never')

# 1PW exchange files are made of single lines of JSON (O_o)
# interleaved with separators that start '**'
while (<>) {
    next if (/^\*\*/);    # skip separator
    my $rec = decode_json($_);

    # throw out records we don't want:
    #  - 'trashed' entries
    #  -  system.sync.Point entries
    next if ( exists( $rec->{'trashed'} ) );
    next if ( $rec->{'typeName'} eq 'system.sync.Point' );

    print '  <entry>', "\n";    # begin entry

    ################
    # title field
    print '   <title>', xq( $rec->{'title'} ), '</title>', "\n";

    ################
    # username field - can be in one of two places
    my $username = '';

    # 1. check secureContents as array
    foreach ( @{ $rec->{'secureContents'}->{'fields'} } ) {
        if (
            (
                exists( $_->{'designation'} )
                && ( $_->{'designation'} eq 'username' )
            )
          )
        {
            $username = $_->{'value'};
        }
    }

    # 2.  check secureContents as scalar
    if ( $username eq '' ) {
        $username = $rec->{'secureContents'}->{'username'}
          if ( exists( $rec->{'secureContents'}->{'username'} ) );
    }

    print '   <username>', xq($username), '</username>', "\n";

    ################
    # password field - as username
    my $password = '';

    # 1. check secureContents as array
    foreach ( @{ $rec->{'secureContents'}->{'fields'} } ) {
        if (
            (
                exists( $_->{'designation'} )
                && ( $_->{'designation'} eq 'password' )
            )
          )
        {
            $password = $_->{'value'};
        }
    }

    # 2.  check secureContents as scalar
    if ( $password eq '' ) {
        $password = $rec->{'secureContents'}->{'password'}
          if ( exists( $rec->{'secureContents'}->{'password'} ) );
    }

    print '   <password>', xq($password), '</password>', "\n";

    ################
    # url field
    print '   <url>', xq( $rec->{'location'} ), '</url>', "\n";

    ################
    # comment field
    my $comment = '';
    $comment = $rec->{'secureContents'}->{'notesPlain'}
      if ( exists( $rec->{'secureContents'}->{'notesPlain'} ) );
    $comment = xq($comment);    # pre-quote
    $comment =~ s,\\n,<br/>,g;  # replace escaped NL with HTML
    $comment =~ s,\n,<br/>,mg;  # replace NL with HTML
    print '   <comment>', $comment, '</comment>', "\n";

    ################
    # icon field (placeholder)
    print '   <icon>2</icon>', "\n";

    ################
    # creation field
    my $creation = localtime( $rec->{'createdAt'} );
    print '   <creation>', $creation->datetime, '</creation>', "\n";

    ################
    # lastaccess field
    my $lastaccess = localtime( $rec->{'updatedAt'} );
    print '   <lastaccess>', $lastaccess->datetime, '</lastaccess>', "\n";

    ################
    # lastmod field (= lastaccess)
    print '   <lastmod>', $lastaccess->datetime, '</lastmod>', "\n";

    ################
    # expire field (placeholder)
    print '   <expire>Never</expire>', "\n";

    print '  </entry>', "\n";    # end entry
}

# print xml footer
print <<FOOTER;
 </group>
</database>
FOOTER

exit;

sub xq {                         # encode string for XML
    $_ = shift;
    return encode_entities( $_, q/<>&"'/ );
}

To run it,

./1pw2kpxxml.pl data.1pif > data.xml

You can then import data.xml into KeePassX.

Please be careful to delete the 1PIF file and the data.xml once you’ve finished the export/import. These files contain all of your passwords in plain text; if they fell into the wrong hands, it would be a disaster for your online identity. Be careful that none of these files accidentally slip onto backups, too. Also note that, while I think I’m quite a trustworthy bloke, to you, I’m Some Random Guy On The Internet. Check this code accordingly; I don’t warrant it for anything save for looking like line noise.

Now on github: scruss / 1pw2kpxxml, or download: 1pw2kpxxml.zip (gpg signature: 1pw2kpxxml.zip.sig)

SHA1 Checksums:

  • 3c25eb72b2cfe3034ebc2d251869d5333db74592 — 1pw2kpxxml.pl
  • 99b7705ff30a2b157be3cfd29bb1d4f137920c25 — readme.txt
  • de4a51fbe0dd6371b8d68674f71311a67da76812 — 1pw2kpxxml.zip
  • f6bd12e33b927bff6999e9e80506aef53e6a08fa — 1pw2kpxxml.zip.sig.txt

The converter has some limitations:

  • All attached files in the database are lost.
  • All entries are stored under the same folder, with the same icon.
  • It has not been widely tested, and as I’m satisfied with its conversion, it will not be developed further.

Mozilla Mail Went Nuts

Update, 7/8/2004: It’s probably better to use Mozilla’s configuration editor from the URL about:config, rather than using a text editor on your prefs.js file. This is explained in Restoring mozilla mail local folders

For the last few days, Mozilla has been bugging me with “Please
enter a new password for user scruss@…”. It happened after I
added an extra mail profile to reply to a mailing list
unsubscription from my bigfoot.com address. The popup would appear
randomly, sometimes several times a minute. Using Password Manager
to save the POP3 password didn’t seem to help.

If you ever notice an extra account in the “From:” drop-down in
the Compose window, you could be having this problem. Today, I
found a messy way of fixing this.

After making a backup copy of prefs.js, edit it. Look for the
section that begins something like:

user_pref("mail.account.account1.identities", "id1");
user_pref("mail.account.account1.server", "server1");
...
user_pref("mail.accountmanager.accounts", "account1,account2,...");

Check the mapping between

  • id1, account1 and server1
  • id2, account2 and server2
  • id3, account3 and server3

and so on. You’ll probably find an id that’s pointing somewhere
broken. In my case, id2 was pointing to that dummy entry to bigfoot.com that I thought I had deleted.

Edit out all references to the broken id, and restart Mozilla.
All is well, for me at least.