Akom's Tech Ruminations

Various tech outbursts - code and solutions to practical problems

Migrating one Gmail account to another Code and Hacks

Posted by Admin • Friday, May 13. 2011 • Category: Code and Hacks

I have a setup where a Google Apps Premier account is used for the active employees, but when they quit I want to archive all their mail but free up a paid user account for the next employee. We archive mail in a different domain, also on google apps - a standard edition, free. For a while now I've gone through a variety of ways of copying the mailboxes from one place to the other, and each approach was either partial, unreliable, or too time-consuming.

So I think I've finally settled on a viable approach that is neither.

Here is the list of typical solutions:

  • Forward all email (eg via filters). Somewhat unpredictable, I think, and the labels are lost
  • Use a desktop mail client (eg: Thunderbird) to access both new and old IMAP accounts, then drag messages around. This is reliable only to the extent that a human is error-free - as in, not very. But if you do it right, you keep your labels
  • Using gmail's built-in POP fetcher (gmail->Settings->Accounts). This will also lose the labels, but the biggest issue is that it's currently very slow. It took 4 hours for me to see the first messages that it claimed it fetched right away.


The preferred way


There are two applications I know of that work well enough to satisfy my needs: ImapCopy and Imapsync. In either case, you will first need to configure the email accounts:

Email Accounts

Make sure that IMAP is enabled for both source and destination accounts by logging into gmail as both and checking Settings->IMAP/POP tab. Make sure that you know the passwords for both accounts.

ImapSync


"imapsync" may be a simpler alternative to imapcopy - it's written in perl, it supports SSL, and it may not be as prone to crashing on network trouble. Moreover, it seems to skip messages that are already copied, saving a ton of time when resuming the transfer.

Setup

Grab the free copy of imapsync. You don't need to do much with it besides run it. Since you probably won't have the prerequisite libraries, run
sudo cpan Mail::IMAPClient
.

Running

Basically boils down to specifying all the required command line options. Run ./imapsync to see the list.
For Gmail it will look like this:
./imapsync --host1 imap.gmail.com --host2 imap.gmail.com --port1 993 --port2 993 --ssl1 --ssl2 --user1 something@gmail.com --user2 somethingelse@gmail.com --password1 your_password --password2 your_password
(The accounts may be @your.domain as well, naturally, and you may find it necessary to restrict which folders are copied using --folder X. If you really don't care about tags, you can specify --folder "[Gmail]/All Mail")


ImapCopy



I was fortunate to come across an application called ImapCopy. This is a command-line app written in Pascal that appears to work quite well, diligently copying mail from many accounts to many accounts (batched execution). That's the good news. The bad news is that it hasn't been updated since 2006 and while it works fine, it does not support SSL, which is required on imap.gmail.com

Fortunately that is a limitation that's easy to overcome - use stunnel. Stunnel is fairly easy to set up to work in "client" mode - which is what we want. All it needs to do is accept an insecure connection locally and SSL encrypt it on its way to the actual destination - which is imap.gmail.com:993 .

Setting it up

Prerequisites
  • Make a directory where you'll set all this up, eg "imapcopy". We'll do everything in this directory.
    mkdir imapcopy cd imapcopy
  • If you're on a typical linux distribution, you have openssl already installed - if not, install it, eg:
    sudo apt-get install openssl
  • Also install stunnel and imapcopy (all 3 are nicely packaged in the standard Ubuntu package repositories), eg:
    sudo apt-get install stunnel imapcopy
  • Stunnel
    1. First we'll need to generate a certificate:
      openssl req -new -x509 -days 3650 -nodes -out stunnel.pem -keyout stunnel.pem
      It doesn't matter what you enter in the fields, we just need a certificate
    2. Now we need a config file for stunnel, let's name it stunnel-to-gmail.conf:
      stunnel-to-gmail.conf
      cert = stunnel.pem foreground=yes ; Some performance tunings socket = l:TCP_NODELAY=1 socket = r:TCP_NODELAY=1 ; Some debugging stuff useful for troubleshooting debug = 7 ; Use it for client mode client = yes ; Service-level configuration [imap] accept = 1143 connect = imap.gmail.com:993 retry = yes
    3. This lets us start stunnel without needing root (in theory, anyway) by running:
      stunnel4 stunnel-to-gmail.conf
      You can background it if you like, but I'd keep it running in a shell.
    4. If there are no errors, then you should now be able to telnet localhost 1143 (just to test) and see a nice IMAP protocol prompt from google
    imapcopy config
    1. We'll also need a config file for imapcopy. On Ubuntu it comes bundled with the distribution as an example in /usr/share/doc/imapcopy/examples/ImapCopy.cfg, so:
      cp /usr/share/doc/imapcopy/examples/ImapCopy.cfg .
    2. Edit the file and change: the port for both src and dst to 1143, and the usernames/passwords at the bottom of the file.
    3. Running ImapCopy

      You should theoretically be ready to go. Simply running imapcopy in your imapcopy directory should be all you need to do. It will walk through each account, duplicating folders and reading/writing each message, and it will take ages, but you don't have to sit there and watch it. And next time you need to migrate an account, just edit ImapCopy.cfg and change the user accounts.

      Additional Thoughts

      • It may be beneficial to edit ImapCopy.cfg a bit to filter out duplicate IMAP folders - things like [Gmail]/All Mail seem to cause it to copy messages multiple times.
      • It may be possible to use one destination account for archiving many source accounts via the use of DstRootFolder setting in ImapCopy.cfg - if I understand it correctly that is
      • If you find yourself using this a lot, you may want to leave stunnel running. To do this: copy stunnel.pem and stunnel-to-gmail.conf to /etc/stunnel, enable stunnel (/etc/default/stunnel), and let the init script start it from now on (manually this time: /etc/init.d/stunnel start). You will also want to comment out foreground and debug in the conf file

    0 Trackbacks

    1. No Trackbacks

    1 Comments

    Display comments as (Linear | Threaded)
    1. Thanks a lot for this tutorial. It helps me a lot to migrate one Google Apps domain to another one quickly.

    Add Comment


    You can use [geshi lang=lang_name [,ln={y|n}]][/geshi] tags to embed source code snippets.
    Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
    Standard emoticons like :-) and ;-) are converted to images.

    To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
    CAPTCHA

    What is the primary language of this blog? (Anti-SPAM question)


    Submitted comments will be subject to moderation before being displayed.