Using Offlineimap with the Gmail IMAP API

In a previous guide I documented my mutt + offlineimap + notmuch setup.  If you’re using Gmail with IMAP enabled you can still utilize the superpower of this trio but you’ll need to do some things differently.  I’ll show you how to use offlineimap with the Google Gmail OAuth2 API and configure it.

UPDATE: 2022-08-24:  October 3rd 2022 Google will turn off their API support for cited security reasons.  Although I find their reasoning somewhat flimsy in this regard this is what they’re doing.

I have reverted to using Google App passwords for OfflineIMAP for now.

You can (until there is another API way to do this) go to your Google App Passwords settings and create an application password (requires turning 2FA on) and then in your offlineIMAP configuration use the following settings:

remotehost =
remoteuser =

I am going to keep the old blog post guide intact below for posterity and if alternate API access becomes available in the future I’ll update it accordingly.

Why API?
You can simply enable 2-factor authentication and create an app password for offlineimap, that’s fine too.  I like the API method because it’s more secure (you require two unique items + a refresh token).

Setting up the Gmail API
Offlineimap has some steps listed to get your offlineimap setup and running on their Github but we’ll walk through them in detail.

Make sure the IMAP API is enabled (toggled in the above link) then you’ll just need three things for API access for Offlineimap which we’ll cover:

  1. Client ID
  2. Client Secret
  3. Refresh Token

Create a New OAuth2 Application

You first need to head here and register a new OAuth2 application.  You may also be prompted for some 2-factor information here or a general authorization screen which I didn’t capture but should be self-explanatory.

  • Create a new project (e.g. offlineimap, name doesn’t matter)

  • In API  and Auth:  select credentials
  • Click Enable
  • Click Create Credentials

  • Create Client ID (Application type – Other)

  • Client ID and Secret

Above you should have now have an enabled client ID and secret available, copy these down because you’ll need them later.

Generate Oauth2 Refresh Token
The last thing we’ll need is an OAuth2 refresh token, to retrieve that you’ll need to use the Google OAuth tools repository.

git clone

Next you’ll need to generate your oath2 token and refresh token.  Make sure to change the YOUR_* strings below with your client ID and your client secret.

Note:  This should be run with python2, thanks to folks in comments who discovered this.  At the time of writing this guide I was using Python2.

cd gmail-oauth2-tools
python2 python/ --generate_oauth2_token \
--client_id=YOUR_CLIENT_ID --client_secret=YOUR_CLIENT_SECRET

You should see the refresh token string as the output for the above command, copy this as well since we’ll need all three to configure offlineimap.

Configure Offlineimap for API Access
We’re almost done!  Now you need the following lines in your offlineimap configuration file.  I’ve also made a sanitized copy for your reference.

[Repository ExampleCompanyRemote]
auth_mechanisms = XOAUTH2
oauth2_client_id = YOUR_CLIENT_ID_HERE
oauth2_client_secret = YOUR_CLIENT_SECRET_HERE
oauth2_request_url =
oauth2_refresh_token = YOUR_REFRESH_TOKEN_HERE

Fill in the above values you have received and your refresh token, remove any other authentication options.

Filter Gmail Specific IMAP Prefixes
Gmail wants to apply a [Gmail]/ prefix to all your existing IMAP folders (they call them labels, more on that later).  Let’s remove this from our mail so our beautiful mutt sidepane isn’t tarnished.

Add the following to your local repository configuration in .offlineimaprc

[Repository ExampleCompanyLocal]
## Remove GMAIL prefix on Google-specific IMAP folders that are pulled down.
nametrans = lambda f: '[Gmail]/' + f if f in ['Drafts', 'Starred', 'Important', 'Spam', 'Trash', 'All Mail', 'Sent Mail'] else f

You’ll need a similiar line for your remote repository configuration in .offlineimaprc

[Repository ExampleCompanyRemote]
## remove Gmail prefix on IMAP folders
nametrans = lambda f: f.replace('[Gmail]/', '') if f.startswith('[Gmail]/') else f

Awesome, we’re almost done.

Omit Duplicated Mail Labels for IMAP
Google thinks they can re-invent email smarter and better than everyone else, it tries to do this with the labelling system by treating all your mail as one big pile (All Mail) and then using labels (which can suffice as IMAP folders on the client side) to simply tag messages to show up in user defined pseudo-folders.  This is good enough for mutt for the most part.

Gmail sort of does IMAP, it is standards-compliant enough to let you pull mail but it also does some irritating (from an IMAP client perspective) message de-duplication.  You need to omit a few labels from being visible from IMAP so you don’t pull down your messages two or three times (one time for All Mail, the other times for Starred and Important).

Go to the gmail interface –> Settings –> Labels and uncheck show in IMAP for ‘All Mail’, ‘Starred’ and ‘Important’.

Turn Off Smartypants Predictive Analysis and Filtering
While Google is scanning all of your email for marketing purposes it tries to also do predictive analysis by default.  I turn all of this off because it can mess add unnecessary flags to your emails which don’t jive well in mutt.

I also set filters to not be overridden so you can control exactly how things are filtered as this setting is related to the first one.  This can be set in Settings -> Inbox

Sync Offlineimap
At this point you’re ready to sync offlineimap for the first time!

while sleep 50; do time offlineimap; done

Note that I am building in extra fault tolerance into the first sync command because of some issues highlighted below.  The above command will take a while to do a first full, initial sync of your mailbox and emails if it’s quite large like mine.  I believe it took me around 1-2 weeks to sync around 2million emails and attachments with countless crashes (see below).

Subsequent sync commands will not need this hack.

Once you’ve gotten a full sync you can proceed with step 3a from my original guide to setting up mutt + offlineimap + notmuch.

Drafts and Sent Mail
On the mutt side you can utilize the default mutt configuration settings to ensure your Sent Mail and Drafts are synced by ways (my config file has good examples)

# sync sent mail
set record="~/Maildir/Sent Mail"
# sync drafts
set postponed="~/Maildir/Drafts"

Syncing Mail – Gmail and Reliability
I was able to crash gmail countless times just by syncing my mail, it always came back relatively quickly but I incurred quite a bit of 500’ish errors on both offlineimap sync, mailbox unavailable and the web interface.

I am guessing my mailbox is run in a Kubernetes container because I could detect slowdowns accessing it from more than one location while syncing and when it was unavailable via IMAP it was also unavailable via the web interface.

Bulk actions also tend to make it angry like applying labels across a few hundred thousand emails at a time.  That’s ok, refresh and wait a few seconds and drive on citizen.

It always came back quickly but this is why we put some some .. ahem .. resiliency into our initial IMAP sync command, especially if you have a lot of mail.

Gmail – The Good and the Bad
On principle I don’t care for most closed source technology nor services that are not 100% RFC standards compliant.  Gmail itself also doesn’t seem as resilient as other enterprise IMAP solutions I’ve used, but it recovers quickly.

The label system also has some quirks and it’s not as flexible at adding multi-condition filtering for people who have meticulous IMAP organization or subscribe to a lot of Open Source mailing lists.  Generally you can work around this with label ordering or multiple filters per list.

One of the benefits of Gmail is that if your company uses it for hosted email you no longer require VPN access to retrieve your mail from clients.  Another benefit is that the sync just seems faster.  Being located in Ireland I’m always using Google servers very close to me which may not be the case if your company collocates their datacenter in one geographical continent for email services.

Overall I’m happy with this solution and it works well for me, with a little bit of modification I can continue to use my preferred setup.

Thanks to John Eckersberg who gave me the idea to use the Gmail API and for being the best around.

Extending Desktop Notifications with mail-notifications
Victor Kareh has suggested the usage of the mail-notification program on Linux to extend mail notifications further via inotify.  From his experience he writes

mail-notification is almost too easy, just add a new email
mailbox on Autodetect, and hard-code the location to your INBOX
directory. e.g. file:///home/vkareh/Mail/

Then you’ll have a notification tray icon showing you the number of
unread emails across any mailboxes. Optionally can show popup
notifications with the sender and subject. This works well in MATE and
probably XFCE and other “old-school” desktops, not quite sure if it
works in GNOME (since I think they’ve dropped support for the tray?).

About Will Foster

hobo devop/sysadmin/SRE
This entry was posted in open source, sysadmin and tagged , , , , , , , , , , , . Bookmark the permalink.

16 Responses to Using Offlineimap with the Gmail IMAP API

  1. Doug says:

    Thank you for this information. I’m a new-ish user- can you (briefly) explain why this API route is more secure? Isn’t the client secret still sitting in a plain text config file? I don’t understand why this would be preferable to storing your password in a gpg encrypted file. Thanks again.

    Liked by 1 person

    • Will Foster says:

      Thank you for this information. I’m a new-ish user- can you (briefly) explain why this API route is more secure? Isn’t the client secret still sitting in a plain text config file? I don’t understand why this would be preferable to storing your password in a gpg encrypted file. Thanks again.

      Hey Doug, that’s a good question. Ultimately if someone has control of your config file they’ll be able to usurp the same level of access so you’re right there, however I do see the API method being a bit more secure because there’s a refresh token, a forced length private secret, and an unique client id whereas with a simple application password is just a generated singular item. In this regard you’d need the sum of all the parts (API) to steal access whereas someone only needs to find the application password by itself to do the same. As you pointed out having access to the configuration file here invalidates the security of any approach.


  2. anataua says:

    I just followed all the steps, I was able to get refresh oauth2_refresh_token, I didi averithing well, but still I get
    error: [ALERT] Please log in via your web browser: (Failure)

    is there a log or other stuff I can use to diagnose this issue? I am desperately trying to sync gmail account by more the 10 hours :(


    • Will Foster says:

      Hey Anataua,

      Are you using 2-factor authentication? Mine is setup this way. I’d say that if you’re in a hurry you can just generate an application password and use that, it will do the same thing.


  3. dekks herton says:

    File “”, line 291
    print ‘Missing options: %s’ % ‘ ‘.join(missing)

    the script throws that error after pasting and inserting my details, any ideas?


  4. Vinod says:

    Hello, I am stuck at the generating refresh token section and get the error (like dekks herton):
    File “python/”, line 297
    print ‘Missing options: %s’ % ‘ ‘.join(missing)
    Any idea how to fix this???


  5. Stefan says:

    how do you handle archived mails? Are they kept in the inbox folder and only tagged archived?
    I don’t see how you could get archived mails from IMAP when not syncing “All Mail”


    • Will Foster says:

      I believe they are located in “All Mail”, all archiving really does as you mention is apply a tag. All the Inbox is in reality the same thing, just a tag. If you had another tag associated with these and defined an IMAP folder structure in your ~/.muttrc then they’d show up there too.

      Personally, I don’t even include “All Mail” in my ~/.muttrc config so I don’t see it on the IMAP pane but I do strip the Gmail-specific stuff from the folder name along with the rest of the gmail-specific folders.

      e.g. in ~/.offlineimaprc


  6. Hi there, thank you for writing up this great article, I followed your article and have an error:
    Appreciate if you would like to help.



    • Will Foster says:

      Hey Winston, looking at your .offlineimaprc settings they look like mine (no problems), perhaps your issue is with the actual oauth / setup on the Google side.

      I would double-check that the OAUTH2 parts of the setup are done correctly and your token is indeed active and can be used – the error seems to point to invalid auth resource on the Google/Oauth2 side and not your config (assuming the token, secret etc. is correct).

      Liked by 1 person

      • winstonfan says:

        Hi Will,

        Thanks for your input.
        I re-used the client id and secret which I created for my another project. But the refresh token was new and was just generated.

        Yes, I did look at the configuration of Google Gmail API, but I cannot see any place which allows me to update the description.

        If I provide screenshots of the configuration of Google Gmail API, would it be useful for spotting the issue?


  7. winstonfan says:

    Thanks for this great article,
    after I followed it I have an error:
    Appreciate if you could help,


  8. Tonus says:

    Thanks for your tricks ! All well in 2022 as Gmail is shutting down “less secure” method of auth.
    Just had to carefully look at your offlineimaprc. FTR here’s what I’ve ended with :
    [Repository account_Remote]
    type = IMAP
    remotehost =
    ssl = yes
    starttls = no
    ssl_version = tls1_2
    remoteuser =
    auth_mechanisms = XOAUTH2
    oauth2_client_id =
    oauth2_client_secret =
    oauth2_request_url =
    oauth2_refresh_token =
    realdelete = yes
    createfolders= False
    sslcacertfile = /etc/ssl/certs/ca-certificates.crt

    Liked by 1 person

  9. Andreas Schamanek says:

    Starting October 2022, Google announced that this approach won’t work anymore because OAuth out-of-band (OOB) will be deprecated:
    In case someone’s wondering: `REDIRECT_URI = ‘urn:ietf:wg:oauth:2.0:oob’` is to be found in the python/ script from Google’s own gmail-oauth2-tools.
    Sadly, I don’t know how to proceed from here. Also, official instructions as in still use OOB.

    Liked by 1 person

  10. Will Foster says:

    I have reverted to using Application Passwords (requires 2FA) for now given the 3-OCT 2022 cutoff for using OOB API Auth for IMAP. If another API-based workflow becomes available in the future I’ll update this guide to reflect it, otherwise I guess it’s App passwords.

    Liked by 1 person

Have a Squat, Leave a Reply ..

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.