Thunderbird: From mbox to maildir

Discussion in 'other software & services' started by summerheat, Oct 30, 2016.

  1. summerheat

    summerheat Registered Member

    Joined:
    May 16, 2015
    Posts:
    1,197
    Thunderbird has always been my favourite email application. I've tried a few other ones but always went back because no other one is so flexible and extensible, IMHO. (And Outlook and the likes are out of question as I'm running Linux.)

    One disadvantage of Thunderbird is that it is still using the mbox file format by default. Basically this means that in each folder all mails are comprised in one big text file. On the other hand, when using the maildir format each mail is kept in a separate file.

    The advantages/disadvantages of both formats are discussed here and in this thread (obviously started by Wilders member @WildByDesign ). In my opinion, the most important advantages of maildir over mbox are the following:

    1. The maildir format is much more robust. The risk of losing data is much smaller if Thunderbird or your computer crashes.
    2. Incremental backups are much faster as only the new and modified files have to be backupped.
    3. Scanning your mails with an AV scanner is easier, and infected mails can be removed without the risk of data loss.

    Those advantages are particularly important if you have a large database of emails going back many years.

    This Mozilla page says that maildir is still disabled by default because there are still many bugs. But hey - I've been running Thunderbird with maildir for several weeks and five mail accounts and haven't run into a single problem! On the contrary, it's my impression that Thunderbird is snappier than ever before.

    Besides, several of those bugs in that list refer to problems which arise when mixing mbox and maildir. Don't do that! This is what I've done and suggest:

    1. Start Thunderbird with the -p switch (thunderbird -p) and create a new profile. So you can always go back to your old profile if you run into problems! This is really a very important step!
    2. Start TB with this new profile. Before doing anything else go to
      Options - Advanced - Advanced Configuration - Message Store Type for new accounts and select the maildir format.
    3. Create your email accounts. If you're using IMAP just retrieve the mails anew from the server and you're done.
    4. If you're using POP3 (which is what I do for various reasons) it's more complicated: I suggest in this case that you start TB with your old mbox profile and install the ImportExportTools addon. Then go to each folder, right-click it and select Export all messages in the folder > EML format. Then close Thunderbird, execute thunderbird -p again and start your new maildir profile. Install ImportExportTools, create the (sub)folders of your old profile manually, right-click each, select Import all messages from a directory and choose the corresponding folder where you exported the messages earlier. Just in case that TB already retrieved messages from the mail servers and you have duplicates, I suggest to install Remove Duplicate Messages and remove them. In order to avoid duplicates from the beginning I would after creating the account(s) unselect Check for new messages at startup and Check for new messages every ... minutes in the account settings. Close Thunderbird and copy the correct popstate.dat files from the old profile. If all is well, you can enable those settings again.
    5. In order to import your data (filters, address books, etc.) to the new profile go to Transferring data to a new profile - Thunderbird, and meticulously follow the steps there.
     
    Last edited: Oct 30, 2016
  2. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,228
    Location:
    Toronto, Canada
    @summerheat Excellent topic. There did not end up being much discussion on it over at DSLReports forum, so hopefully it will have more attention here. But anyway, I am still 100% rocking the maildir mail store within my own Thunderbird setup since before I created that thread over there. I've got my main POP3 account, Local Folders (though not used), and a pretty extensive Blog & News Feeds (for all my RSS needs) all setup within maildir and it is lightning fast. No need to compress large mail files anymore. You delete an email, that single file simply gets deleted. No bloat, no mess.

    I have not followed up with that GSoC effort that was intended to create a mbox>maildir conversion utility, but your new thread here now has reminded me to look into that further and see how far along that effort is. :thumb:
     
  3. summerheat

    summerheat Registered Member

    Joined:
    May 16, 2015
    Posts:
    1,197
    I don't know, either, if there is any progress with that utility. On the other hand I would always create a new profile when switching to maildir. So I don't know if such a utility would yield that much - unless it would be able to convert a complete profile with maintaining the complete folder structure.
     
  4. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,228
    Location:
    Toronto, Canada
    Here is the bug report (mbox>maildir converter and vice versa) with lots of recent activity: https://bugzilla.mozilla.org/show_bug.cgi?id=856087
    Seems it will be built-in instead of as an extension. Questionable whether they will land this in time for stable 52 release, though they are discussing hiding behind a pref initially.
     
  5. summerheat

    summerheat Registered Member

    Joined:
    May 16, 2015
    Posts:
    1,197
    Thanks, that's good to see! Too late for me, though ;)
     
  6. haakon

    haakon Guest

    I have to admit, this discussion is the first I've noticed re mbox and maildir in TB.

    I've been using TB since v0.6 for my personal stuff. With occasional support under some SOHO support contracts.

    Several years ago having ended the professional need (mine and enterprise admin) for Outlook, I migrated years of email from the PST.

    My Mail folder will soon exceed 900MB.

    The profile folder is backed up daily but I've never had to recover from a TB failure.

    Nonetheless, I recognize the likely advantages maildir has to offer.

    Without me having to scour all the discussions and threads, has there been any reports regarding the effect of the mbox to maildir conversion on integrated extensions?

    Specifically, I have many years of data in XNote++ and Lightning and therefore much dependency on them.

    Thanks!
     
  7. summerheat

    summerheat Registered Member

    Joined:
    May 16, 2015
    Posts:
    1,197
    Well, I'm not familiar with XNote++. But with all my 25 addons I haven't had any issues regarding maildir.
     
  8. summerheat

    summerheat Registered Member

    Joined:
    May 16, 2015
    Posts:
    1,197
    Just a short update: I've been running Thunderbird using the maildir format with 4 POP3 accounts, 1 IMAP account and a bunch of RSS feeds for more than 3 months. No problems at all, everything is working absolutely flawlessly, and Thunderbird is very snappy.

    So my recommendation is:
    1. If you set up Thunderbird anew, choose maildir from the beginning (step 2 in post #1).
    2. If you're using IMAP accounts, create a new profile, choose maildir, and synchronize with the mail servers. Only step 5 above is relevant of you want to import settings from your old profile.
    3. If you're using POP3 accounts, it's a bit more complicated if you want to import your old mails into the new profile. Follow the steps 1-5 above.
     
Loading...