Thunderbird: From mbox to maildir

Thunderbird has always been my favourite email application. I've tried a few other ones but always went back because no other one is so flexible and extensible, IMHO. (And Outlook and the likes are out of question as I'm running Linux.)

One disadvantage of Thunderbird is that it is still using the mbox file format by default. Basically this means that in each folder all mails are comprised in one big text file. On the other hand, when using the maildir format each mail is kept in a separate file.

The advantages/disadvantages of both formats are discussed here and in this thread (obviously started by Wilders member @WildByDesign ). In my opinion, the most important advantages of maildir over mbox are the following:

1. The maildir format is much more robust. The risk of losing data is much smaller if Thunderbird or your computer crashes.
2. Incremental backups are much faster as only the new and modified files have to be backupped.
3. Scanning your mails with an AV scanner is easier, and infected mails can be removed without the risk of data loss.

Those advantages are particularly important if you have a large database of emails going back many years.

This Mozilla page says that maildir is still disabled by default because there are still many bugs. But hey - I've been running Thunderbird with maildir for several weeks and five mail accounts and haven't run into a single problem! On the contrary, it's my impression that Thunderbird is snappier than ever before.

Besides, several of those bugs in that list refer to problems which arise when mixing mbox and maildir. Don't do that! This is what I've done and suggest:

1. Start Thunderbird with the -p switch (thunderbird -p) and create a new profile. So you can always go back to your old profile if you run into problems! This is really a very important step!
   
2. Start TB with this new profile. Before doing anything else go to
   Options - Advanced - Advanced Configuration - Message Store Type for new accounts and select the maildir format.
3. Create your email accounts. If you're using IMAP just retrieve the mails anew from the server and you're done.
   
4. If you're using POP3 (which is what I do for various reasons) it's more complicated: I suggest in this case that you start TB with your old mbox profile and install the ImportExportTools addon. Then go to each folder, right-click it and select Export all messages in the folder > EML format. Then close Thunderbird, execute thunderbird -p again and start your new maildir profile. Install ImportExportTools, create the (sub)folders of your old profile manually, right-click each, select Import all messages from a directory and choose the corresponding folder where you exported the messages earlier. Just in case that TB already retrieved messages from the mail servers and you have duplicates, I suggest to install Remove Duplicate Messages and remove them. In order to avoid duplicates from the beginning I would after creating the account(s) unselect Check for new messages at startup and Check for new messages every ... minutes in the account settings. Close Thunderbird and copy the correct popstate.dat files from the old profile. If all is well, you can enable those settings again.
   
5. In order to import your data (filters, address books, etc.) to the new profile go to Transferring data to a new profile - Thunderbird, and meticulously follow the steps there.

 

@summerheat Excellent topic. There did not end up being much discussion on it over at DSLReports forum, so hopefully it will have more attention here. But anyway, I am still 100% rocking the maildir mail store within my own Thunderbird setup since before I created that thread over there. I've got my main POP3 account, Local Folders (though not used), and a pretty extensive Blog & News Feeds (for all my RSS needs) all setup within maildir and it is lightning fast. No need to compress large mail files anymore. You delete an email, that single file simply gets deleted. No bloat, no mess.

I have not followed up with that GSoC effort that was intended to create a mbox>maildir conversion utility, but your new thread here now has reminded me to look into that further and see how far along that effort is.

 

I don't know, either, if there is any progress with that utility. On the other hand I would always create a new profile when switching to maildir. So I don't know if such a utility would yield that much - unless it would be able to convert a complete profile with maintaining the complete folder structure.

 

Here is the bug report (mbox>maildir converter and vice versa) with lots of recent activity: https://bugzilla.mozilla.org/show_bug.cgi?id=856087
Seems it will be built-in instead of as an extension. Questionable whether they will land this in time for stable 52 release, though they are discussing hiding behind a pref initially.

 

Thanks, that's good to see! Too late for me, though

 

I have to admit, this discussion is the first I've noticed re mbox and maildir in TB.

I've been using TB since v0.6 for my personal stuff. With occasional support under some SOHO support contracts.

Several years ago having ended the professional need (mine and enterprise admin) for Outlook, I migrated years of email from the PST.

My Mail folder will soon exceed 900MB.

The profile folder is backed up daily but I've never had to recover from a TB failure.

Nonetheless, I recognize the likely advantages maildir has to offer.

Without me having to scour all the discussions and threads, has there been any reports regarding the effect of the mbox to maildir conversion on integrated extensions?

Specifically, I have many years of data in XNote++ and Lightning and therefore much dependency on them.

Thanks!

 

Well, I'm not familiar with XNote++. But with all my 25 addons I haven't had any issues regarding maildir.

 

Just a short update: I've been running Thunderbird using the maildir format with 4 POP3 accounts, 1 IMAP account and a bunch of RSS feeds for more than 3 months. No problems at all, everything is working absolutely flawlessly, and Thunderbird is very snappy.

So my recommendation is:
1. If you set up Thunderbird anew, choose maildir from the beginning (step 2 in post #1).
2. If you're using IMAP accounts, create a new profile, choose maildir, and synchronize with the mail servers. Only step 5 above is relevant of you want to import settings from your old profile.
3. If you're using POP3 accounts, it's a bit more complicated if you want to import your old mails into the new profile. Follow the steps 1-5 above.

 

Thanks for this. When I did the export, I got the expected eml files. When I did the import I got files that have no extension. I expected to get files that had an eml extension. So then I tried another way: I created a new account in my "mbox" profile after changing the "Message Store Type for new accounts" setting. Then I dragged and dropped files from a pre-existing folder into a new folder in the new account. Same result: a bunch of files with no extension.

Does that (a bunch of files with no extension) match your experience?

This KB article says "Thunderbird 38 introduced optional support for Maildir, which allows a single unique file per email (using the EML file extension) ..." I guess the "(using the EML file extension)" detail is not correct.

 

I'm sorry - but I can't remember all details as I made the switch to maildir more than one and a half years ago.

However, one question: Did you make sure - as I wrote in the first post - that you did not import single files but rather selected Import all messages from a directory and chose the corresponding folder where you exported the messages earlier? AFAIR, that's a crucial step.

 

This is correct about no file extensions. Each single email is stored in it's own individual file. So you could have dozens or hundreds of individual files with no extensions. Each individual mail folder should contain two folders; cur and tmp.

The .EML extension reminds me more of individual email exports. For example, if you wanted to save/export a single email to another location on disk it would generally be saved in that format. So that is not related to maildir at all.

 

Yes, I did "Import all messages from a directory". If you look in your profile's mail folder, do you see .eml files or do you see files with no extension?

 

Thanks. I appreciate the confirmation.

 

You're welcome.

 

Ah, sorry, I misunderstood your question. As @WildByDesign already mentioned, those files are without an extension.