Mozilla back in November 2012 improved Firefox's private browsing feature by adding per-window private browsing to the web browser. Firefox had to close down completely prior to that if a user launched the browser's private browsing mode. The change enabled users to launch private browsing windows next to normal windows, a feature that other browsers, Google Chrome or Opera for instance, had been offering for some time before the change.
The improvement makes Firefox's private browsing mode identical to that of Google Chrome but not to the Opera implementation. Opera users can create private browsing tabs in the same browser window, to mix regular websites with private browsing websites in the same window.
Mozilla in February added a per-tab private browsing option to Firefox for Android beta allowing users to switch between private and standard tabs in the same browsing session and browser window. It appears that the feature also made its way into desktop versions of Firefox even though it is not enabled natively in the browser UI.
The Firefox extension Private Tab adds a per-tab private browsing mode to Firefox that works as expected for the most part. You can create a new private tab with the click on Nightly and the selection of New Private Tab from the menu. This creates a new blank tab in the browser that is set to private browsing mode. This is indicated by a color change of the Firefox button when the private tab is active, and the site title displayed in the tab as it is underlined.
The left tab that is currently not active is a regular tab, the active tab is a private tab as indicated by the underline and the Nightly icon. You can also right-click existing tabs and make them private browsing tabs but I have not explored that option during tests.
You can add a toolbar button to Firefox if you prefer that. Simply click on Firefox > Options > Toolbar Layout and drag and drop the New Private Tab icon to the toolbar you want it to be available in. Keyboard ninjas can use the Ctrl-Alt-P shortcut to open a new private tab (as opposed to Ctrl-Shift-P for a private browsing window).
I did some preliminary testing and came to the conclusion that the per-tab private browsing works well for the most part. The only thing that did not work out - a bug I assume - was that favicons of sites that I opened in private tab mode where still saved to the disk cache. The other files of the website were on the other hand not saved to the cache.
Note that the extension works only in Firefox 20.0 or higher and SeaMonkey 2.17 or higher and not in previous versions.