Proton explains the technical issues that caused the recent ProtonMail outages
Last week was a tough one for Proton users, the company experienced multiple outages spread across 3 days. Yesterday, Proton sent out an email to its users to explain the issues behind the disruption of its services.
I didn't think the issue was a big deal, as I never noticed any downtime, maybe because I'm in a different time zone than the users who had reported about the outage. That's why I didn't write about it here, because I didn't experience the issue first hand. Some users wrote that they couldn't access Proton VPN and ProtonMail, while some could use the VPN.
When I saw new complaints about another outage the next day, I was quite surprised. Though I could still access ProtonMail, I noticed that the site was slower than usual. I thought it was trying to connect but failed, but it worked after several seconds. Interestingly, or worryingly, the issue recurred for a third consecutive day, and I reckoned this could be a major problem. I don't think I've ever seen an email provider experience so many outages in such a short time frame. You can check the logs at Protonstatus.com for more information.
Image via reddit
Many users were concerned about these outages, the biggest complaint was that they were unable to access their mails, receive new messages, or reply to them. There was another question that had been raised, has Proton been hacked or under attack? Would this result in loss of mails? The short answer is no, the company was experiencing some technical difficulties.
What caused the ProtonMail outage?
As you may know, Proton had been working on various improvements for its products. We just noticed the changes on the front-end, but it appears that the company had also been working on upgrading its infrastructure by adding new servers, and new software with better performance and higher storage capacity. Proton says that it upgraded its servers that were running its core database to a new version on July 10th.
The software had passed the quality checks without any hiccups, before the main database was upgraded. But, it turns out that the core database, which has higher activity than others, was overloaded after the upgrade. High user activity isn't something that could be simulated, which might explain the tests could not predict the outcome. The security system, which Proton had in place, noticed the high activity, and misinterpreted it as a threat. So, it activated some protective measures (on July 11th), which in turn caused a roadblock for users who were trying to log in to their Proton accounts, including users who had previously logged in on the web and on their mobile apps.
Proton states that it solved the problem by splitting the load of its database over multiple servers, and slowed down lower priority services like push notifications. The company had to add more servers, and eventually managed to restore the stability and performance of its systems. Proton has promised to publish more details about the outage on its blog this week.
I don't use Proton as my main email provider, so I wasn't really impacted by the outage. Some users may not be satisfied by the company's approach to fix the issues, after all it did take 3 days to sort out the problems completely. If I were a paying customer who relied on ProtonMail, I would have certainly been annoyed by the downtime, and not being able to get to my receipts, documents, etc.
That's why I think it's good it's important that Proton sent out the mail to users as an apology and explain what happened. It shows that they are willing to take responsibility for the issues that they inadvertently created. The transparency would be appreciated by users, but it is also a neat way by the company to reassure users that they weren't hacked, and that their data is safe.
Were you affected by the Proton outage?
I am getting
“This site has been reported as unsafe
Hosted by mail.proton.me
Microsoft recommends you don’t continue to this site. It has been reported to Microsoft for containing phishing threats which may try to steal personal or financial information.”
Can’t get in and because of that I can not log into my tumblr account
I still AM affected.
ios – OK;
Windows – OK
Android (phone) – completely buggered. Long connection time with some crashes.
Protomails new unified frontend was not a smooth sailing implementation far and wide, first the login experience was diminished, sometimes up to 20 minutes to get the login inputs to show up. This was a know internal issue and they finally solved it in the end.
Then the way this was all implemented, the JavaScript was developed via TypeScript which for your non techies out there compiled down to JavaScript, but they offload this to the client side which for slower platforms adds more time.
Then Due to the unified nature of the implementation, further overheads were introduced and things were quite optimized, so add to this more than a couple of users and you have a perfect storm.
I was affected by the outage and have not yet received any explanations or apologies, and tbh, they can use that time to actually do better with their backend and actually answer support tickets, which last I checked was more than 3 days. Though when answered, there was no passing the buck or excuses, just a professional experience where even despite issues you aren’t left feeling with some weird after taste.
Shame that Protonmail cooperates with corrupt foreign governments and provides private identifiable data on their users just because these users are activists fighting for a better world.
https://www.wired.com/story/protonmail-amends-policy-after-giving-up-activists-data/
https://proprivacy.com/privacy-news/protonmail-cooperation-with-swiss-authorities
Shameful and disgraceful, just goes to show fighting for something better will get you persecuted and harassed by Western Governments after all.
And this is why I use protonmail as my spam/bullcrap email for everything. The PREMIUM offer doesn’t look sooo good now anymore does it…? This is the equivalent of building a boat, made of sugar.
I use Proton as my primary email. But I was not aware they had an issue until I received their email explaining what happened (appreciate that they sent it and appear to be transparent). However I may not have noticed as I’m not a big email user, only occasionally receive and rarely ever send email. Even so I have never had an issue with their service (via the Android app or their bridge application to sync with Outlook) since I started using them about four years ago.
I don’t use Proton.
Ashwin states in his article that he wasn’t affected “[…] maybe because [he’s] in a different time zone than the users who had reported about the outage.”
Considering the technical explanation provided by Proton the fact that the outage is region-specific may seem odd at first view.
Anyway, a good place to check a url’s connection availability is [https://www.uptrends.com/tools/uptime], especially given the user has the choice to check form :
– All checkpoints
– Europe
– North America
– Asia
– Australia
– South America
– Middle East
In the case of Proton’s outage as well this would come in handy of course.
Proton must have sent explanations only to those who tried to access mail in that window of time. I never received any sort of notification of an outage, as I do with another email service I pay for.
I got the whole explanation in regard to the outage by email from Proton yesterday.
I was not affected. Well, nothing is infallible. Can happen. Not that bad at all.
Overall, I’m highest satisfied with their services they offer. CERN makes it happen. :)