Google Is Preparing to Spank Symantec

And it is about time.

Note: Historically, the Google Chrome team has not used the Blink Process for Certificate Authority-related security issues, of which there have been a number over the years. However, we are interested in exploring using this process for such changes, as it provides a greater degree of transparency and public participation. Based on the level of participation and feedback we receive, we may consider using this for the future. However, as CA-related security incidents may require immediate response to protect users, this should not be seen as a guarantee that this process can be used in future incident responses.

Primary eng (and PM) emails:
[email protected] [email protected]

Since January 19, the Google Chrome team has been investigating a series of failures by Symantec Corporation to properly validate certificates. Over the course of this investigation, the explanations provided by Symantec have revealed a continually increasing scope of misissuance with each set of questions from members of the Google Chrome team; an initial set of reportedly 127 certificates has expanded to include at least 30,000 certificates, issued over a period spanning several years. This is also coupled with a series of failures following the previous set of misissued certificates from Symantec, causing us to no longer have confidence in the certificate issuance policies and practices of Symantec over the past several years. To restore confidence and security of our users, we propose the following steps:

  • A reduction in the accepted validity period of newly issued Symantec-issued certificates to nine months or less, in order to minimize any impact to Google Chrome users from any further misissuances that may arise.
  • An incremental distrust, spanning a series of Google Chrome releases, of all currently-trusted Symantec-issued certificates, requiring they be revalidated and replaced.
  • Removal of recognition of the Extended Validation status of Symantec issued certificates, until such a time as the community can be assured in the policies and practices of Symantec, but no sooner than one year.

As captured in Chrome’s Root Certificate Policy, root certificate authorities are expected to perform a number of critical functions commensurate with the trust granted to them. This includes properly ensuring that domain control validation is performed for server certificates, to audit logs frequently for evidence of unauthorized issuance, and to protect their infrastructure in order to minimize the ability for the issuance of fraudulent certs.

On the basis of the details publicly provided by Symantec, we do not believe that they have properly upheld these principles, and as such, have created significant risk for Google Chrome users. Symantec allowed at least four parties access to their infrastructure in a way to cause certificate issuance, did not sufficiently oversee these capabilities as required and expected, and when presented with evidence of these organizations’ failure to abide to the appropriate standard of care, failed to disclose such information in a timely manner or to identify the significance of the issues reported to them.

These issues, and the corresponding failure of appropriate oversight, spanned a period of several years, and were trivially identifiable from the information publicly available or that Symantec shared.

The full disclosure of these issues has taken more than a month. Symantec has failed to provide timely updates to the community regarding these issues. Despite having knowledge of these issues, Symantec has repeatedly failed to proactively disclose them.  Further, even after issues have become public, Symantec failed to provide the information that the community required to  assess the significance of these issues until they had been specifically questioned. The proposed remediation steps offered by Symantec have involved relying on known-problematic information or using practices insufficient to provide the level of assurance required under the Baseline Requirements and expected by the Chrome Root CA Policy.

In January 2015, Symantec-issued certificates represented more than 30% of the valid certificates by volume. While changes in the CA ecosystem have seen that share decrease over the past two years, there is still a significant compatibility risk for an immediate and complete distrust. Further, due to overall TLS ecosystem concerns, we understand that it may take non-trivial effort for some site operators to find suitable solutions, as the need to support older devices may necessitate the use of particular CAs, meaning that distrust of new certificates also has significant compatibility risk.

To balance the compatibility risks versus the security risks, we propose a gradual distrust of all existing Symantec-issued certificates, requiring that they be replaced over time with new, fully revalidated certificates, compliant with the current Baseline Requirements. This will be accomplished by gradually decreasing the ‘maximum age’ of Symantec-issued certificates over a series of releases, distrusting certificates whose validity period (the difference of notBefore to notAfter) exceeds the specified maximum.

The proposed schedule is as follows:
Chrome 59 (Dev, Beta, Stable): 33 months validity (1023 days)
Chrome 60 (Dev, Beta, Stable): 27 months validity (837 days)
Chrome 61 (Dev, Beta, Stable): 21 months validity (651 days)
Chrome 62 (Dev, Beta, Stable): 15 months validity (465 days)
Chrome 63 (Dev, Beta): 9 months validity (279 days)
Chrome 63 (Stable): 15 months validity (465 days)
Chrome 64 (Dev, Beta, Stable): 9 months validity (279 days)

The proposed schedule attempts to avoid making changes in Chrome 63 Stable, as that would be released during the winter holiday production freeze many organizations undergo. This is solely to reduce disruption for site operators and users, and attempts to resume with Chrome 64 following the holiday season. Further, the practical impact of the changes in Chrome 59 and 60 are relatively minimal, due to many of the certificates issued during that period of time being issued using SHA-1, which is no longer supported for certificates in Chrome.

In addition, we propose to require that all newly-issued certificates must have validity periods of no greater than 9 months (279 days) in order to be trusted in Google Chrome, effective Chrome 61. This ensures that the risk of any further misissuance is, at most, limited to nine months, and more importantly, that if any further action is warranted or necessary, that the entire ecosystem can migrate within that time period, thus minimizing the risk of further compatibility issues.

By combining these two steps, we can ensure that the level of assurance in Symantec-issued certificates is able to match what is expected by Google Chrome and the ecosystem, and that the risks posed both from past and possible future misissuance is minimized as much as possible.

Given the nature of these issues, and the multiple failures of Symantec to ensure that the level of assurance provided by their certificates meets the requirements of the Baseline Requirements or Extended Validation Guidelines, we no longer have the confidence necessary in order to grant Symantec-issued certificates the “Extended Validation” status. As documented with both the current and past misissuance, Symantec failed to ensure that the organizational attributes, displayed within the address bar for such certificates, meet the level of quality and validation required for such display. Therefore, we propose to remove such indicators, effective immediately, until Symantec is able to demonstrate the level of sustained compliance necessary to grant such trust, which will be a period no less than a year. After such time has passed, we will consider requests from Symantec to re-evaluate this position, in collaboration with the broader Chromium community.

Compatibility and Interoperability Risk
As with any reduction in trust in a Certificate Authority, this poses a non-trivial degree of compatibility risk. This is because site operators desire to have their certificates recognized in all client browsers, and if one or more browsers fail to trust a given CA, this is prevented from happening.

On the other hand, all site operators expect that certificates will only be issued for their domains upon their request, and the failure to have that assurance significantly undermines the security of HTTPS for both site operators and users.

This compatibility risk is especially high for Symantec-issued certificates, due to their acquisition of some of the first CAs, such as Thawte, Verisign, and Equifax, which are some of the most widely supported CAs. Distrusting such CAs creates further difficulty for providing secure connections to both old and new devices alike, due to the need to ensure the CA a site operator uses is recognized across these devices.

Further, the immediate distrust of a CA, as has been necessary in the past, can significantly impact both site operators and users. Site operators are forced to acquire certificates from other CAs, without having the opportunity and time to research which CAs best meet their needs, and users encounter a substantial number of errors until those site operators act, conditioning them to ignore security warnings. In the event that only a single browser distrusts such a CA, the error is often seen as the browser’s fault, despite it being a failure of the CA to provide the necessary level of assurance, and the site operator to respond in a timely fashion.

Assessing the compatibility risk with both Edge and Safari is difficult, because neither Microsoft nor Apple communicate publicly about their changes in trust prior to enacting them.

While Mozilla conducts their discussions regarding Certificate Authorities in public, and were the first to be alerted of these latest issues, they have not yet begun discussion of the next steps to how best to protect their users. Our hope is that this proposal may be seen as one that appropriately balances the security and compatibility risks with the needs of site operators, browsers, and users, and we welcome all feedback.

Alternative implementation suggestion for web developers
This proposal allows for web developers to continue to use Symantec issued certificates, but will see their validity period reduced. This ensure that web developers are aware of the risk and potential of future distrust of Symantec-issued certificates, should additional misissuance events occur, while also allowing them the flexibility to continue using such certificates should it be necessary.

Usage information from UseCounter:
For a variety of non-technical reasons, we do not currently instrument the usage of CAs. Further, few public metrics exist for intersecting usage information with the validity period, since only certificates valid greater than nine months will be affected outside of their normal replacement cycle. From Mozilla Firefox’s Telemetry, we know that Symantec issued certificates are responsible for 42% of certificate validations. However, this number is not strictly an indicator for impact, as this number is biased towards counting certificates for heavily-trafficked sites, and whose issuance is fully automated and/or whose validity periods will be unaffected, thus significantly overstating impact. By phasing such changes in over a series of releases, we aim to minimize the impact any given release poses, while still continually making progress towards restoring the necessary level of security to ensure Symantec issued certificates are as trustworthy as certificates from other CAs.


Google Announces Open Source JPEG Encoder

Google recently released a new open source JPEG encoder called Geutzli, that promises to reduce the size of high quality JPEGs by up to 35 percent, but at a price–longer encoding times.

The visual quality of JPEG images is directly correlated to its multi-stage compression process: color space transform, discrete cosine transform, and quantization. Guetzli specifically targets the quantization stage in which the more visual quality loss is introduced, the smaller the resulting file. Guetzli strikes a balance between minimal loss and file size by employing a search algorithm that tries to overcome the difference between the psychovisual modeling of JPEG’s format, and Guetzli’s psychovisual model, which approximates color perception and visual masking in a more thorough and detailed way than what is achievable by simpler color transforms and the discrete cosine transform. However, while Guetzli creates smaller image file sizes, the tradeoff is that these search algorithms take significantly longer to create compressed images than currently available methods.

When Google says it takes “significantly longer to create compressed images than currently available methods,” they’re not exaggerating. John Gruber reported it took 8 minutes on a high-end Mac to compress a single iPhone camera image. Other reports on the Internet have suggested the process also consumes very large amounts of RAM while it is working its magic.


Google Flu Tracker Fail

New Scientist sums up a study finding that Google’s much-hyped flu tracker–where the company supposedly can predict flu outbreaks based simply volume of search queries for flu-related terms–doesn’t actually work very well.

The system has consistently overestimated flu-related visits over the past three years, and was especially inaccurate around the peak of flu season – when such data is most useful. In the 2012/2013 season, it predicted twice as many doctors’ visits as the US Centers for Disease Control and Prevention (CDC) eventually recorded. In 2011/2012 it overestimated by more than 50 per cent.

Interesting. The most surprising part of this outcome, however, is that lead author of the study is actually surprised that Google hasn’t fixed this,

The study’s lead author, David Lazer, of Northeastern University, says the fixes for Google’s problems are relatively simple – much like recalibrating weighing scales. “It’s a bit of a puzzle, because it really wouldn’t have taken that much work to substantially improve the performance of Google Flu Trends,” he says.

That wouldn’t surprise anyone who actually uses any Google products or services at all. There are dozens of simple, easy things that Google could do to improve any number of their products, including bugs that users have been complaining about for years., that Google chooses to ignore.

Why should it be any different with the flu? Clearly the problem with the flu tracker is the influenza virus’ unwillingness to conform to Google’s business practices rather than any actual defect in its software or approach to predicting flu cases.

Google Sucks…and It’s Going to Get Worse

One of the odd things about technology is that a good portion of users of a given company’s technology tend to become fanboys of that company. So when Apple bans a game that mocks its (and other tech companies) labor practices, you will find no shortage of defenders of that and even worse decisions or practices.

For some people, apparently, the technology they use and love becomes such a part of their identity that the thought that the company that makes the technology might do awful things is threatening and has to be countered at every opportunity.

A better way to think about our relationship to corporations who make the products we use everyday might go something like this: these companies suck. They don’t care about the same things their customers do (except, to the extent their customers are also worried about the company’s quarterly profits and stock price). Occasionally, the interests of customers and a corporation interests overlap, but when something changes the corporation will screw the customers over and not even bother to buy them dinner.

Take Google’s Android. Every phone I’ve ever owned has been an Android. When all is said in done, I prefer both the Android OS to Apple’s iOS, and I prefer Google’s business practices to Apple’s.

But I harbor no illusions that Google is anything but a ruthless profit maximizer whose “Don’t be evil” slogan is about as reliable as Tim Tebow’s deep pass threat.

AdBlock Plus

Google had no problem, for example, banishing AdBlock Plus from the Google Play store. Google’s justification for removing AdBlock Plus and other ad block apps was that they “interfere with or accesses another service or product in an unauthorized manner,” which is a violation of the Play Store’s Terms of Service.

What AdBlock Plus actually does is run in the background on Android as a proxy and removes ads. The nice thing is it removes ads both within web browsers as well as in-app ads.

So Google’s claim is that any software that lets a user install a proxy and to use that proxy to filter incoming data is a violation of the Google Play Store’s TOS. A clear-cut case where Google’s interests as an advertising sales company suddenly conflicts harshly with the interests of its customers.

The nice thing about Android is I can get AdBlock Plus up and running on my phone without needing to root/jailbreak it. Unlike Apple’s model for iOS, I can download AdBlock Plus directly and install it or turn to alternative app stores. Of course, it also knows that the average Android phone owner is probably not going to know how to do that or be willing to do that.

The Future

I suspect things like this are going to happen more frequently.

One of the things that Android fan boys have been celebrating is Android overtaking iOS in smart phone market share. This is a mistake–personally, I was hoping Android would continue to play second fiddle to iOS.

Many of the things that make Android great, including Google’s general handling of Android, are byproducts of Android needing to overcome the iOS juggernaut. Android is more open than iOS simply because Google had no other choice if it wanted to overcome Apple’s market lead.

As Google’s market lead becomes solidified and some of its decisions to open source Android start to bite back at Google, I suspect it will take Danny Sullivan’s advice and ultimately create a closed phone OS.

And then it will be time to look for other projects or companies where, for now, the interests of customers and the company coincide, such as the phone OSes being developed by Canonical and The Mozilla Foundation.