SERIES: Security Now!
DATE: April 21, 2015
TITLE: Great Firewalls & Cannons
HOSTS: Steve Gibson & Leo Laporte
DESCRIPTION: Leo and I catch up with the most interesting and significant security and privacy news of the week. Then we take a close look at what's known of the mechanisms China has developed - both filtering and offensive weaponry - to provide for their censorship needs and to potentially attack external Internet targets.
SHOW TEASE: It's time for Security Now!. Steve Gibson's here. Yes, as always, the news ripped from the headlines of today's paper including the Great Firewall of China, and now the Great Cannon. How do they work? Steve explains all, next.
LEO LAPORTE: This is Security Now! with Steve Gibson, Episode 504, recorded Tuesday, April 21st, 2015: Great Firewalls and Cannons.
It's time for Security Now!, the show that protects you and your loved ones online. This guy is the Great Firewall of Gibson. He protects us all. And you know what I love about this show, you also listen and learn. He's a great teacher. Thank you for being here, Steve Gibson.
STEVE GIBSON: Hey, you know me, this is a pleasure. On the 500th episode I thanked everyone for their being here because they're always thanking me. So, and I know that this is a useful couple of hours that we spend every week.
LEO: Useful, fun, and enlightening.
LEO: And, by the way, thanks to Mike Elgan for filling in last week.
LEO: We were down at the, or up at the, or over at the NAB show.
STEVE: He did a great job with the Q&A.
STEVE: And it's funny because, when I don't have you, I'm always a little self-conscious of, like, that we'll be talking about things that you won't get so that - because this podcast, probably more than any other one…
LEO: You cannot miss an episode.
STEVE: It builds, yeah.
LEO: Yeah, yeah.
STEVE: You know, it really does build on the things we've covered in the past. How many times am I saying, oh, we talked about this three weeks ago, we talked about this four weeks ago, and now this is happening, blah blah blah. So I'm always thinking, oh, Leo won't know about this. Well, that's okay, you know. But you're rare - although, I was going to say, you're rarely gone. But you have, you are talking about trips coming up. So is that in the summertime sometime?
LEO: Yeah, just so you know, I'll be leaving June 27 through July 14th.
LEO: Lisa and I are going on a river cruise down the Rhine…
LEO: …the Main and the Danube.
STEVE: Well, and you need to refresh.
LEO: You've got to take a break. But I've got to say, because this is - you're the same way. It's our passion. I do nothing but read tech news day in, day out, wherever I am. So it's - few are the stories that I haven't been reading about and cognizant of. What frequently happens, though, and I know I'm not alone in this, is I read a story, and I say, I can't wait to hear what Steve has to say about this.
STEVE: Yeah. I'm seeing people tweeting that a lot, too. They'll say, oh, this looks bad, but we'll have to wait and see what Security Now! says. So, because a lot of this stuff does beg some interpretation. We've got a really interesting, sort of a very brief follow-up on TrueCrypt audit that we talked about two weeks ago. That was the topic of our show two weeks ago. Everyone's talking about a nasty IIS bug that was part of the Patch Tuesday, last Tuesday's Patch Tuesday, which is bad because it is just so easy to do. We'll talk about that.
I found an interesting, sort of from a privacy standpoint, new service that Google is offering that I want to talk about. Also some changes in Chrome. A popular library in iOS that has some people worried. News of Let's Encrypt, which is the automated server is able to issue its own SSL certs service, which is moving forward.
And then our big topic for the day, the title of the show, is China's Great Firewall and Great Cannon. There's been, in the wake of the cannon blasting that they were doing of GitHub, like mid-March, now we understand much more about the architecture of the system. We've never really addressed the Great Firewall at all. I understand exactly how it works now. And we did talk about the Great Cannon. But I want to tie that all together because this is a major actor on the Internet, not only choosing to censor the content that gets into China, but now, as we know, actively using other people's machines for cyberwarfare.
And it's just, I mean, I feel like it's science fiction, but it actually happened a few weeks ago. And it was embarrassingly public. China never took any responsibility for it. They sort of deflected the questions. We now have hard evidence that it was a state-sponsored attack, intimately tied to the Great Firewall, which we know is a state-sponsored Internet filter. So lots of new information that I think will be interesting to share with our audience this week.
LEO: Very, very exciting.
STEVE: So the picture of the week on the front page of our show notes is a pop-up that I got when I began to experiment with a new feature that Google has made available, that we'll be talking about in a minute. But it was big, so I stuck it up there on the front page, and we'll come back to it.
I wanted to quickly do a little bit of a follow-up to the TrueCrypt audit coverage from two weeks ago because I spotted something on the 'Net from the - Tom, he would be upset if I called him the “lead cryptographer.” He was the team lead or the managerial lead. And Tom Ritter is a cryptographer. He was one of the three who were part of the TrueCrypt audit. And I found a quote by Tom saying, “The audits of TrueCrypt get a lot of press because it's something flashy. But the development effort that went into TrueCrypt at the beginning are immense and incredible. And the developers don't get as much credit as they should for producing a disk and volume encryption project for multiple platforms and for maintaining it for a decade or more. There are successor projects, and they are improving it in their own ways. I'm excited to see those projects grow and thrive and last as long as TrueCrypt did. I still use TrueCrypt,” says Tom Ritter, who audited it and is a cryptographer.
LEO: Now, that's high praise.
STEVE: That was my point. “I still use TrueCrypt,” he says, “and want to see it supported in the future.” And I have looked at, what is it, VeraCrypt, and can't think of the other one right off the top of my head. They're making a point of being forward compatible. That is, being able to take an existing TrueCrypt volume, presumably, and still be able to interpret it. Although, for example, they're improving some ciphers. No doubt they're keeping the older legacy ones around in order to be able to still function with an existing TrueCrypt container.
But I just thought it was interesting that Tom Ritter, who would know better, says, “I still use TrueCrypt and want to see it supported in the future.” So, and for what it's worth, I'm using it because it still solves the problem. And I will be forced away from it when at some point it turns out that it can't handle GPT partition types or something like that, where some compatibility gotcha occurs, just because of TrueCrypt's age.
LEO: And that's fairly soon; right? I mean, in some cases, I think, yeah.
STEVE: Yes, yes. Yeah. And so I'm glad that it is being moved forward. And as I said, I'm so glad for the audit because it spotted a few areas, they weren't critical, but might as well fix them in the code that is now alive and fixable. And I guess my point was that I'm always going to feel more comfortable with a third-party solution. I just, you know, using BitLocker from Microsoft is like, you know, they're not going to show us everything that it's doing. And it's hard to trust that there isn't some way that they could honor a demand for a drive to be decrypted. And I only, I mean, the reason for using it is that it really, really is safe. So the industry…
LEO: And where can we get it, by the way? I think it's worthy of a note there. Because you're preserving the - because you can't just go to TrueCrypt.org anymore.
STEVE: No. I grabbed the 7.1a, all the archives and resources at that time. So it is at GRC.com. I think if you just put, like, TrueCrypt into Google, I was, for a while at least, GRC was one of the links that came up pretty quickly in a Google search. So, and a lot of people are visiting that page and grabbing the content from me. And I think it might have been, I don't remember if it was - somebody was also maintaining the hashes on a separate site. So, oh, yeah, there it is, GRC, TrueCrypt, the final release archive.
LEO: Now, if you get it from - if you get 7.2 from the TrueCrypt site, don't.
STEVE: Ah, see, that's the one that they deliberately neutered. This is why we knew it wasn't something done overnight because 7.2 only removes TrueCrypt. You can't use it to install TrueCrypt. And that's a nontrivial change. They had to go through all of the code and, like, remove all of the aspects of it that were about creating a new volume. Turns out there was a lot there that they had to do. So 7.2, it will interpret a TrueCrypt volume, but it will not create one, and it will allow you to remove it. It'll remove, it'll decrypt a drive for you. So that's the only thing that they're now making available.
LEO: I mean, obviously Tom - is that his name?
STEVE: Yeah, Tom Ritter.
LEO: Is not using VeraCrypt. But do you recommend that people go to your site, download the final release of TrueCrypt? Or would you recommend they use VeraCrypt?
STEVE: I guess my sense is, I mean, I'm sure VeraCrypt is fine. But TrueCrypt is bulletproof. And my recommendation would be, if it works - but you know me. I'm still sitting - I'm sitting in front of XP, Leo, so I'm really - I'm a bit of a skewed sample. But my sense is, if it works, use that one. It's been really just pounded to death by hundreds of thousands of people, that is, the original TrueCrypt 7.1a. But at some point it won't work for you because your system will no longer be compatible, in which case you'll want to go explore the forks of TrueCrypt. That would be my strategy, I think. There's nothing that they have done that makes theirs superior to what TrueCrypt was because there was just nothing wrong with it the way it was.
LEO: Yeah, yeah. And it has been vetted. And it has been audited.
LEO: Which we can't say for anything else. So there's that, too.
STEVE: Yes. And it is so easy to make a mistake. In fact, that is our next story because this bug was not in Windows Server 2003. It apparently got introduced in Windows 7 and the same codebase, which is Windows Server 2008, and 8.8 and 8.1 and Windows Server 2012 and the core installation. Anyway, this is the bug that the security industry has been abuzz about, mainly because it is horribly easy to do.
Microsoft gets in trouble always when they move stuff from the user space into the kernel. How many times have we run across JPGs or image files that are able to take over your computer? It's crazy to have an image file able to execute code on your machine. But they famously moved GDI, the Graphics Device Interface, from user space down into the kernel, at a time when they were desperate for more performance. And the so-called “ring transition,” when an application needs to call an operating system service, the application has limited privileges, and it needs to connect to code in the kernel that has infinite privileges. I mean, god. I mean, literally, it can read any address it wants to. It can do anything. The kernel is maximally privileged.
It turns out that the architecture to make that transition in a carefully controlled, secure fashion takes some time. There's lots of caches and buffers that need to get flushed and switched. And so every single time the code in the user space has a request for anything from the operating system, there's some overhead. So when you're desperate for performance, as Microsoft has been at various stages during the history of Windows, one of the things you look at is, ooh, you know, if instead of having code in the user space, which is making lots of calls into the kernel, it's so tempting to move that code itself into the kernel because then there's no more transitions. You'd still have to talk to that from the user space.
But, for example, the user might say, draw a rectangle at these coordinates. And so that command goes one transition into the kernel. Then the graphics device interface code is able to just go bzzz, you know, and do it much faster. All of the individual colors and pens and corners and roundings and shapings and mergings, all the complicated variations can be done far faster than if it was living with the application in the application space and had to keep begging the kernel to do all these little things for it. So Microsoft did that when they moved GDI in the kernel.
They faced another temptation, which they succumbed to, when they moved a chunk of IIS, the server, into the kernel. And they did it in a module, a kernel driver called http.sys. So it is, basically, they looked at a way that they could cut the server in two so that, with a minimal amount of relocation, that is, moving the smallest amount that made sense into the kernel, they could get the most bang for the buck. And so what they did was they created in Windows 7, which corresponds to Windows Server 2008 and all the subsequent OSes, because this has been there, they allow the kernel to parse the HTTP query enough to see whether the item being requested is in the cache. And they also put the cache in the kernel.
So the performance gain is potentially dramatic because we know that so many websites are fulfilling requests from cached resources, like all the images, the same images that they're serving, not just over and over to the same browser, but to all the different people who come whose browser needs to get at least one copy of the image. Well, if it's already been found in the file system and pulled down into memory, and it's sitting in the kernel, there's all kinds of optimizations you can do. Windows can just sort of point some pointers at it, and it just shoots that asset back to the requester.
Well, it turns out they made a mistake, which is pretty simple, but of course all of these mistakes are simple in retrospect. And on the show notes here I show the piece of code which is incorrect. Basically, one of the request headers that a browser or an HTTP client can include is called “range.” If you don't specify a range, then the server assumes you just want the whole thing. But you can also say, for whatever reason, maybe you're actually running four connections in parallel, and each connection is getting a different piece of something large, and you're doing that, you know, that's what the download accelerators did for years. For whatever reason, it's in the HTTP spec. And that allows the browser to say, okay, I want this, but start at byte 702 up through 5,027. You can just arbitrarily say this is the first byte of the object I want; this is the last byte.
Well, if you have a first and last byte, one of the things you need to do is determine how many bytes that is. The way you do that is you subtract the first, the beginning offset of the range, from the ending offset. That gives you the difference. But this is one of the tricks in programming. The difference is not the number because say that you wanted - you said I want to start with byte two, and I want to end with byte three. Well, if you subtract two from three, you get one. Except that the range is inclusive, so you actually need two. So you subtract the smaller, the beginning from the ending, and add one.
And in that code that you've got on the screen now, Leo, from the show notes, that is exactly what they're doing. They have two 64-bit values. And this is 32-bit code because, for example, the EAX register and EDI, those are 32-bit registers. But the range, but 32 bits only gives you 4.3GB, as we know, and things can be bigger than that. And so IIS is treating them as 64-bit sizes. And so that first subtraction is subtracting the low 32-bits of each of the ends of the range. And then that second one, the SBB, that's subtract with borrow. In case there was a borrow from the first, the result of the lower side subtraction, that then subtracts an extra one from the high side subtraction.
And so once they're done, then you can see they add one to EAX, and then they add with carry a zero to ECX. That's because, if the add of one to EAX does overflow, EAX goes back around to zero, it'll leave the carry bit set. And so when you add with carry a zero to EAX, that essentially adds that one that fell off the high end to ECX. So what they missed, and you can see it right there, no test to see if that add of ECX overflows. They didn't check for overflow.
So it turns out that in practice, simply using a crazy range header, and I've showed here in the show notes, pretty much anything, and then as the starting location, and then this special number, 18446744073709551615, that's the maximum value that a 64-bit quantity, unsigned quantity, can contain. And so I'm sure this code was written in C. And so in C they were subtracting a long long, which was the starting point of the range, from a long long, which is a 64-bit quantity, from the ending point, and adding one. And they didn't check for overflow. And as a consequence, anyone making a request of IIS with that range line will crash the server. It just BSODs.
Microsoft called it a remote code execution exploit. Nobody has figured out yet how to actually make that execute code. There are some people who have figured out how to get some information disclosure. Apparently ranges can also have multiple components. It's crazy. You can say, like, I want from two to five, and then 27 to 46. And so you can create, like, compound ranges. And it turns out that people who've been playing with this have been able to get the server to send them chunks of memory beyond the object that they are requesting. So that is definitely information leakage, and we don't yet know how bad it is. But turns out it's much easier to crash the server than it is to get it to give you additional information. Nobody's figured out how to make it execute. Maybe if you had - I can't even imagine how you could do that from what we know.
LEO: But we've talked about this before. The first step is always a crash, which can be used as a denial of service. Second step is to see if you can get it to jump somewhere and execute some arbitrary code.
LEO: And that takes a while. But the fact that you can get it to crash is always a very, very, very bad sign. It's interesting - so we were showing assembly language. It was written in C, most likely.
LEO: It did say, I noticed the subroutine was @32.
STEVE: Yes. Actually, that says UI, a pointer to parse range, and then it says @32 [[email protected]]. That's a notation for the number of bytes of arguments. So whatever this parse range routine…
LEO: The disassembler gave you or something.
STEVE: Yes. Yeah. So it's a clue to say, essentially, once you're done, pop 32 bytes off of the stack in order to clean up the stack because the caller pushes the bytes on the stack, and so the routine itself cleans up the stack afterwards. So that doesn't refer to the bitness of it. For example, it wouldn't say 64 if it were 64-bit code. It's how many parameters were required by that subroutine.
LEO: It's interesting. Whoever did the disassembly must have had access to a symbol table because usually you don't get real symbol names, real routine names.
STEVE: That's true, although the developer tools do have the debugging versions of the code.
LEO: Oh, okay. There you go.
STEVE: And so you are able to figure out what all that stuff is. And the way Windows works with dynamic link libraries, they sometimes need that also. I mean, they link by name. And so the DLLs will have the names of the routines in order for the linker to glue it all together. So there is some of this inefficiency. Netcraft estimates that there are about 70, seven zero, million publicly exposed IIS servers on the Internet. Apache and Nginx have way surpassed IIS, IIS basically sort of losing the war for dominance. Not that it ever had it.
LEO: But you still use IIS; right?
STEVE: I do because I'm a Windows programmer, and I know Windows, and I started. If I were doing it, if I were starting it today, I would never consider using IIS. But I have a huge investment in server-side code. And I've gotten to know it, and I can't quite say that I've gotten to love it because Microsoft keeps going this to us. It turns out that I would have been vulnerable except that one of the workarounds is to disable that kernel caching. And I generate so much content dynamically from my server-side code that I never had kernel-caching on. And so I was never vulnerable to this, just, you know, happily. Otherwise I would have immediately patched.
So anyway, this is just, you know, this is a perfect example of a problem that Microsoft would not have had, had they not moved what arguably should really stay up in the user land down into the kernel, purely for the sake of performance. They're under pressure to make IIS competitive, and the architecture that they've got necessitates that they do everything that they can. And putting more of your web server in the kernel, horrible as that is, is a way to achieve that. And the risk is a little tiny mistake like this can really ruin your whole day. And what's happening is people are getting - oh, this is, by the way, when this was released, Microsoft said there are no known attacks. It's been completely reverse engineered, and it is being scanned for actively on the 'Net, and IIS servers are crashing constantly because they have not been patched. And the default is to cache content in the kernel for the sake of performance.
LEO: Although crashing the server usually means just it reboots, restarts, you know.
STEVE: Right. Right. And that's why it's called a “denial of service,” because you're denying users service while it gets itself going again.
LEO: Right, right.
STEVE: Now, this is interesting. Google is allowing us, anyone who has a relationship with Google, to download their search history. Back, I'm not sure how far back it goes. In my case, because I was curious about this, you go to, guess it's Google.com/history. Or the link I have is history.google.com/history. And there's a nice little page that shows some stats about you. And if you click on the little gear icon in the upper right, there's the download option. And you can have Google prepare a ZIP file containing every single Google search you have made, in JSON format, for - in my case it goes back to January of 2012. So I'm looking here, and I just stuck it in the show notes here, so quarterly summaries of every search I've made through 2012, 2013, 2014, and up to now in 2015. They caution you that there's a lot of potentially sensitive personal information here. I mean, this is everything I've searched Google for.
LEO: I don't think this is new.
STEVE: This isn't. Okay.
LEO: So Google, well, and I'm sure some tech blog discovered this and said it's new. But Google's always had a site called “Google Takeout,” where you can download everything.
STEVE: Ah, yes. And that's - okay.
LEO: And you've always been able to do that. And you get to choose what you want to have in here. I don't think search history is new. There's lots more than search history, including your location history. Every once in a while somebody clicks a link - Google's had this for a long time, Google.com/dashboard - and suddenly realizes, oh, my god, they're keeping track of everything. And Google's always offered Google Takeout. They've always been upfront about this. I don't think this is new, unless I'm missing something.
STEVE: And I don't know. I mean, I can't…
LEO: Yeah, some tech blog published it. I saw it all of a sudden all over the place. Google does this. It's like, yeah.
STEVE: Anyway, from my standpoint…
LEO: It's good to know.
STEVE: I just thought it was cool…
STEVE: …that, I mean, it's sort of a blast through the past when you browse through, like, what you were searching for three and a half years ago. It's like, oh, yeah, I remember that. Actually, I have some tabs that are that old, but that's another story.
LEO: I bet you do. You can clear this. You don't have to let Google save it. Most of us do because it gives us some value in Google Now and other things. And Google tailors its searches to previous searches. But Google's always been very clear about this. At Google.com/dashboard, you can see what they take, what they keep track of. And the one that scares people more often is the location history because, if you have a smartphone with Android on it, they've been keeping track of where you are, you know, unless you turned that off, for a long time. This is all in here. Even my Orkut postings are still here.
STEVE: So speaking of Google and Chrome, they are moving forward in their never-ending march to tighten things up. And with Chrome 42, which is out now, they have finally disabled what's called the NPAPI. They gave plenty of ample warning, notice that they were going to do this. The NPAPI is the Netscape Plugin Application Programming Interface that dates back to the '90s. And I think, like, Navigator 2.0 is where this first happened. Essentially, Netscape realized that their browser would be able to render HTML pages and JPGs and GIFs, and I don't think there were PNGs back then, but sort of the standard resources. But there would be other content, for example, a PDF is a perfect example. How could a browser render a PDF in its UI? How could it contain it?
So what you need is you need to create a standardized, so-called API, Application Programming Interface, that other developers can use to talk to the browser guts in order so that they just have to handle, for example, if we extend this PDF notion, they would create a PDF rendering plugin which you would then add to Netscape. You register it with the browser. And then when you request something with a content type of whatever it is, like application/pdf, Netscape would look to see if it has anything registered to handle that, if there's a plugin that says, oh, yeah, I know how to render those. In which case it would hand control to the plugin, and the plugin would then get access to the physical screen real estate in order to render what it wants to.
So this has been - it was very widely supported. Even IE briefly added support, although they dropped it a long time ago, back with v5.5. So IE has not had support for this Netscape Plugin API. However, Safari and Firefox still both do and continue to support it. Chrome, the Chromium people were just, I don't know, their claim, for what it's worth, is that it was causing stability problems. It's interesting that Java and Silverlight are both hosted under this Netscape Plugin API. So if Java is going to continue to be supported in Chrome, it's going to have to probably change.
Chrome has its own API that they call “Pepper.” It's PPAPI, which is sort of essentially the same thing. And, for example, that's the API that hosts Chrome's built-in support for Flash. So the Flash that's in Chrome, it's sitting in this Pepper container using this PPAPI. So if Microsoft or Oracle chose to move their Java and Silverlight over, then they could do that. But as of this version of Chrome, 42, it's now disabled by default. So it's not gone, it won't actually be gone until Chrome 45, which is targeted for September of this year. So there's about another six months. At the moment, you may find that stuff you're using will not function. But you can turn that back on.