SERIES: Security Now!
DATE: October 23, 2018
TITLE: Libssh's Big Whoopsie!
HOSTS: Steve Gibson & Leo Laporte
DESCRIPTION: This week a widely used embedded OS (FreeRTOS) is in the doghouse, as are at least eight D-Link routers which have serious problems, most of which D-Link has stated will never be patched. We look at five new problems in Drupal 7 and 8, two of which are rated critical; trouble with Live Networks RTSP streaming server; still more trouble with the now-infamous Windows 10 Build 1809 feature update; and a longstanding zero-day in the widely used and most popular plugin for jQuery. We then discuss what can only be described as an embarrassing mistake in the open source libssh library, concluding by examining a fun recent hack and posing its solution to our audience as our Security Now! Puzzler of the Week.
SHOW TEASE: It's time for Security Now!. Steve Gibson is here. And as usual it's a roundup of significant flaws, one in a free real-time operating system, another in a well-known router - actually many well-known routers - and an exploit that uses jQuery's File Upload plugin, plus a red alert for Drupal users. It's all coming up next on Security Now!.
LEO LAPORTE: This is Security Now! with Steve Gibson, Episode 686, recorded Tuesday, October 23rd, 2018: Libssh's Big Whoopsie!
It's time for Security Now!, the show where we cover your security and privacy and how things work online with Mr. Steven “Tiberius” Gibson. Hello, Steve.
STEVE GIBSON: Leo, great to be with you again. Trying to get my fingers in the proper Spock Vulcan salute. But I think it's…
LEO: It's not easy, is it.
STEVE: Yeah, I think I'm a little slow on the uptake today.
LEO: I can do it on my right hand. And I'm a lefty. I can't, for some reason, do it on my left hand.
STEVE: I think I've told the story before, but the best man at my wedding, he had way too much information on me, and I was terrified that he was going to embarrass me horribly…
LEO: Oh, yeah. That's his job.
STEVE: …when he got up to give a toast. And I said, “Gary, do not, do not embarrass me.” Anyway, so being the clever person that he is, he got some small red rubber bands because he needed them in order to get his fingers to do the salute unaided. And all he did was he stood up and he said, “Gibson told me I wasn't allowed to embarrass him. So I'll just say 'Live long and prosper.'”
LEO: Perfect, perfect.
STEVE: I said, “Thank you, thank you.” So we've got a great podcast. The title is “Lib” - or you say “lib,” I say “libe,” potato, potahto - “Libssh's Big Whoopsie!” This was just so fun that it immediately became the topic of the episode because there's a mistake in a fortunately not widely used - but still, I mean, it's around - SSH library, an open source library. But the mistake is just so, as they would say somewhere in the U.K., “gobsmacking,” that we just have to take a look at it because it's just too fun. Also, in covering another story that we will also conclude with, the hack was so fun that I thought, this would make a great Puzzler of the Week for our audience.
STEVE: So we haven't done this as often as we might.
LEO: I would love to do it every week. It's fun, yeah.
STEVE: I know. It just never occurs to me.
STEVE: But sometimes the setup is perfect. And so we will examine a really fun hack, and then I will propose to our audience to, in the intervening week, find a solution in your head. With everything that we've done on the podcast, all of the tools are available to our listeners who have been following along for a while. There is a - well, anyway, now I'm stepping on the story. But there's a super elegant solution. And it'll be fun to see, just this little self test, who can come up with it, so.
But in the meantime, we're going to talk about a mistake found in a widely used embedded OS, actually it's the number one embedded OS, FreeRTOS. RTOS stands for “real-time operating system,” free as in free. Anyway, they're in the doghouse. And interestingly, Amazon has picked up FreeRTOS and decided to run with it and support it and sort of be its benefactor. But I'm very impressed with the things that I've seen that Amazon has done. We'll talk about the problems and their solutions.
Also there are at least eight D-Link routers which have serious problems, most of which D-Link has stated will never be patched. And so there's some interesting takeaway lessons for that. We're also going to look at new problems in Drupal 7 and 8. And if you're using Drupal 7 or 8, and you didn't update in the last few days, go do it now. Stop listening. Hit pause.
There are five problems. Two of them are remote code execution, which affect virtually all of Drupal 7 and 8. So this is another bad security problem that has hit Drupal. We talked about them not too long ago because I remember I was - you guys use Drupal, Leo, and your guys were already up on the news and had fixed it before we even got to it on the podcast.
LEO: Oh, yeah. Oh, yeah. We have a good team on that, yeah.
STEVE: There's also trouble with Live Networks RTSP streaming server that I'll just mention as sort of a public service for those who might be using it. Still more trouble with the now infamous Windows 10 Build 1809 feature update which we need to touch on. There's a longstanding zero-day in the widely used, most popular plugin for jQuery. And it looks like it's for eight years this thing has been there. And it's not even unknown to bad guys. Somehow the security industry missed it. Anyway, we'll talk about that.
And then we're going to talk about what can only be described as a consequence, I mean, it's where I got the title “Libssh's Big Whoopsie.” It's just it's too wonderful a mistake. So I think - oh, and we have a great Picture of the Week. So I think another great podcast for our listeners.
LEO: Busy, busy, busy.
STEVE: This number 686th podcast.
LEO: And I am not complaining that we don't have puzzlers every week because you put so much - I don't know how much people realize how much work you put in. If you've ever read the show notes, you know that this is a book Steve writes every week. So having a puzzler in addition to all the other stuff you do, I don't know, that's asking an awful lot. So we're happy, Steve.
STEVE: Well, this one just dropped into my lap. And I thought, okay.
LEO: Yeah, free puzzler.
STEVE: This is just too fun. This week's picture, I got a kick out of it. It had been in my queue for a while. I don't know where it came from. I'm sure somebody tweeted it to me, and I said, oh, that's just too funny. Sort of a modest prompt for an account creation, this dialogue says: “Pick a password.” And then it says: “Don't reuse your bank password. We didn't spend a lot on security for this app.” So just caveat emptor, you know. Just be careful. We wanted to get this out the door. We asked Moe if he would make sure that it was secure, but we're not too sure about it, so don't use your bank password.
LEO: That's hysterical.
STEVE: Okay. So FreeRTOS is the number one most popular real-time operating system currently. On the first page, well, the second page of the show notes I've got a graph, or a chart, showing that the result of a questionnaire that was of 568 people asked last year what of all the various operating systems were they looking at. And, I mean, everything is there. QNX is down there at 3%. Wind River Linux, that's another popular one at 5%. TI has an RTOS, Texas Instruments, at 8%. The larger OSes, Debian is at 12%, Ubuntu at 11%.
LEO: Were they asking, though, for real-time operating systems, or just generally what operating system?
STEVE: Just operating systems because I wouldn't call Debian…
LEO: Debian is not a real-time…
LEO: Can you characterize a real-time operating system for people who are wondering what that is?
STEVE: Yes. And so that's a great question. Essentially the idea is it's what you would want if you were literally going to be a light bulb and not a console with a file system and all that. We know, for example, that our routers have Linux in them because they've got file systems and all kinds of modules and all kinds of stuff. But imagine if you are an appliance, if you're a device, if you're a glucose meter or a parking meter or a light bulb or something that wants to use software, but wants to be more like an appliance than a general purpose thing.
So you need a few things. You have to have a processor. And it doesn't really make sense for the things that are always going to be done again to be done again, like memory management or thread management. You might want to have like several different execution threads. We've talked about execution threads a lot. They're an abstraction of a single-threaded processor where you have a scheduler that jumps the processor around between different things so that everything sort of seems to be going at once, because none of them needs all of the processor, so they can share it. So a real-time operating system will have a scheduler.
There's also typically a common memory pool. And so this real-time operating system will have a memory manager which allows the threads to say I need to use some memory for a minute or two, which it then gives back to the operating system. So the operating system doles out memory and recovers it after it's been freed. Oh, and you often have interthread communications. Sometimes you might have a couple threads that all may need access to a shared resource like the LCD on the parking meter. And so you can't have two threads using it at the same time. So there's something known as a “mutex,” a mutual exclusion event object, where one thread says give me access to the LCD. And then any other thread that also wants it has to wait until the first one is through.
Well, so the point is that that's sort of a common resource. And again, the RTOS, the real-time operating system, is the one that manages those things. So these threads of execution are clients of the operating system; but it's not an OS, as I've said, like we're used to - Debian, Windows, Mac or anything. It's just it's like the bare necessities to have a processor, the processor hardware, being able to appear to be doing lots of things at once. Like, for example, the LCD is being refreshed while the buttons on the parking meter are being scanned, while the credit card slot is being checked. And maybe it's got WiFi into the city's network. All of those things are sort of - they're not heavyweight processes, but they all have to kind of happen at once. And so the RTOS does that.
So anyway, the point is that in this chart FreeRTOS, this particular one, is 28%. It is the number one operating system where this group of people, of engineers, who were asked what they're looking at, this was the one that they were aiming for. So as a consequence of its popularity, last Thursday's blog posting by Zimperium Labs, also known as zLabs - and we've talked about Zimperium in the past. They've been a source of interesting discoveries on the security side. The blog posting was titled “FreeRTOS TCP/IP Stack Vulnerabilities Put a Wide Range of Devices at Risk of Compromise: From Smart Homes to Critical Infrastructure Systems.” And of course this is not good news to anybody who's using FreeRTOS. And it's been around for a long time, so like 14 years, and it supports over 40, four zero, different hardware platforms.
STEVE: I mean, every processor you could imagine. Because it's a C library that is easily transportable. If you've got a C compiler for a piece of hardware, you can port FreeRTOS onto it, and you only need a little bit of assembly language to deal with some things that C can't do because it needs to get right down to the bare metal. But anyway, so it does have a TCP/IP stack. And it turns out that there are some problems with it. It's now at v10.0.1, and everything up to and including 10.0.1 are vulnerable.
LEO: And the kinds of systems this is in may not be easily updated.
STEVE: Oh, that's the problem, exactly. That is, yes, that is exactly - and we know this is classic IoT stuff that may not be, as you said, subject to update. Yet if it's using the TCP/IP stack which comes along with FreeRTOS, it's got problems. So this is, I mean, there's a lot of concern in the community. When Amazon took it over, they reset the numbering and named their fork, essentially, of it “AWS FreeRTOS.” And it, too, was vulnerable until fixed, up to v1.3.1. And then there's a commercial version that is sort of like a - it's a functionally identical, so you wouldn't know you weren't using it, but it's not open source.
An outfit called Wittenstein High Integrity Systems (WHIS), they call theirs OpenRTOS and SafeRTOS, although it isn't. And they're vulnerable, too. So what Zimperium said in their blog was, they said: “As part of our ongoing IoT platform research, zLabs recently analyzed some of the leading operating systems in the IoT market, including FreeRTOS. FreeRTOS,” they wrote, “is a market leader in the IoT and embedded platforms market, being ported to over 40 different hardware platforms over the last 14 years. In November of 2017,” so one year ago nearly, “Amazon Web Services took stewardship for the FreeRTOS kernel and its components.”
They write: “AWS FreeRTOS aims to provide a fully enabled IoT platform for microcontrollers, by bundling the FreeRTOS kernel together with the FreeRTOS TCP/IP stack.” However, I was impressed with what I discovered as I dug into this a little bit because these guys said “including modules for secure connectivity, over-the-air updates [yay], code signing, AWS cloud support, and more.” They said: “With the infrastructure that AWS provides, and the AWS FreeRTOS platform, developers can focus solely on innovation, thus reducing development time and costs.” Sounds a little bit like an AWS ad, but it's not. This was from the Zimperium guys.
And then they go on to explain what I already did about the commercial version from Wittenstein High Integrity Systems that also has these problems. Anyway, they said: “During our research we” - meaning Zimperium Labs - “discovered multiple vulnerabilities within FreeRTOS's TCP/IP stack and the AWS security connectivity modules. The same vulnerabilities are present in the Wittenstein Connect TCP/IP component for OpenRTOS and SafeRTOS. These vulnerabilities” - and, yes, they are as bad as they get - “allow an attacker to crash the device, leak information from the device's memory, remotely execute code on it, thus completely compromising it.”
They said: “We disclosed these vulnerabilities to Amazon and collaborated, and continue to do so, with them to produce patches to the vulnerabilities we detected. The patches were deployed for AWS FreeRTOS versions 1.3.2 and onward. We also received confirmation from Wittenstein that they were exposed to the same vulnerabilities, and those were patched together with Amazon.”
They wrote: “Since this is an open source project, we will wait 30 days” - so that counter has begun - “before publishing technical details about our findings, to allow smaller vendors to patch the vulnerabilities.”
Zimperium's Ori Karliner, who conducted the research, discovered four critical remote code execution vulnerabilities, one denial of service, seven information leaks, and a partridge in - no, no, and one other which was unspecified. So anyway, we don't have details. But I just wanted to sort of put the word out. This is, as you immediately reacted to correctly, Leo, I mean, this is big. This is, I mean, this is like the microkernel hardware platform of choice. Not all devices will be affected that use it because they won't be connected. They won't have TCP/IP. For example, the blood glucose meter probably doesn't have TCP/IP support.
LEO: Whew, yeah.
STEVE: And hopefully the pacemaker doesn't. It uses something other than a WiFi link. It's like electromagnetic encoding or something with a magnet placed over the person's chest to talk to it. But, I mean, FreeRTOS is probably what's in these things because it's what you use. It's very small, so it doesn't - it leaves maximum space for the application to use most of it and provides the same set of services that the users, the developers, the designers would have to implement anyway. So why not use it?
The good news is it is getting fixed. The bad news is not everybody will be as responsible as Amazon has been. I mean, the fact that, I mean, I immediately dug into this, wondering what Amazon had done. And they said in their material: “Amazon FreeRTOS consists of the following components: a microcontroller operating system based on the FreeRTOS kernel; Amazon FreeRTOS libraries for connectivity, security, and over-the-air updates; a configuration wizard that allows you to download a zip file that contains everything you need to get started with Amazon FreeRTOS; and over-the-air updates.”
So they're, like, part of it is that they're offering this as a cloud service, the idea being, though, that they've made it like the default is that your device would check in with the Amazon cloud, probably not for free, but for a reasonable price. In fact, as I'm saying this, I'm sure we've talked about this recently - well, not too recently, but like a while ago - that Amazon was going to be doing this; and that while, yes, you didn't have to use Amazon's service, you could use your own, if the price is right, why not just fall back on AWS and let them keep your device up to date? So props to Amazon for doing this.
And the question is what non-AWS-based recent systems, because this only began with Amazon not quite a year ago, do use this always buggy TCP/IP stack with FreeRTOS and WiFi, I mean, you can imagine home connectivity IoT things, webcams and baby cams and baby monitors and security systems, that are using WiFi connectivity. If they're WiFi, they're TCP/IP; and, more likely than not, they're using a now known buggy version of FreeRTOS. So this may not be the last time we're talking about this on the podcast. Anyway…
LEO: So sad.
STEVE: Again, props to Amazon. This is what we need. It's got to be in there from the beginning so that it's like a no-brainer for the developer who says, “Oh, yeah, I'd like to have the kernel updated if any problems are found in the future, thank you very much.” And why not? If you have a TCP/IP stack, that means you could be on the 'Net, which means you can ping Amazon to see if they've got any news for you.
Okay. Now, unfortunately, the flipside of doing it correctly is how not to do IoT deployment correctly. And for that we have D-Link in the doghouse this week. Once again, a trio of vulnerabilities can be combined to result in a complete takeover of at least eight D-Link routers. A Polish researcher at the Silesian University of Technology in Poland discovered and reported responsibly to D-Link that eight routers that he found and tested all had, I mean, like really bad vulnerabilities. However, D-Link informed him that they would only be fixing two of the eight, that is, the DWR-116 and the DWR-111.
However, I went to D-Link - and this was responsibly disclosed some time ago. The timeline is really disturbing. On May 9th he notified D-Link. On June 6th he asked the vendor, having never heard anything, what was going on. On the 22nd of June, he received a reply that a patch will be released - okay, June 22nd - that a patch will be released for the DWR-116 and the 111, but the other devices were EOL. An announcement would be released. Okay? Then still nothing happened. September 9th, still no reply from the vendor about the patches or announcement. He wrote: “I have warned the vendor that if I will not get a reply in a month, I will publish the disclosure.” And on October 12th he did so.
So they had plenty of time. I went to the firmware pages for those two routers out of curiosity, the ones they said they would update, and they were still offering the vulnerable firmware yesterday. So just something to be aware of. I would argue that the height of concern, I mean, on one hand, I guess you would have to suggest that, okay, routers have a right to be EOLed, end-of-lifed; and how long should a router vendor be expected to maintain firmware for a really old router? The problem is this looks like it's all the same firmware - the DWR-116, the DIR-140L, the DIR-640L, DWR-512, 712, 912, 921, and the 111. All of those eight routers are apparently using the same firmware, which would argue that, if they just fixed it - just fixed it - they could make that firmware available to the whole family, for people who did want to maintain security.
Okay. So what are the vulnerabilities? We've got, as I said, a trio of them. The first one - and this is publicly posted now so everybody knows this. The first one allows remote attackers - right, remote - to read arbitrary files via the classic forward slash dot dot, which we've talked about before. Dot dot, as we know, is the go back a level in the directory hierarchy, known as the “directory traversal attack,” where you go /.., /.., /.., each one taking you up a level in the directory hierarchy until you get to the root. Then you move back down to the directory you want. He discovered that this works after a GET command to a /uir, which was some resource on the router. So he then posted a proof of concept where he issues a curl to http://routerip/uir// - that's the double forward slash that also performs the backup - and then etc, you know, et cetera, /passwd. Yes, that's the passwords file.
LEO: Is it only on the router? It's not reaching into the network, though.
STEVE: No, it's only on the router.
LEO: Any file on the router, okay.
STEVE: Well, at this point it's only…
LEO: Oh, because now you have the passwords.
STEVE: Now you have the password file. And he writes: “The vulnerability can be used to retrieve administrative passwords using the other disclosed vulnerability. This vulnerability was reported previously” - get this, previously reported - “by Patryk Bogdan in a 2017 numbered CVE, but he reported it as fixed in a specific release. But unfortunately, it is still present in newer releases.” So they had some sort of regression where they apparently briefly patched it, but then it became unpatched in subsequent releases.
He says: “The vulnerability is also present in other D-Link routers and can be exploited not only, as the original author stated, by double dot, but also absolutely using double slash.” So that suggests that the double slash immediately takes you to the root, and then you do etc/passwd in order to get the password file.
LEO: Yeah. And fortunately, that's encrypted; right?
STEVE: Okay. Vulnerability. You've been reading ahead, Leo.
LEO: No, I haven't. It's obvious this is worthless if it is.
STEVE: It's so painful. Vulnerability number two: Password stored in plaintext in several series of D-Link routers. And it's so bad that he even redacted the file from his own public vulnerability disclosure, after waiting half a year. He says: “Note: I have redacted the filename in the description to XXX because the vendor leaves some end-of-life routers” - and even those that aren't, as I verified yesterday - “unpatched, and the attack is too simple.” So in other words, this is so awful that even the researcher was unwilling to disclose it fully.
So he wrote: “The administrative password is stored in plaintext in the /tmp/” - and here's where he redacted - “XXX/0 file.” Now, of course, anybody can reverse-engineer any of the D-Link router firmware, get the name of that file, and now you know where to find the plaintext admin password. He says: “An attacker having a directory traversal can easily obtain full router access.” Right? Because it's under /tmp/something/0. Well, the first vulnerability gives you directory traversal.
And so now he has, again, a proof of concept: $ curl http://routerip/uir//tmp/XXX/0. He says: “This command returns a binary config file which contains admin username and password, as well as many other router configuration settings.” Meaning that it's binary, but they're right there unencrypted in ASCII, standing out. He says: “By using the directory traversal vulnerability, it is possible to read the file without authentication.”
And, finally, as if that wasn't enough, vulnerability number three: “Shell command injection in httpd server of several series of D-Link routers.” So “An unauthenticated attacker may execute arbitrary code by injecting the shell command into the chkisg.htm page Sip parameter. This allows for full control over the device internals.” And proof of concept: “Log into the router. Request the following URL after login.” And there he provides - now, remember, logging in is not hard because we can now be admin. Username and password, we can get that using the first two vulnerabilities. Now that we have those, we use this - and he provides the full HTML query which gives you a full shell command injection.
And so he finishes, saying exploiting all three together - “taking all three together it is easy to gain full router control including arbitrary code execution.” And then he gives us a link to a description with a video. So where we are today is that, as far as we know, I mean, every router, every D-Link router he checked was using the same firmware with the same vulnerabilities. They probably only have the firmware which they have sprayed across who knows how many D-Link routers through time. We know old ones. They're not going to fix them. And even the ones that are still being maintained, they said to him on September 9th that - oh, no, June 22nd was the word back, yeah, we'll fix two of them, but not the other six. Well, those two still haven't been patched.
So I think the takeaway here is that old routers are bad. I do accept the idea that a vendor shouldn't be responsible forever. And I was tempted to go back and figure out like when these various routers were first released. But I thought, okay, I do want to get SQRL finished someday. So I don't know how old they are, but I do know that every D-Link user is now in danger, and there is no recourse currently from D-Link to get yourself out of danger. This is now public. As we know, I mean, we've been talking about problems with routers. It's going to take minutes for Mirai botnet to add this to their collection of things to try as they scan for routers on the Internet. You don't want to be using a D-Link router is what it comes down to.
And unfortunately, that message will not get - it will not ever get out to most of the users of D-Link routers in the world. All of our listeners now know and need to be concerned, unfortunately, whether their particular model of D-Link router is vulnerable. There is absolutely no reason to believe at the moment that it isn't because every D-Link router tested of eight have been, including current ones and especially old ones for which D-Link is never going to provide a firmware update. So I think at this point we need to consider routers as a commodity which ages out of use for reason of the vendor no longer maintaining it.
And certainly in this case, I mean, as we said, routers are the attack target of the year, maybe of the decade, because they're all sitting on the Internet. They've got networks behind them that may have juicy tidbits on them. And even if not, facing outward, botnets are grabbing them up and using them for attacks and reflecting traffic and more. So I just - any listener, I mean, again, if I had the spare cycles I would add a test to GRC to check for this particular problem. If you're a D-Link router user, certainly if you have any of the D-Link routers I mentioned and that are written in the show notes, there's nothing you can do, as far as I know, except make sure that nothing is exposed publicly. Apparently this does require a public server access for someone outside to get to your router. So if that could be turned off so that there's no WAN-side admin…
LEO: If you turn off the WAN. Okay, good. All right. That's good.
STEVE: Yes. Presumably that will protect you.
LEO: Unless there's a bug there, too. I mean, who knows; right?
STEVE: There was no mention of mitigation in anything that I found. So I can't say one way or the other. But I do think that at this point we have to take the position that an unsupported router, obviously one where there's known vulnerabilities the manufacturer has said they have no interest in fixing, even though it appears they could just fix it once and make the same firmware available, they just don't care. So at that point you just have to say, okay, they're not that expensive. It's worth saying, you know, it's worth having a garage sale and sticking it out there on the table and say to your neighbor, well, good luck with this. I've got a new one.
LEO: Here. Here's a broken router. Good luck.
STEVE: Works great. And the hackers love it.
LEO: Yeah, works great for everyone.
STEVE: That's right. Okay. So Drupal. As I said at the top, if you're using Drupal 7 and 8, US-CERT, the United States Computer Emergency Readiness Team, announcement said a remote attacker could exploit some of these vulnerabilities to take control of an affected system. So it really needs to be Drupal 7 and 8 have a problem. There were five problems. Two were critical, and three Drupal's own security team said were moderately critical.
One of the two critical bugs is an injection vulnerability in the default Drupal email backend, which uses PHP's mail function, which is DefaultMailSystem::mail in both Drupal 7 and 8. When using this default mailer to send email, some variables were not being sanitized - get this - for shell arguments. As is common, when untrusted input is not sanitized correctly, remote execution may result. And in this case it does.
The second of the two remote code execution bugs exists in Drupal 8's Contextual Links module. In Drupal these modules supply contextual links that allow privileged users to more easily perform tasks related to regions of the page, thus contextual, without having to navigate to the admin dashboard. However, the Contextual Links module also doesn't sufficiently validate the requested contextual links, which allows an attacker to launch a remote code execution on those links. That is to say that the links themselves are to code in Drupal which assumes its own variables haven't been tampered with. But you can tamper with them, use the same link target URLs, and execute your own code on that Drupal service.
So then in addition to those two baddies, the Drupal security team acknowledged that there were three other moderately critical ones. They said users of any version of 7 should move to at least 7.6. Users of 8.6.x should move to at least 8.6.2. And users of 8.5.anything or earlier should move to Drupal 8.5.8. And then they noted in their security advisory that minor versions of Drupal 8 prior to 8.5.anything are not supported and do not receive security coverage. So sites running older versions should update to the above 8.5.x release, which is currently 8.5.8. And those older Drupals, 8.5 series, will receive security coverage until May of 2019. So if you're going to jump, it's probably worth jumping to 8.6.2. Just bite the bullet now so that you can continue to get coverage.
Now, although Joomla and Drupal both lag far behind WordPress's nearly 60% domination of the content management system (CMS) market, Joomla and Drupal having 6.6 and 4.6% usage in the CMS market, respectively, even 4.6% of Drupal CMS-driven sites being vulnerable to remote code execution is no laughing matter. So I hope any admins using Drupal are signed up for security updates and are going to take these problems seriously and get this fixed.
LEO: I have to say, I mean, as Drupal users here, we love Drupal, and I've used Drupal since the beginning. You know, it's easy to - any software can have bugs, and Drupal does a good job of keeping it up to date. And they always have said for years, don't use old versions. Keep it up to date. But sometimes the jump is huge. Sometimes it's a big discontinuity, as we mentioned before, between major versions.
STEVE: And in fact it's funny you should mention that. There does look like there's some things that they changed so that you do need to dig around in the code a bit. So it's not just a completely seamless jump. They're not happy with some of the functions that they have defined, and they had to change them.
LEO: They changed - yeah. So, yeah, that's always been the problem. I blame PHP. I really do. A lot of this - and I think the RTOS, FreeRTOS, as well - goes back to the choice of language. And I just - people need to use type-safe languages and capture these problems at compile time, not run time. That's just…
LEO: Yeah. We know how to do that, you know? So let's, let's.
STEVE: Yes. Declare all your variables and make sure you don't use them until you've declared them.
LEO: Yeah. Things like that, yeah.
STEVE: Gee, what a concept.
LEO: Because a lot of this comes to referencing null pointers and things like that, and you can avoid that. And a compiler should catch it anyway. All right. I'll get off my high horse.
STEVE: Okay. So this is sort of mostly just a public service announcement for anyone who might be using and have a publicly available real-time streaming protocol (RTSP) media server. There's a company called Live Networks that has a very popular multi-format RTSP media server known as LIVE555. It contains, unfortunately, a critical remote execution bug. Boy, is this becoming a broken record.
LEO: Heck, yeah. Yeah, yeah, yeah.
STEVE: Which affects versions prior, all versions prior to last Wednesday's release of 0.93. So that just happened on October 17th. And if you haven't updated, if you're using LIVE555 streaming media server to offer anything publicly, if it's just internal Intranet then you're okay, assuming you can trust all your internal users. But there is a remote execution bug which would allow any publicly exposed version of this media streaming server to be taken over remotely. So anyway, I did not get a sense for how widely used it was. But again, it only takes one, if you're the one who uses it, and somebody's able to scan, find the server, and say, oh, thank you very much, we want to crawl inside your network through this little portal that you've created.
LEO: We don't use it, but I'm well aware of Live. They've been around for a long time. And in fact…
STEVE: Yeah, Live Networks is like the real deal.
LEO: Yeah, they're one of the biggies. The only thing I know, though, is I have a real-time streamer, Facebook streamer, that I bought the Mevo cam from them, which probably uses that protocol.
STEVE: Yeah, probably.
LEO: Yeah, something to be aware of.
STEVE: So we're unable to stop talking about, for better or for worse, the Windows 10 October 2018 update. The infamous Build 1809 has another problem. So as we know - the good news is it's still not rereleased yet. So they found this in some preview build 18234, also known as 19H1. I've not been tracking all this cryptic insider…
LEO: 19H1 is the next one.
STEVE: Okay. Okay.
LEO: So 1809 is the one that was supposed to come out now, September 2018. Now you're talking about 1903, basically, which is 19H1, the first half of 2019.
LEO: So that's in the Insider - the Insiders are getting this now.
LEO: You know, there's something wrong with the process at Microsoft. This is actually getting to be problematic. I mean, seriously.
STEVE: Yeah. And you know, Leo, we're looking for that new era of enhanced productivity.
LEO: Oh, lord.
STEVE: Was that what they were advertising?
LEO: That's something, and the most secure version of Windows ever.
STEVE: That's right. I love that one.
LEO: As we well remember.
STEVE: Good old XP.
LEO: I've seen a number of articles recently saying the problem really is the way Microsoft does this, which is they've got a code base. They do these short, like six-week sprints to add a feature. So they spend a long time thinking of the features. In six to eight weeks they create the feature. There's no testing at that point. They lay it into a testing version which they then test for a long period of time. This is how they did it when they did three-year-releases, but they're still doing this now for these biannual releases. And it's not an effective testing process. They need a better way of testing before they get them into these beta releases. That may be. There may be a structural problem here. We'll talk about it tomorrow on Windows Weekly, I'm sure.
STEVE: So as we know, when the content, those of us who are power Windows users and who understand zip files, when the content of a zip file extraction would cause the overwrite of an existing same named file within the archive, the user - I know, Leo. Are you sitting down? The user should be prompted about the pending overwrite.
LEO: Collisions, yeah.
STEVE: That's right, a file naming collision, and asked whether to replace or skip the extraction of the colliding file. It turns out…
LEO: It doesn't.
STEVE: …that Build 1809 is reportedly and reproducibly either overwriting existing zip file content without notification or silently failing and doing nothing. So the good news is this problem has been caught before, well, what I wrote - now I'm not sure. So it's been caught before the full formal re-release of Build 1809. But it was reported as being in 1809. I assumed it was the pre-release people.
LEO: Yes, yeah.
STEVE: A recent tweet by an IT staff engineer at Microsoft on the Windows Insider Program Team indicated that this problem has been resolved back on October 6th with the Windows Insider Preview Build 18234, which is 19H1.
LEO: So that means they fix it for the next generation.
STEVE: Yeah, well, they clearly have to fix it now.
LEO: Fix it for both, yeah.
STEVE: And I know that this is just - I'm kicking this dead horse one last time. But in some recent reporting over on Computer World I noted that Microsoft's forensic analysis revealed that as many as 1,500 instances of Build 1809's pre-release testers had their files deleted and complained without Microsoft noticing.
LEO: Microsoft said it was 0.01%.
STEVE: Yeah, well…
LEO: What did you just say? What percentage?
STEVE: 1,500 by number.
LEO: Oh, okay.
LEO: That might fit. That might fit. Because a lot of people try these builds. This is a public build.
STEVE: They did. They did. And but a lot got bit. And Microsoft said, oh.
LEO: Yeah, that's a lot of people.
STEVE: We missed that one.
LEO: Lot of people with bad [crosstalk].
STEVE: Also I should just note that, after talking about this for the last few podcasts, in between then and now I updated the machine I'm talking to you on, Leo. This camera that I'm looking at is running Windows 10. It was running Windows 10 Home, which is what came preinstalled on the little Windows 10 box that I just grabbed, just a little turnkey box. Anyway, it's now running Pro because I'm an MSDN subscriber, so it doesn't cost me anything to update.
So I updated to Pro since I definitely decided that I want to begin hanging back from each month's security updates as well as the biannual “feature” update. There's nothing I need that much each month that's worth being bit like this. And I'd rather let them, you know, I think that the security release we would know within a week if it was causing problems. So I've set that to give me two weeks. And I think I set it to 30 days for the feature update because we would know by then if it's something, if you really should put it off further.
So again, and as I mentioned, Windows 10 Home does not give you the option to delay, to defer these. You take them when they make them available. I hope that our listeners consider, after this painful set of October surprises, consider deferring, as I now have, I mean, so much so that I switched to the Pro version just so that I could have that feature. It just seems wrong that Microsoft is being stingy about that. No, we're going to make you have Pro if you want to defer. It's like, my god, okay.
Okay. So the most popular, second only to the jQuery platform itself, the most popular jQuery plugin, which has been around for 10 years, is vulnerable. This is the jQuery File Upload plugin which was released, like I think it was within a week, just like five days before the Apache team changed the way the .htaccess file is handled in Apache. As a consequence, the use of .htaccess, which is, as people know, .htaccess, anyone who's configured Apache, you're able to use that to place that file in a directory to apply access restrictions to that directory. Well, it turns out that 10 years ago, with Apache v2.3.9, the Apache maintainers deliberately disabled support for the .htaccess file, apparently as a performance improvement, because then the server would not need to check for this file every time it accesses a directory.
LEO: I remember when this happened because I used .htaccess frequently, and it broke a lot of stuff, yeah.
STEVE: Yes, yes. And also the problem is this left some developers - oh, and the other reason was the Apache people didn't want the local application of .htaccess to interfere with the server-wide configuration because they had alternative means for providing that protection.
LEO: Yeah, you just use sig file now.
STEVE: Right, exactly. Okay. So the story goes that the Messaging Malware Mobile Anti-Abuse Working Group met in Brooklyn, New York two weeks ago, Monday through Thursday. Attending that meeting was Akamai's Larry Cashdollar. That's actually his last name.
LEO: What a great name.
LEO: For a guy running a CDN. Awesome.
STEVE: He expected the weather to be nice, so he failed to bring a raincoat, and it rained throughout the week. So Larry was hotel bound. Having therefore nothing else to do - he couldn't walk around, sample the local fare - he decided to poke around at the various add-on packages available for Node.js at NPM, which is the packet manager for Node.js, npmjs.com.
Okay. So I read into the story pretty far, as you can tell. I'll skip the details of how he arrived at what he found. Two days later Larry posted an entry on Packet Storm titled “jQuery-File-Upload 9.22.0 Arbitrary File Upload,” with the description “jQuery-File-Upload versions 9.22.0 and below” - meaning all previous - “suffer from an unauthenticated arbitrary file upload vulnerability that allows for” - you guessed it - “remote code execution.”
Okay. So first of all, it's always been the case that allowing uploaded files to a server is extremely fraught, I mean, it's inherently fraught with danger. This is not to say that it's not possible to do so safely. But few things should instill more fear in the heart of the responsible web designer than enabling file uploads. How many times have we here on this podcast covered buffer overrun exploits in image renderers? As we've said, interpreters are very difficult to get correct. And there have been JPEGs and GIFs and PNGs, I mean, all these image formats.
STEVE: Yes, yes.
STEVE: Yeah. So imagine that you're able to put anything you want anywhere, and then you invoke it from a URL outside…
LEO: And that's the problem with PHP. It's a URL-invokable protocol.
STEVE: Yes, yes.
LEO: So stupid.
STEVE: So what we have here is much worse.
STEVE: Than even that, yes. Due to a presumably, as I mentioned, well-meaning change that the Apache group made back in 2010.
LEO: Well, that's one of the ways they used .htaccess is to keep people from uploading files to directories; right?
STEVE: Yes, yes.
LEO: To block a directory.
STEVE: So starting with that version of Apache, 2.3.9, the httpd server offered an option that would allow server admins to ignore custom security settings made to individual folders via the .htaccess files. This setting was made for security reasons, was enabled by default, which means, as I've often said, the tyranny of the default, enabled by default, and remained so for all subsequent Apache httpd server releases. So in the process this jQuery file upload, which is the most popular plugin, second only to the platform itself on GitHub, its assumption that it could protect its file uploads using a local .htaccess file was rendered, since 2010, invalid.
So on GitHub this plugin says: “File Upload widget with multiple file selection, drag-and-drop support, progress bar, validation and preview images, audio and video for jQuery. Supports cross-domain, chunked, and resumable file uploads. Works with any server-side platform - Google App Engine, PHP, Python, Ruby on Rails, Java, et cetera - that supports standard HTML form file uploads.” In other words, it's been around for years. Not surprisingly, lots of people use it. It's very popular.
So our Larry Cashdollar says in his vulnerability disclosure: “The code in” - and then he cites the URL on GitHub - “doesn't require any validation to upload files to the server. It also does not exclude file types. This allows for remote code execution.” As Larry wrote in his description of this discovery, he said: “I started looking through the package's source” - this is during a rainy day in Brooklyn - “and found myself peering at two PHP” - there's your favorite acronym, Leo, or abbreviation - “files under the directory server/php. The files are named upload.php and UploadHandler.php. The upload.php file calls the main file UploadHandler.php where all of the file upload code resides.”
He says: “I also saw that all files were uploaded to the files/directory in the web server's root path. I wrote a quick command line test with curl, and a simple PHP shell file confirmed that I could upload a web shell and run commands on the server.”
STEVE: And it's literally one line, and he used example.com just for the safety of posting, where he provides shell.php. And shell.php is a simple PHP program that just launches the system command shell into the HTTP response. He says: “A browser connection to the test web server with cmd=id returned the userID of the web server's running process.” He said: “I suspected this vulnerability had not gone unnoticed.” And get this, Leo. “A quick Google search confirmed that other projects that used this code or possibly code derived from it were vulnerable. There are a” - get this - “a few YouTube videos demonstrating the attack…”
LEO: Is there anything YouTube can't do?
STEVE: “…for similar software packages.” Okay. So this is of extra concern because the jQuery File Upload bug is not some obscure widget. It is an extremely capable, as we noted, it is extremely capable and an extremely popular add-on - get this - having been forked on GitHub 7,828 times to create descendant projects of that base package which are widely spread throughout the industry, deployed on websites far and wide. So that means right now, once again, this is public, and all PHP-based sites which chose to use this jQuery file upload in its 7,828 variations are currently subject to any attacker uploading any file of their choosing, executable, and running it on that hosting server. Since discovering this critical vulnerability, Larry's been busy. He's examined 1,000 out of the 7,828 forks of the plugin. Every one of them was also exploitable.
LEO: Wow. Of course, because you just copy the code, yeah.
STEVE: Exactly. And still worse, it turns out that at least some of the underground hacker community have been aware of this widespread backdoor for years. As ZDNet explains in their coverage under the title “Zero-Day in popular jQuery plugin actively exploited for at least three years,” they said: “A fix is out, but the plugin is used in hundreds, if not thousands, of projects.” They say: “Patching will take ages.”
ZD said: “For at least three years, hackers have abused a zero-day in one of the most popular jQuery plugins to plant web shells and take over vulnerable web servers. The plugin is the second most starred jQuery project on GitHub, after the jQuery framework itself. It is immensely popular, has been forked over 7,800 times, and has been integrated into hundreds, if not thousands, of other projects, such as CMSes, CRMs, Intranet solutions, WordPress plugins, Drupal add-ons, Joomla components, and so on. A vulnerability in this plugin would be devastating, as it could open gaping security holes in a lot of platforms installed in a lot of sensitive places.”
They say: “This worst-case scenario is exactly what happened. Earlier this year” - and as we know it was a few weeks ago - “Larry Cashdollar,” they write, “a security researcher for Akamai's SIRT (Security Intelligence Response Team), has discovered a vulnerability in the plugin's source code that handles file uploads to PHP servers.” And on and on and on.
Cashdollar says that attackers can abuse this vulnerability to upload malicious files on servers such as backdoors and web shells. He said the vulnerability has been exploited in the wild. “I've seen stuff as far back as 2016,” he told ZDNet in an interview. And apparently the vulnerability was one of the worst-kept secrets of the hacker scene and appears to have been actively exploited even before 2016. Larry found several YouTube videos containing tutorials on how one could exploit the jQuery File Upload plugin to take over servers. One of the three YouTube videos that Larry found and shared with ZDNet was dated August 2015.
So actually I'll note, I've mentioned this before, but it is for exactly this reason that my own forthcoming PHP-based - as I mentioned, I did go to v7.2 because, since I was setting up a server fresh, why not? The SQRL public forums are hosted on that. But they are running on their own physically separate and network-isolated machine that has no connection to any of the rest of GRC's network because there's just no way to trust a system like that. These sorts of things are going to happen. And while, yes, as long as somebody is wired into security events in the industry, you can keep up with things, I can't have code that I didn't write that was sourced from hundreds of different places in order to glue together a solution. I can't have that on my network.
So yikes. I hope that anybody who is aware of what's going on, knows that they used a descendant of this jQuery File Upload, will recognize that the author had the best of intentions. He worked with Larry. Initially he could not duplicate what Larry was seeing because the author's PHP test server was not configured to ignore the .htaccess file. By default, as we know, for eight years, since 2010, Apache has been. So anyway, the author immediately put file type restrictions on last week's fix of this. But it needs to be fixed comprehensively. Wow.
Oh. And we've talked before about the dangers of lapsed domains. Since domain ownership is valuable, we have systems in place to rigorously protect that ownership. As a consequence, over time, trust is created since domains are rarely successfully hijacked. But what about when a domain that's in use for some purpose is deliberately abandoned, and its name is allowed to lapse? We've talked about the problems of overlapping security certificates in the past where somebody would have a certificate that was still valid for a domain that was reregistered. Lapsing domains is something we see all the time since the Internet, as we know, is a constant churn with domains being abandoned and created.
We sometimes find that a link we haven't visited in a long time now takes us to some weird search engine or a marketing page or something. Advertisers long ago figured that lapsed domains would see some traffic, some non-zero level of traffic. So they began snatching up any that lapsed to camp out their own nonsense there. And in fact that happened to me. I used to - I referred in Podcast 44 to a domain that I had, grcmail.com, which I deliberately allowed to lapse. And if you go to grcmail.com, my uBlock Origin immediately blocks it because there's some horrible marketing junk. Some marketer grabbed that domain name when I allowed it to expire because I didn't want to keep paying for it every month, and I decided I wasn't ever going to use it. And now somebody's camped out there.
But what happens when a supplier of active content, like in this case embedded web page scripting, decides to throw in the towel and no longer host something that they have been providing for years? The Sucuri blog tells the story of that very nicely. I've paraphrased from what they wrote. They said when Twitter announced their new design for Tweet and Follow buttons back in October of 2015, so just about exactly three years ago, marketers across the web developed a mild anxiety, Sucuri wrote. The new design came with a decision to nuke their beloved Tweet Count feature. Social signals can be a huge credibility indicator for visitors and site content. So who doesn't think there's a psychological relationship between the number of social shares and the credibility of the content that's there? It's social validation, they write, plain and simple.
Naturally, bloggers and website owners with an aversion to change started looking for alternative solutions that offered the same feature. Marketers breathed a sigh of relief when easy-to-use services started popping up to offer Twitter share counts, and one specific one called “New Share Counts” quickly gained traction. It even integrated with other existing social share plugins, they write, like SumoMe, AddThis, and Shareaholic.