Tag: Dropbox

Dropbox Details Its Shift to SMR Drives

Dropbox posted an analysis of its adoption of shingled magnetic recording (SMR) drives in its data centers.

SMR is a storage method that writes data to overlapping magnetic tracks. The technique allows for higher storage density and lower power consumption but generally results in slower write speeds, as multiple tracks may have to be overwritten to write new data to the drive.

The adoption of SMR got Western Digital into legal trouble a few years ago after it surreptitiously began shipping SMR drives as part of its Red NAS line of hard drives without informing consumers.

In its analysis of its adoption of SMR, Dropbox notes that as of 2023, about 90 percent of the hard drives in its data centers are SMR.

The advantages are a roughly 20 percent increase in density while leading to large declines in power usage requirements,

The energy required to operate our hard drives is measured in power consumption per terabyte (TB/watt). Since our first 4 TB deployment, TB/watt has decreased by around 5-6x—largely because our SMR drives can cram more terabytes into the same physical and energy footprints as conventional PMR drives.

Our very first 14 TB SMR drive almost cut our power footprint in half for idle and random read workloads compared to its PMR predecessor. Our latest 18 TB and 20 TB drives show an amazing ~.30 watts per 1 TB in idle and ~.50 watts per 1 TB for random read workloads. Data from our vendors suggests this trend will continue, even as capacities increase.

Even as it is extolling the virtues of SMR, however, Dropbox is already preparing for the future, which it believes is heat-assisted magnetic recording (HAMR). HAMR is a technique that heats the hard drive to 450 degrees Celsius for a nanosecond as it writes data. This allows data to be stored in smaller areas on the disk, allowing overall disk density to be much higher. HAMR promises future 3.5″ hard drives with 50TB capacities.

In anticipation of this next jump in areal density, our focus has shifted from increasing the number of HDDs in our enclosures to minimizing the impact that physical vibrations can have on the I/O performance of higher density drives. While there was much more margin for vibrations in prior designs, that margin is now much less as HDD data tracks become smaller and spaced more closely together. It’s common to see high frequency vibrations cause head positioning errors, which can, in turn, cause performance degradation. Vibration can come from fans, the rotational forces and seek actions of nearby HDDs, even the HDD itself—or, when frustrated enough, a yelling engineer. ?

Our focus in the future will be to minimize HDD performance degradation from system vibrations by suppressing structural vibration of the system chassis and reducing fan noise. Putting more focus into this area will be critical as we onboard next generation HDDs, and it’s great to see some efforts already underway in the Open Compute Project (OCP) community. We are planning to leverage the OCP’s HDD Acoustical Surrogate—a new industry-standard specification for vibrational testing—in our seventh generation designs.

Routing Around Dropbox’s “Restricted Content” Fail for Ebooks

I am an avid reader and have about 17,000 books on my phone (because why not?) Recently I bought a Sandisk 400gb MicroSD card for my Note 8 and started syncing my Dropbox /books/ folder with the card when I started seeing an error message I’d never seen before:

File not downloadable from Dropbox, “restricted_content”

What’s going on here is that Dropbox is creating hashes of all the files in user accounts. It then compares the hashes of the files you have uploaded to Dropbox with the hashes that have been submitted to it by copyright holders of files that are allegedly being made available illegally on torrent and other sites.

If a file in your Dropbox folders matches the hash of one of these copyrighted files, then Dropbox will restrict what you can do with the file (specifically, making it difficult to sync or share in some contexts).

Fortunately, for a lot of types of content this is fairly easy to fix. Two compare the hash functions of two files, the two files have to be bit-for-bit copies. Even slight changes in a file will lead to an entirely different hash, which will then no longer flag the file.

So, for example, 90 percent of the books in my collection are EPUB files. An EPUB file is actually just a ZIP container that includes the book text, images, and other data for that format. So to avoid the “restricted_content” nonsense I:

  1. Renamed bookxyz.epub to bookxyz.zip
  2. Created a text file called “lol.txt” that just contained the text “lol” (you could probably just leave the file blank, and this would still work).
  3. Copied the lol.txt file into the zip archive.
  4. Renamed bookxyz.zip back to bookxyz.epub

Now the hashes for the old and new versions of the file are wildly different, so Dropbox no longer flags the files as “restricted_content”

 

Syncthing–Open Source File Syncing

Syncthing is an open source alternative to things like Dropbox or Google Drive for syncing files to multiple devices (including mobile…well, at least Android) across the Internet.

Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it’s transmitted over the Internet.

. . .

  • Private. None of your data is ever stored anywhere else other than on your computers. There is no central server that might be compromised, legally or illegally.
  • Encrypted. All communication is secured using TLS. The encryption used includes perfect forward secrecy to prevent any eavesdropper from ever gaining access to your data.
  • Authenticated. Every node is identified by a strong cryptographic certificate. Only nodes you have explicitly allowed can connect to your cluster.

OwnCloud

OwnCloud is an open source software package that lets you store, access and sync data across the Internet on a server that you control rather than having to rely on Google, Dropbox or any number of other cloud-based services.

OwnCloud’s server software requires PHP5 and MySQL, which means it will run on many web hosting services out there. Lifehacker has a nice walkthrough of the installation process.

OwnCloud appears to be actively updated and has a community that has added a number of apps and plugins that allow it to be used for task management, as a media streaming service, etc.

The one thing that OwnCloud is still missing is file encryption. OwnCloud can be configured to use SSL to encrypt data that is being transferred back and forth between the server and clients, but the files stored within OwnCloud itself are not encrypted. This means if someone hacked your server, they would have access to your files.

Almost all cloud-based systems, including Google Drive and Dropbox, also store your files without encrypting them, but it would be nice to see encryption as an option here. This still leaves something like Spider Oak as the best alternative for cloud-based syncing and storage that also encrypts files.

Before you went all-in on something like OwnCloud, you would also want to ensure you understand your web host provider’s limits and policies on monthly bandwidth very well. For example, this blog is run on a dedicated server and the contract I have with my provider allows me to transfer up to 3 terabytes of data per month. Typically the actual amount of data I use falls somewhere between 25gb to 100gb per month. If I started using something like OwnCloud, that would usage rise dramatically and I’d need to monitor my usage to make sure I don’t that cap and risk being charged extra fees.

Given how companies like Google, Microsoft and Dropbox rarely have their users’ best interests in mind, however, it is good to see that there are open source alternatives that can still route around them if needed.

SpiderOak — Like Dropbox, Except They Don’t Lie to You

I’m not a fan of Dropbox ever since it was clear they had deceived customers. I work on a freelance project where everyone uses Dropbox (after I go them hooked on it), and giving it up for that isn’t really an option. For everything else I was using Dropbox for, however, I long ago removed all of my files off their service.

Instead I’m using SpiderOak. Like Dropbox, SpiderOak is intended as tool to backup files to the cloud and then sync those files with other computers and devices such as smart phones.

What SpiderOak has that Dropbox doesn’t is the option for genuine encryption — my SpiderOak account is set up so that nobody, including SpiderOak employees, can access my files without my password. Dropbox promised users this capability, but it turned out they were not telling the truth and their employees could access user files at any time (or accidentally expose them for a couple hours to the entire Internet as they did earlier this year).

Of course with great power comes great complexity, and this is the major downside to SpiderOak. Dropbox is dead simple to install and use — it took me no time at all to get people who barely understood how to use their computer to install and use Dropbox with no problem. SpiderOak, on the other hand, requires a lot more thinking about what you’re doing.

So in SpiderOak, first you have to create a Backup set (for example, all files in a directory), and then the application backs that up. Then you need to go in and sync the backup, authorizing specific devices and/or specific directories or files. Don’t get me wrong — it is not that SpiderOak is obsessively complex, but rather that it is just not drop dead simple like Dropbox is.

On the other hand, SpiderOak is much cheaper than Dropbox — I’m currently paying $10/month for 100gb of space (and, like Dropbox, there is a free account with a 2gb limit).

The only other thing I’d add is that I think the Dropbox app for Android is crap. the SpiderOak app wasn’t much better until recently, but the latest version does a nice job of letting me pick and choose which files, directories, etc. I want my phone to keep in sync.

Dropbox Lied. End of Story.

For the past couple years, I’ve paid for a 50gb Dropbox account and actively promoted the service among friends and colleagues. Drobpbox has been extremely useful in managing some freelance projects I’m involved in. So when potential security issues surrounding Dropbox emerged back in April, I was concerned about just how private and secure the files I was sharing were.

Since April, there seems to be two basic schools of thought on Dropbox. The first is that Dropbox’s problems are really no big deal. That fact that employees of Dropbox can potentially access files are inherent to any synced system. On the other side are folks who have shut down their Dropbox accounts and forsworn the service forever.

Here’s what I take away from the debacle: Dropbox lied, both to me and to the other folks to whom I recommended their system. When I signed up for Dropbox, the service promised that all of my files were encrypted and that,

Dropbox employees aren’t able to access user files, and when troubleshooting an account they only have access to file metadata (filenames, file sizes, etc., not the file contents).

When the security concerns emerged, Dropbox weaseled out of the above promise by clarifying that Dropbox employees are, in fact, able to access user files, but they are typically not granted access to do so. As Dropbox put it in a response to this issue,

In our help article we state that Dropbox employees aren’t able to access user files. This is not an intentionally misleading statement — it is enforced by technical access controls on our backend storage infrastructure as well as strict policy prohibitions. The contents of a file will never be accessed by a Dropbox employee without the user’s permission. We can see, however, why people may have misinterpreted “Dropbox employees aren’t able to access user files” as a statement about how Dropbox uses encryption, so we will change this article to use the clearer “Dropbox employees are prohibited from accessing user files.”

Whether they intended to or not (and it is hard not to see the original statement as intentionally misleading), Dropbox lied about its security model. I and others took those assurances seriously and assumed our files were being encrypted client side.

Regardless of whether this or that feature or security option is a good or bad idea, the fact remains that I simply don’t trust Dropbox anymore. I have better things to do than worry about when/if Dropbox is going to have to release another “sorry, we didn’t mean to mislead you, but …” statement.

I already set my account up so it reverts to the basic free version once the renewal date hits later this summer. I’m moving everything out of Dropbox except for my Keepass file and any other files which are already encrypted, and for working with clients who are going to continue to use the service despite the security risks. I’m currently testing SpiderOak — which is like Dropbox but uses a client-side encryption model — for all my cloud-based file syncing needs.