Photos app floods broadband while seeding the cloud

iCloud Photo Library is an excellent idea. So is Amazon Cloud Drive, Google Drive, Dropbox, online hosted backups from Backblaze, Crashplan, and many others, and the like. You get an extra backup of all your data—you do have local backups, too, right?—and access from everywhere. With some services, there’s the added benefit of synchronization, creating local copies of some or all material on every connected computer.

But seeding the cloud is just as important as floating on top of its layer of data. How do you get the first giant blob of data loaded over broadband? It’s an oddly myopic missing piece, predicated on thinking most people don’t have much data they want to store.

Evaporation

In America, at least, most broadband is asymmetrical unless you’re in a unique market or paying a fortune. My home service is 16 Mbps downstream and 3 Mbps upstream (typically abbreviated as 16/3Mbps). This was predicated on the idea of us being consumers. The FCC’s recent redefinition of “broadband” as a term is 25/3Mbps.

Even if you have 100Mbps downstream service, you might be limited to 5 to 20Mbps upstream, and also have bandwidth caps or overage fees. Use more than a certain amount, typically 250GB per month with Comcast in the markets in which it’s testing overage fees, and you pay $10 per 50GB. Let’s say you have bounteous broadband with a 20Mbps upstream connection: you can upload 9GB per hour. But if you uploaded continuously at that rate, you’d run through 250GB in just under 28 hours. (In the few areas of the country with head-to-head competition caps mysteriously don’t exist.)

That may seem difficult to cross. However, because it’s counted both upstream and downstream, if you’re streaming and downloading digital video on a regular basis, you might already be bumping up against a monthly limit. Add to that seeding hosted backups or iCloud Photo Library, and you’re in the hole, have your broadband throttled, or are warned your account might be closed if it continues.

And that’s only if you’re lucky enough to have high throughput. Reader William Green wrote in to waggle his finger at broadband snobs, and rightly so. He wrote, “We in rural America struggle with good internet connectivity. I am on the end of a DSL line (which only came into service for me in the last two years) and speed on download is only 1Mbps.” If he also has 1Mbps upstream, which is unlikely, it takes him over 2 hours to upload 1GB.

The FCC’s research shows that 17 percent of Americans lack access to—much less subscribe to—rates that meet its 25/3Mbps broadband definition, and that falls to under 50 percent of rural Americans—20 percent are in William’s camp, and can’t even get 4/1Mbps access.

Apple did a smart short circuit of this problem with iTunes Match. Rather than have everyone upload their own ripped music, iTunes Match identified overlaps with songs that Apple already sold. It only needed to uploaded a subset of most people’s libraries, using placeholders for the rest. (If you’re like me, you probably took advantage of the better-quality encodings, and downloaded matches.)

But it didn’t seem to give this a thought with iCloud Photo Library. Apple is going on 15 years of its digital hub strategy, much of which has involved importing media—music, photos, and video—to both computers and mobile devices. I’d warrant that most consumers who make normal use of iOS devices have tens of gigabytes of images, if not more.

It’s raining up

Right now, I’m babying my iCloud Photo Library uploads as I have 100GB to upload. I’m not sure this is unusual, although I’m on the edge. My wife and I have a shared family library, and it includes photos taken on consumer cameras and iOS devices. (I have a mirrorless camera that I shoot in RAW mode, and manage those currently in Adobe Lightroom.) The photos primarily date back to 1999.

When I let Photos upload as it wishes, my broadband connection is flooded in a manner I don’t think I’ve ever seen other synchronization software do. It grabs every available slot, making even downstream activities, like retrieving a webpage, become glacial or fail. Yes, even a few bytes of an website request are overwhelmed by the might of Photos.

You can find similar stories on social media, as well as from other Mac experts. Jason Snell wrote of his puzzlement and then discovery of the problem at Six Colors after starting up iCloud Photo Library:

The next day, my Internet connection seemed to die, or at least become sporadically inert. Traffic would sometimes squirt through, but after long delays. It was weird, and intermittent, and I was really sad.

Joe Kissell noted in a Q&A at TidBITS on Photos:

Q: Will iCloud Photo Library be considerate of my bandwidth and data cap?

A: No. Not even remotely.…It happened to me, and it has happened to other people I know. It’s pretty awful.

I routinely push 100MB and larger files into Dropbox, and it doesn’t fill the upstream pipe. Likewise, I use CrashPlan for cloud-hosted backups, and it seems to keep track of network demands and mediates those to keep my connection solid.

Apple also adds to the problem by not using LAN-based sync, a feature found in Dropbox and other software. If you have full-resolution downloads enabled in Photos on multiple machines on the same network, any non-duplicated media item in each library will be downloaded from iCloud, not synchronized over the same network. Dropbox discovers other locally accessible computers and side-copies to reduce bandwidth usage and vastly increase speed.

Per ardua ad nubem

Apple had choices and still has the ability to make them to mitigate this network flooding.

• Implement sensible network-performance checking as is used in many other pieces of software to prevent broadband saturation.

• Offer an advanced setting to throttle maximum broadband consumed. CrashPlan and other backup software offers this, sometimes with a split between local network and Internet transfers.

• Use peer-to-peer syncing and downsampling to prevent duplicative broadband use. Dropbox does this; Apple surely could, too.

• Allow users to set an iCloud-wide maximum monthly upload with a priority of what media is uploaded first, such as newest-first or oldest-first. Synchronized over multiple computers, perhaps with an override when you’re on a different network, this would prevent exceeding caps and incurring fees. This is a feature many sync, cloud, and backup services need as well.

• Offer seeding services at Apple Stores. Apple doesn’t have gigabit broadband at all of its stores, though I believe it does at some. Allowing customers to bring in a laptop or even desktop and upload, or handing over an exported library on a drive would be a dramatic improvement. Some online backing and synchronization services let you pay to mail in a drive to seed files (or they send you one that you load up and return), as the mail or shippers are much faster than our broadband connections.

Broadband isn’t rocket science. One would think Apple would have been eating its own dog food (as the phrase goes) with its engineers and perhaps executives testing this in advance. One would expect their libraries would be huge, and judging the availability of high-speed Internet in California, most home connections probably suffer the same limits and issues.

I currently have a bedtime procedure to help with this problem. During the day, Photos iCloud sync is paused (File > Preferences > iCloud). Before I hit the sack, I click Resume. I wake up in the morning and click Pause for One Day. I started with over 13,000 photos to sync, and between 1,000 and 2,000 are uploaded each evening.

With perseverance, I believe I’ll reach the cloud.