External Blues: Google Has Brought Big Changes To SD Cards In KitKat, And Even Samsung Is Implementing Them
In recent years, Google hasn’t exactly been known as particularly hospitable toward SD cards with regard to its Android operating system. This theme is most often associated with the Nexus line of devices - the Nexus One was the only such handset to ever offer expandable storage. But despite arguments from Dan Morrill and Matias Duarte suggesting this stance is about keeping the Android interface simple and file picker-free, people still want more space. Google is apparently firming up its position on expandable storage even further, though, and in a way that limits flexibility and changes how we can use it.
A Bit Of History
Let’s start with just a little bit of terminology. Almost every type of storage in Android is considered "external storage," including the non-removable flash memory that comes in every device, which is designated "primary storage." Everything else is considered "secondary storage." Since the very early days of Android, an app simply had to request the WRITE_EXTERNAL_STORAGE permission to have access to any and all external storage.
Back in March 2011, almost 3 years ago, a small modification was made to the Android source code that would change how secondary storage mediums (SD cards) were mounted by the operating system. The commit message simply read, "Mount secondary external storage writable by AID_MEDIA_RW rather than AID_SDCARD_RW." The side-effect of this change was that applications would now have to belong to the media_rw group to modify the contents of an SD card. To gain access to this group, a permission called WRITE_MEDIA_STORAGE was added.
<!-- Allows an application to write to external storage. Starting in API level 19, this permission is not required to read/write files in your application-specific directories returned by android.content.Context#getExternalFilesDir and android.content.Context#getExternalCacheDir. --> <permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:permissionGroup="android.permission-group.STORAGE" android:label="@string/permlab_sdcardWrite" android:description="@string/permdesc_sdcardWrite" android:protectionLevel="dangerous" /> <!-- Allows an application to write to internal media storage @hide --> <permission android:name="android.permission.WRITE_MEDIA_STORAGE" android:permissionGroup="android.permission-group.STORAGE" android:label="@string/permlab_mediaStorageWrite" android:description="@string/permdesc_mediaStorageWrite" android:protectionLevel="signature|system" /> source: /platform/frameworks/base/core/res/AndroidManifest.xml (@hide added later)
In essence, WRITE_MEDIA_STORAGE duplicated the original functionality of WRITE_EXTERNAL_STORAGE, but with a catch: it was impossible for regular apps to request it. The new permission had a protection level of systemOrSignature, which limited it to system applications (usually those included by Google and OEMs) and anything signed by the creator of the permission (the OS itself).
The end result was that the original WRITE_EXTERNAL_STORAGE permission could only give apps the ability to write to the primary storage, but not secondary storage. The newer WRITE_MEDIA_STORAGE permission could write to secondary storage, but regular apps couldn’t access it. Basically, this cut off any possibility for 3rd-party apps to modify data on SD cards. There is a little bit more to this story, but we’ll get to that later.
Left: Attempting to write a file, Right: Attempting to delete a file
This change actually went live in Honeycomb 3.2, but it received fairly little attention. We can attribute that to a few factors. To begin with, Honeycomb's source code only became public with the release of Ice Cream Sandwich, meaning this change was buried amidst thousands of others. Further, the Nexus One never received an official OTA to Ice Cream Sandwich, so access to the SD card was never visibly lost. In fact, the only device known to be affected by the change was the Motorola XOOM. But the time the XOOM ran Honeycomb its SD card slot was actually disabled, and so the issue was overlooked.
However, the most significant reason the change never made news is because OEMs and custom ROM developers didn’t follow along with Google. For example, Samsung’s solution was to automatically grant the WRITE_MEDIA_STORAGE permission to any app that requested WRITE_EXTERNAL_STORAGE. Effectively, any developer expecting to have full access with the older permission would get exactly that. In similar fashion, CyanogenMod simply set the access group back to sdcard_rw, as it had once been.
Note: A huge credit for this background belongs to Chainfire, who researched and posted his discoveries more than a year ago. When he first covered this, the code in AOSP simply defaulted to mounting all non-primary storage devices under media_rw. Since that time - in Android 4.2, I believe - filesystem permissions have been restructured to use the FUSE (Filesystem in UserSpace) model. The code looks and works differently from that of Chainfire’s original posting, but the end results are basically the same.
The New Stuff
I mentioned earlier that 3rd-party apps couldn’t modify the SD card, but that’s not entirely accurate. As of KitKat, Google has a new documented behavior that directly relates to this:
[For all external storage]
“Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. This enables apps to manage their package-specific directories on external storage without requiring they hold the broad WRITE_EXTERNAL_STORAGE permission. For example, the app with package name com.example.foo can now freely access Android/data/com.example.foo/ on external storage devices with no permissions. These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.”
…
[For secondary storage, if it exists]
The WRITE_EXTERNAL_STORAGE permission must only grant write access to the primary external storage on a device. Apps must not be allowed to write to secondary external storage devices, except in their package-specific directories as allowed by synthesized permissions. Restricting writes in this way ensures the system can clean up files when applications are uninstalled.
This basically says apps may now have a folder on an SD card designated for their private use, where they can do anything and absolutely no permissions are required. It’s just like the private folders that already exist on primary storage, but with negligible security, since the SD card can be removed and freely accessed by a computer. The WRITE_EXTERNAL_STORAGE permission still grants unrestricted access to public folders on primary storage, but writing anything to secondary storage - outside of the designated folder - is totally off limits to all 3rd-party apps.
It’s also important to notice the last sentence, because it says data written to any private folder will be erased when the app is uninstalled. This means that common apps like alternative cameras, image editors, and GPS loggers probably shouldn’t store data in these folders because it will all be erased if you uninstall them for any reason. We’ll come back to this later.
While it’s not clearly documented, there is another tweak to the WRITE_EXTERNAL_STORAGE permission. Outside of those private folders on secondary storage, everything is fair game to be read. This makes it possible for users to put photos, music, and movies onto an SD card for later use. However, there is still no way to modify, delete, or add such files through 3rd-party apps.
It should be noted: transient storage devices, like USB drives, are not covered by these rules. The precise nature of how they are handled is still largely up to the OEM.
Why Talk About This Now?
OEMs have historically turned away from Google’s intentions for SD cards. After all, this all started in Honeycomb and most people still haven’t experienced it. However, KitKat OTAs from Samsung appear to have adopted Google’s intended behavior. The reason for the course correction isn’t clear, but the quoted text doesn’t seem to leave a lot of room for interpretation. In other words, OEM partners may be required to fall in line going forward.
Keep in mind, there is no certainty that the final firmwares from any other OEMs will function this way. The only evidence we have at the moment is a leaked firmware from a single OEM and documentation from the Android Developer portal. Of course, the Google Play Edition variants of the Samsung Galaxy S4 and LG GPad also function as documented, so there is precedent.
The Consequences
Just to sum up, here are the options 3rd-party apps have on KitKat:
- An app without any permissions:
- Automatic read and write for designated private folders on the primary and secondary storage
- With WRITE_EXTERNAL_STORAGE, they also have:
- Read and write for any public folder on the primary (built-in) storage
- Read (not write) for any public folder on the secondary (SD card) storage
Overall, it will probably depend a lot on perspective if somebody will see this as a serious issue, an inconvenience, or completely innocuous. It’s definitely unfriendly toward many power users, if for no other reason than it the extra hurdles it adds to simple tasks like making backups or cleaning up old files. It’s not that these things won’t still be possible through some other method, but they will become tedious and annoying.
These problems become exponentially worse for content creators. Photographers will be the first to run up against these restrictions, especially those that use high-end cameras for shots and then make edits on an Android device. Viewing photos on an SD card might not be an issue, but what happens when it’s time to delete bad shots or save back edited versions? This quickly becomes a painful and confusing experience. These problems will become exponentially worse as Android becomes a viable platform for creating and editing music and video.
Of course, some will suggest that rooting is the answer, and they aren’t exactly wrong. Power users tend to be more comfortable with this solution. But what about everybody else? Android was finally at a point where root seemed like a tool just for serious modders (especially the Xposed crowd) and enthusiasts with specific needs (backup apps and such). On a fundamental level, Android should not be giving the average person a new reason to hack their phone or tablet.
Perhaps There’s A Plan
If this new behavior is going to become a requirement in Android, things don’t look great for SD cards. We’re losing one of the most basic capabilities that we’ve had since the early days of Android. Since managing content via SD will be even more of a hassle, people are less likely to bother with in the first place. On the surface, it looks like this is setting up SD cards to become little more than a dumping ground for cached content and downloaded game files. But what if the goal isn’t to eliminate demand for expandable storage? What is Google trying to achieve here?
One possible explanation takes us back to Dan Morrill’sabout "sneeze and a file picker appears" syndrome. Perhaps the Android team determined that file pickers were just too unavoidable, so the a new plan was formed to make the experience as elegant as possible by hiding the filesystem from users. Enter the Storage Access Framework, a new feature in KitKat. Coincidence? Probably not.
The Storage Access Framework fits most of the criteria that the Android team has been driving for since Ice Cream Sandwich: the UI is attractive and consistent across apps, each provider can enforce its own security policies, and files can be represented as more than just a list of names. It doesn’t matter if the file is stored on the phone, on an SD card, or in the cloud. Through this interface, Android could be headed toward a paradigm where files are "owned" by applications, not just clumps of data loosely organized by a mess of folders. Assuming both apps are configured appropriately, the SAF can even allow one application to modify or delete files belonging to another app (if that app has allowed such access through the SAF), as though it had accessed those files directly. The actual location - SD card, primary storage, or cloud - would only be relevant insofar as the app providing the access to those files was able to modify them itself.
It’s not hard to see how the Storage Access Framework could look like an obvious replacement for classic filesystem access. That certainly sounds like a bright future and a better experience for users. Admittedly, it does feel like something we might expect from iOS or Windows Phone, but that isn’t always a bad thing.
However, If Google’s plan really is to herd app developers toward turning their apps into Document Providers, this is a pretty heavy-handed way to get there. Without any public direction to app developers, this also feels like a surprise, as if app developers were supposed to intuit this plan. There are also some holes in this solution that feel largely overlooked. In particular, how are users supposed to be warned that deleting an app can lead to the destruction of files they may want to keep? That’s not a pattern people have learned to expect on Android.
Perhaps I’m overreaching and there is no connection here. Maybe Google really does plan to slowly push SD cards out so cloud services can take over. I doubt it, but I might as well say it before somebody else does.
Whatever the case may be, devoted fans of expandable storage are likely to be very angry if their update to KitKat ends up killing write access with no warning or way to restore the old functionality. Seriously, not cool.
Sources: Chainfire, Android Documentation
Cody Toombs  | Cody is a Software Engineer and Writer with a mildly overwhelming obsession with smartphones and the mobile world. If he’s been pulled away from the computer for any length of time, you might find him talking about cocktails and movies, sometimes resulting in the consumption of both. |
source: androidpolice
0 comments :