Analysis of the Galaxy Note 3 Android 5.0 Leak

When I first heard about the Vietnam leak of Android 5.0 Lollipop for the Galaxy Note 3 (SM-N900) I was hesitant to install it. I was comfortable with my highly modded custom ROM which gave me a solid performance boost to the traditional TouchWiz configuration of the Note 3. I had disabled DVFS, overclocked my CPU, changed the kernel, switched the CPU governor, and had per-app profiles to ensure I’d get the last drop out of my phone’s powerful Exynos 5420 Octa-core processor. But the flashaholism I left behind months ago crept back to me and ended up pushing me to getting this new build on my phone.

Reverting is a slightly more complicated process than simply flashing a backup, as it involves a couple of extra steps like switching the bootloader back to the Kit-Kat one. I figured that I was going to try it out then come back, stomping around, to my functional Xposed setup, heavily themed aesthetics, high-level mods and all of my performance tweaks.

But I shortly learned there was little reason to go back. Many of my complaints about TouchWiz were seemingly fixed. While the new aesthetics aren’t the best that I have seen, they are certainly better than the old Gingerbread look of the previous iterations of TouchWiz. But Samsung seems to have focused on their single, biggest, most-criticized fault: General UI Performance.

I am going to start my analysis by saying this: I co-drive a Nexus 5. After purchasing it, TouchWiz wasn’t the same for me ever again. There was something magical to the innate speed and responsiveness of the beautiful, nondescript black slab. Now, after this latest version, TouchWiz doesn’t just seem to match the Nexus 5’s Lollipop in real world usage – in many aspects, it surpasses it.

Now don’t bring out the pitchfork just yet. I’ll provide you with the details and analysis as to why this could be, backed up by research I’ve been conducting these past few days that you’ll be able to witness by yourself. So without further ado, lets get started:

Benchmarks

Now, one of the first things I always do when I install a new ROM is do a benchmark. For those of you unaware of the concept, it is a standardized stress test on a hardware component (or many) of your phone to get results that are commensurable across different devices. However, benchmarks don’t always represent real world performance, so take this with a grain of salt or skip it if you want to see real world performance comments later down this analysis.

When I had installed the developer previews on my Nexus 5 months ago, I didn’t see that many improvements in performance, and the benchmarks I had ran reflected only a small gain in efficiency. AnTuTu benchmark is one of the most popular and referential benchmarks out there. It was one of the first to attempt to crack down on artificial inflation of scores through various cheating techniques by OEMs such as Samsung and others, and it has constantly been revising itself to adjust for hardware improvements over the years. My own Nexus 5 on Lollipop scored a modest 37,636 on this test, a significant gain from the old scores of around 33,000 I used to get on Kit-Kat with OmniROM and using Franco Kernel. That configuration made me no stranger to UI speed and smoothness, and I think anyone that has tried either of those works individually can attest to their amazingness. However. my Galaxy Note 3 saw an even more significant gain. From its similar score of 34,000 it earned a reputable and impressive gain of over 33% at a massive 44,046. This score is a huge improvement, especially considering even the Qualcomm-based Note 3 (SM-N9005) has always performed better than the Exynos version and still had just about the same score.

Due to Samsung being infamous for cheating in their benchmarks by running a piece of code that maximized clockspeed as long as the benchmark app was open, I made sure in my screenshots to show that I ran the same tests on multiple benchmarks. In my testing, none of them showed the frequency shoot up to its default maximum of 1.9ghz, in fact, it comfortably tends to sit at 250mhz, the Exynos’ minimum clockspeed. More on this particular number later. Samsung has stopped cheating. If the scores were inflated up to 20%, the net theoretical performance gain could be of 63%. This is unlikely to be a completely true and valid statement, as such level of optimization is insane. But the benchmarks do show that something was done to address the old running joke of TouchWiz’s performance. There were optimizations here, and it was not just Lollipop’s default improvements over Kit-Kat.

Screenshots:

 

Real World Performance

With their old bloated and unoptimized piece of an OS, Samsung has shown us that benchmarks are not always a perfect measure of performance. They talk more about the hardware than the software, particularly the software’s most important direct goal: User Experience. Now, from the moment I booted this up I noticed how silky-smooth the animations ran – sixty-frames in almost every regard. The benchmark performance gains had a direct consequence in the real world, and it was so evident that soon after disabling S-Voice and running the benchmarks, I went to compare this device side by side with my Nexus 5, and the results blew my mind. TouchWiz was now just as fast and responsive, and even faster at GPU tasks such as drawing items. This will be seen in the final video comparison showdown provided at the bottom.

Disclaimer:  The following clips have been recorded using a third party screen-recording app. Said apps usually have a significant stress on the GPU. The following clips were all captured at uncompressed 720p resolution with a decent bitrate to make it crisp. Despite this substantial additional stress, you’ll be hard-pressed to find frame drops and much less the insulting stutters of previous iterations of TouchWiz. As a result, the following clips may not represent the smoothness of the UI in its entirety, but I am confident you’ll see that the performance gains are stunningly noticeable.

In these clips I did not use the TouchWiz launcher – not just because I can’t stand its sight, but because it might not have been useful for a standardized comparison with non-TouchWiz devices.

Application Transitions

The clip below shows a very speedy transition between applications. Not only do the apps get loaded from the RAM instantly, but they also get cached instantly as well. I think anyone that has ever had a TouchWiz device has noticed, at one point or another, that the home-button does not perform as responsively as in stock android. Now there is a virtually nonexistent delay, and you will see this better in the final comparison video. Launching an app can be very taxing on the GPU, and with Developer Settings’ GPU Rendering Profile option you can see for yourself how the GPU activity spikes rapidly when it needs to draw an app on the screen. More on this setting below.

The general responsiveness is consistent across all apps, even Samsung apps like the dialer, which was a hassle to open on Kit-Kat and back. Later on this analysis, in the final final comparison video at the bottom, you will see that it finished fully loading before the Nexus 5 stock dialer app. All in all, the responsiveness of the app launching and home button was TouchWiz’s biggest performance culprit and it has finally been exiled.

GPU Stutter

The second biggest problems TouchWiz faced were the stutters that broke the fluidity of list scrolling in particularly long lists with drawn elements such as pictures or shapes. In the clip below, you can see that the new version of TouchWiz has silky smooth scrolling (lets not forget that this is with a high-quality screen-capture taking place, adding overhead stress to the chipset).

What is all that at the bottom? It is the GPU Rendering Profile that was mentioned before. This is how it works: The bars span across the horizontal axis, which represent the last 128 frames of every window being displayed. Each column gives an estimate on how long each frame takes to render, meaning that when the GPU has intensive work, the bars get higher. The 3 colors of each bar are blue for the drawing process (usually the shortest, at the bottom), red for the processing of each frame, and orange for the execution (time it takes to send each frame to the compositor). So when there’s stutters, they are seen in the sudden spike during a constant list scroll, and its real world consequence is a frame (or more) loss. If you look closely at the video, the bars only spike during a) The drawing of lots of new elements when entering a new subwindow (like clicking to view an app’s details) or b) when the list reaches the bottom and it has to load new elements. You’ll see that at these points there’s significantly more orange than red, while framedrops are usually represented as a higher amount of red than orange. In short: Even huge, element-filled lists such as the Playstore’s can withstand extremely rapid scrolling without frame skipping. Another point for TouchWiz!

What is also impressive is the stability of the system when running multiple apps at the same time. With the multi-window features, you can have up to 7 applications open at once. I tested performance with 5 different applications running on the same window event. Despite the fact that the RAM is being actively murdered by this, the list scrolling remains smooth, the drawing and redrawing on windows is instant, and the switch between applications also features no delay. Every in-app action works entirely as it should as well. I want to lag the thing but it won’t let me.

Memory, Temperature and Throttling, and Battery

While processing power has been increased, and UI performance has been optimized, there’s also been a few notable changes to the rest of the system.

First of all, the RAM footprint of TouchWiz has been severely reduced. After a working day of use, it’s not rare to see between 1.4gb and 1.7gb of free RAM. On my previous ROM, I usually had about 950mb to 1gb of free RAM. The reason for this is that TouchWiz has been optimized to some degree on this regard (although not as substantially as the performance would suggest, which is odd). In the screenshot you can see the 1.7gb of free RAM that I typically get a few hours after boot. As the day(s) go on, that amount lowers due to Samsung’s system apps getting cached in memory, some apps like Kids Mode that can’t be force-closed. While the OS in the picture takes a measly (for TouchWiz) 575mb of RAM, after 3 days I’ve seen it hit as much as ~940mb of RAM which is more in line with the old TouchWiz we know and don’t love. Despite this and the 600mb of background apps I had at the time, the remaining 1.1gb proved more than enough to warrant no net speed loss.

However, if you are a performance fiend or just paranoid, you don’t have to reboot to gain full performance: Go to the memory button in the recents panel, and under RAM click “Clear Memory”. While this won’t necessarily free up your RAM as if you had rebooted it, in all iterations of TouchWiz is seems to kickstart its speed back in place. But again, it is not needed, as the device performs perfectly fine even with that 1.1gb of free RAM. Old TouchWiz didn’t fare so well under the same conditions.

On a last note (I did it again!), the Nexus 5’s ROM is much more lightweight, and mine usually has about 450mb of system footprint with 1gb of free RAM. Screenshot I took for comparison is provided below.

What also caught my attention, was the fact that the phone never seemed to throttle, even during long Asphalt 8 or Modern Combat 5 sessions. What really alarmed me is that I currently reside in Argentina, known for its hot summers at this time of the year. Last summer was so bad, that because of the heat my device received, Samsung’s aggressive DFVS throttled the performance on a chronic basis – so I had to disable it. Now, despite having had restless nights due to my lack of air conditioner in my room, the device seemed to stay pretty chill at an average of 35° C (95° F). Getting curious about the CPU, I benchmarked the clockspeed distribution and saw that the majority of the time the device was awake, it performed on its minimum frequency. This isn’t extremely uncommon for most stock android phones, but my old ROM seemed to enjoy sitting at a comfortable 500mhz. What’s also interesting is that the processor only reached its maximum of 1.9ghz during 3% of the total computation time. I think this might be a good sign that the OS has gotten easier to compute on some levels, either due to Google or Samsung.

Finally, the quintessential struggle of the modern smartphone: Battery life. Since this one will be very anecdotal, I don’t suggest you to take my word for what I’m about to say, because unlike on other points, I can’t provide you with solid numerical evidence or documentation. But I’ve seen a sharp increase in battery life. My old ROM, despite having Amplify xposed module to reduce wakelocks to a minimum, struggled with standby time and it drained at a firm 2.5% per hour in this state. This was mostly due to the fact that this battery has seen a power user’s abuse for 1 year and 3 months, and due to the insane battery prices in my country I’ve always hesitated to replace it. It was not uncommon for me to end the day with 3.5 to 4 hours screen on time before flashing this. If I wanted to use up my phone straight in a row, it’d only last 5.5 hours. The screenshot below shows a session where I was on my way to achieving a very modest 7 hours of screen on time in a total of 8.4 hours away from the charger. My past few days were filled with days like that. I’m personally more than content with it, considering the weariness of this lithium ion trooper. But again, I fall short with proof on this one, and battery varies greatly from user to user, so don’t take my word for it. Battery charging is still the same at 65% an hour – if this seems like a lot to you, keep in mind the Note 3 uses different charging technology than the usual phone due to its USB 3.0 cable. I provided a screenshot nonetheless.

  

Conclusion

So let’s round up what we’ve learned:

• At least a 33% increase in AnTuTu benchmark
• Samsung seems to no longer cheat on benchmarks
• App launching and opening speeds comparable to stock Android’s
• Home button lag no longer present – you return to your homescreen as soon as you input the command
• List scrolling is now smooth, with virtually no stutters
• A lower System memory footprint despite all the added functionality
• CPU frequencies tend towards minimum like in more optimized ROMs
• Healthy temperature, even for a power user
• No thermal throttling due to either a less aggressive or non-existent DVFS
• A noticeable increase in battery life

The performance of TouchWiz has seen a clear and objective increase. The benchmarks tell the story but the true evidence of it is in the real world experience. The animations are smooth and responsive, and it generally feels like a whole different ROM. As I mentioned earlier, being a flashaholic I am no stranger to speed improvements. I know it when I see it. But the numbers, benchmarks, and general theoretical and numerical evidence of it is just as important in my eyes, and every single thing I’ve tested, researched, delved into, recorded, captured or calculated has shown that this ROM is better in almost every regard when it comes to sheer optimization.

Now, I’m sure that as this ROM rolls out, and as more Lollipop Samsung devices come out, people will engage in all sorts of new debates over the legitimacy of the optimizations, their background, and my personal prediction: the whodunit. I can’t safely attest that this was all Samsung’s doing. Google deserves a lot of credit as they are ultimately the primary developers of Android and Lollipop, and if there’s something that Project Butter, TRIM, Kit-Kat and now ART have shown us, is that Google has been working harder every year to bring the Android experience in pristine silkiness to as many handsets as it can. But I think that we should credit Samsung here too. Some of the improvements are simply insane, and not even the Nexus 5 benefitted this much from Lollipop in apparent and sheer calculational power or optimization. The fact that the OS is seemingly lighter on the RAM and faster in general  is also a feat if we consider that I didn’t even mention that this ROM comes packed with all sorts of new features, even the amazing multi-window capabilities of the Note 4.

There are plenty of faults in this OS, though. The design still falls short to Google’s, and it pales in comparison to the original vision of Material Design. There is app inconsistency (although the same can be same about Google) and there is still a lack of accessibility and intuitivity to the system’s functions and workings. Moreover, Samsung has added even more bloat, with uninstallable apps such as Bloomberg Plus, Trip Advisor, and a bunch of useless Samsung cloneware (you can disable most of the bloat, but sadly not Kids Mode). And I found a couple of bugs on top of some Lollipop ones (although, for being 5.0 and not a later version, Samsung seemingly did some bug squashing on Google’s behalf), like an annoying auto-brightness inconsistency in the lockscreen.

While TouchWiz still has a long way to go in the design department, let’s remember that it has also brought a lot of software innovations adopted by Google itself in their AOSP builds. Even things like toggles and panel sliders have made it into stock, and I fully expect Google to implement multi-window in their next iteration. I think that each year, the Samsung hate gets more and more unjust and unjustified. So I think we should all look at the positives and notice the fact that they seem to be listening to their consumers. The beauty of Android is choice, and I’m glad TouchWiz, like almost every OEM’s skin, is becoming a better one. Let’s stop the pointless flame wars. Listen to Google: Be together, not the same.

I’ll leave you guys with a speed test comparison I did of my own Note 3 and the best 2014 purchase I made, the Nexus 5. They say a picture speaks a thousand words, so the 2,430,000 words this video is worth shouldn’t be taken lightly.

The post Analysis of the Galaxy Note 3 Android 5.0 Leak appeared first on xda-developers.

source: xdadevelopers