Newsletter #15 - August 2020 (LineageOS progress update) (Patreon)

It's been quite some time since I wrote my last newsletter post, and as some of you might remember (those that have been with me since the very early days), I've asked you guys whether it would be okay to work on bigger goals that might span several months of development time at a time.

Well, it has been several months, a lot of fellow Patrons have left, but you, you have shown faith and stayed, despite the long radio silence.

As such it makes me proud to announce to you all that a lot of progress has been made.

When I initially started toying around with the idea of getting the Android software of the XD+ updated to something more recent I had this idea of making the device Treble-compatible to ensure future updates would have an easier time landing on our device.

And what can I say? After months of tinkering, building a whole new kernel, HALs and countless other components that were in need of a facelift, we're finally there.

My XD+ is running on Treble, and along with it it's running a (now near complete version) of Android 8.1 Oreo. With just a handful of spit & polish tasks left to be done, let me take you through the list of hurdles that I had to overcome in the last few months, how I did it, and of course what to expect of the soon to be coming release.

In addition to that I will also give you guys a quick look into my crystal globe, trying to predict what will come next after all of this.

The partition layout (/system & /vendor partition split)

To make a device Treble compatible it is required to split all "vendor-components" (drivers, boot configurations, HALs, and so on...) into a separate partition, away from the rest of the operating system.

Luckily we have pretty much limitless control over the eMMC storage via SPFlash, which in turn allowed me to craft a custom SPFlash package that took care of this issue.

The kernel (and drivers...)

To make a device Treble compatible it is required to run at least Kernel version 3.18, have full hwbinder support and an updated set of supporting drivers (like LCM for example).

This was (by far) one of the lengthier steps, I found myself re-basing kernels over and over, trying a wide range of different angles of merging in all the required changes in an attempt to cut the process short, but in the end I had no choice but to sift through all the diffed commits in Google's reference kernel one at a time.

Countless sleepless nights were sacrificed to this beast.

The HALs

To make hardware work in Android it uses something called a "Hardware Access Layer". Think of it as a translator of sorts that takes requests from the Android operating system and translates it into something the Kernel driver can understand.

(The kernel driver in turn would then translate that into something the actual hardware can understand...)

Either way, these HALs are constantly changing, and have been for a while now.
Every time Google releases a new Android revision, it also releases a new set of HAL guidelines that manufacturers (or in our case, me), has to follow to make their device run properly on the latest revision.

As you know, the XD+ hasn't seen a Android update since 7.1.2... so some work had to be done on that front to make things match up with what Oreo & Treble expect them to be.

Luckily for us we had a donor of sorts that we could copy most (sadly not all) HALs from that near-perfectly matched the XD+'s hardware & drivers.

Enter the Alldocube X tablet: A chinese-made pump & dump MT8176-based Android tablet that shipped with 8.1 Oreo.

I quickly went to ravage their website, grabbed myself the latest software update they had laying around for the tablet, and went digging to see if I could re-use those HALs... and as it turned out several days (and countless experiments) later: Yes, we could.

The HALs (Part 2)

As I said before though, not all the HALs were compatible, and some needed more attention than others.

One particular candidate that ended up being tricky was the sensors HAL, the one that takes care of the accelerometer (screen rotation) and even moreso the Bluetooth HAL, which is giving me trouble even now as I write this post up.

(The Bluetooth HAL is in fact the one road-block I still have to work around before I can post a first beta build for all of you. So bear with me.)

Back to the sensors HAL, our XD+ ships with an implementation that is on par with Android's sensors HAL 1.0 definition, however, 1.3 is required to get us past this roadblock.

I essentially ended up HIDL-wrapping the sensor HAL and implementing the missing two functions myself. While I did that I also applied some flip logic to fix a rotation bug that was present in the original XD+ software.

Testing, more testing, and even more testing

While it is fairly easy now to pinpoint and list the exact steps that had to be taken, this wasn't as straight forward as this post might make it appear to be.

Nothing of this was documented, there was zero support from the manufacturer in this regard, and all of this had to be figured out / reverse engineered bit by bit.

But my 7 months younger self had to figure all of this out via a LOT of trial and error, countless small steps had to be taken, a lot of backtracking was required (as mistakes were made) and compilation downtime sucked up a lot of my time, as I found myself waiting for hours on end just to be able to test a new build / idea of mine that might just get me that one step further than I previously was.

The future

With most of the issues in the 8.1 Oreo build (sans Bluetooth) being ironed out, it's most likely a matter of weeks (rather than months) before the first beta-build will make it to you guys.

It will be posted in the 10$ tier for beta-builds (until it has ripened to a state that I'd consider stable enough for widespread use) and will require a full system format of your XD+ (as the underlying changes were just too big to make a lossless update feasible).

The new build will come with an updated PowerVR 1.9 driver (compared to the 1.7 version that previously shipped with the XD+), and as such one should be able to expect a variety of PowerVR GPU related glitches to be a thing of the past.

Given that this is a completely fresh Operating System, the first build will lack a variety of "convenience" features that CleanROM users might have gotten used to which will gradually be added back in as I update the software.

As for the "far away" future, the new build bases off of Treble, and as such will be used as a base for Android 9.0 updates, and eventually 10+, which you will also find on this Patreon.

It's late, the Bluetooth HAL is glaring at me, requesting my undivided attention, and as such, I'll say goodbye for now.

Let's hope that by this time a week or two from now, all of you will be enjoying your brand new, fully upgraded XD+!

I will see you then.

- Black-Seraph