Newsletter #25 - August 2023 (GarlicOS 2.0) (Patreon)

So what's been going on over the last few months?

Those that follow my Twitter are already well aware that I am trying to bring GarlicOS to a variety of devices both new and old.

This newsletter will try to give you a closer in-depth look at the current state of things, hurdles that had to be conquered to get here and some that I'm still trying to conquer.

What is GarlicOS?

GarlicOS is a game-first / tinker-later operating system for what the community dubs retro-handhelds.

It had its debut on the RG35XX, became somewhat of a sleeper hit there, and allowed me to really figure out what people like and dislike about these handhelds and how to make them better.

It had one weak point however: it was heavily RG35XX-specific.

From the way it scaled its framebuffer to the way it handled controls and inter-process communication between the OS components, every component was custom made for the handheld in question.

I quickly realized that the way the OS was built it was doomed to repeat the mistake several other distributions had made, becoming too device-specific, requiring custom builds for every new device and making a truly cross-platform experience impossible as every new device to the market posed a potential porting / maintenance problem.

The new work-in-progress GarlicOS 2.0 rewrite tries to address most of these problems by utilizing a few (somewhat nasty at times) tricks to become the first truly cross-platform retro-handheld OS out there.

The kernel source code / driver problem

It is no secret. There's good, bad and ugly handheld OEMs out there.

A lot of devices that get released to the market are great on paper, but fall short on one or multiple metrics. Be that a known hardware or software flaw, badly designed operating systems or some other issue that draws the attention away from what should be a fun & game focused experience.

Whenever that happens the first place people go is the independent developer community, asking people like myself for help to fix OEM hardware & software.

To do so we often need access to the kernel source code to rectify key hardware or software issues, but we don't always have that luxury as some OEMs refuse to cooperate in such a way.

When that happens, devices are often written off as a lost cause as there's just no easy way to make them work or feel right, essentially turning them into e-waste.

Having done OS development for nearly two decades now I figured it was time to address these issues once and for all and find a way to make things happen within the limitations set by these bad examples without sacrificing cross-platform compatibility or maintainability in the process.

This is the core of what GarlicOS 2.0 hopes to achieve, blowing new life into hardware that would otherwise fall through the grid because of questionable OEM ethics.

Setting minimum requirements

To achieve the aforementioned, GarlicOS 2.0 operates within the following requirements:

• It must be possible to run without modifications to the stock kernel
• It must be bootable from an ExFAT MicroSD card (to simplify Windows, Linux & Mac file transfers)
• Devices that lack native MicroSD card boot or ExFAT filesystem support must have said support implemented in a device specific daisy-chained bootloader (it does not matter whether the filesystem support is provided via fuse or a kernel module)
• Required software or hardware workarounds as well as hardware specifications must be made known to the core OS by passing it "quirk" environment variables via the daisy-chained bootloader
• Operating system updates must apply to all supported devices by means of sharing the same core operating system image

By following said limitations I was able to build a operating system that works the same across all work-in-progress supported devices, allowing multiple systems to share a single OS MicroSD card.

This extreme level of cross-platform compatibility came at a sizeable upfront development time cost though.

In hindsight I realize I very much underestimated the upfront work required to make these long-term benefits happen.

I really can't thank you all enough for being so patient with me.

Conquering the hurdles of cross-platform compatibility

Forcing several devices to share one common operating system has always been somewhat of a dance on razor's edge for embedded hardware / mobile SoCs.

Google pioneered this idea with its general system & kernel images starting with Android 8.0, but we've never really seen such an effort in the embedded handheld Linux OS sphere otherwise. (That I'm aware of.)

Doing pioneer work like this was bound to pose hindrances, I won't list all of them here but to give you an idea here's a few for all of you to wrap your head around:

• Building an easy to follow porting template / bootloader blueprint
• Dealing with different framebuffer drivers, scalers, soundcards, gamepads, etc.
• Adding support for screen rotation in a cross-platform / porter friendly way
• Reverse-engineering non-standard gamepad interfaces (*cough cough* Retroid)
• Reverse-engineering non-standard soundcards (*cough cough* Unisoc)

You might have noticed that I highlighted one of these points.

That is because that point is still being worked on / giving me trouble.

Bringing us to the next topic at hand...

Unisoc SoC support (and its audio co-processor)

When I first announced that I'd be working on a cross-platform compatible version of GarlicOS I showcased a Anbernic RG405M running the new operating system.

This device was chosen as the poster-boy for a reason.

You see, Unisoc SoCs (previously called Spreadtrum) are quite capable, especially when you factor in their ridiculously low buying price which made them very attractive to device OEMs for use in their retro handhelds, which in turn spawned a variety of affordable, yet powerful, different form factor devices... which is great!

But there's a catch.

Unisoc SoCs are by far the most locked down / proprietary mess I've ever seen with zero public documentation or Linux OS support, which meant I had to do all of the early adopter work myself if I wanted to run a regular Linux OS like GarlicOS on it.

It didn't take long to get the basic hardware running: screen, gamepad, etc. all posed no real problem. Audio output however was a different story.

You see, all audio output on a Unisoc SoC is handled by a proprietary AGDSP co-processor.

On a normal system audio is abstracted by ALSA (the Advanced Linux Sound Architecture) which in turn provides mixers & controls that the regular CPU / OS can utilize to "turn the knobs" and play audio.

While this is partially true here as well, there's quite a few limitations to playing audio on Unisoc devices using ALSA:

• The AGDSP co-processor requires a custom initialization sequence
• It runs on a non-standard period size (which causes stuttering with unpatched ALSA libraries)
• It can only run via MMAP (and MMAP support in ALSA is experimental / buggy)
• It can only run in non-blocked mode (blocking I/O is not supported at all)
• It has no buffer underrun / overflow protection and requires constant sampling
• It lacks pause (start / stop) support
• It panics the kernel whenever a underrun or overflow situation occurs (which means the device reboots by itself randomly)

That last point is what has been keeping me busy as of late...

If I had to explain it in a way everybody can understand: Unisoc feels like a platform that's being held together by tape & glue, its a platform that shows promise due to its affordability and raw performance which, for me, makes it a platform that's worth supporting, especially with its huge adoption rate.

But I also don't want to deliver something that's underbaked or buggy, as such I figure its better to let this one cook in the oven a little longer.

Please have patience with me.

How GarlicOS will be released and monetized (because people asked)

GarlicOS 2.0 will be available for free with all of its source code available on GitHub from day 1 of its release, including bootloader templates and bootloader reference implementations for all currently supported devices. (Yes, it will be a multi-device release.)

I do need to keep the lights on somehow though, so monetization will always be a necessary part of large-scale developments like these.

I think I've found a way to strike a good balance that should please both supporters & regular users alike though.

Newly developed pre-built bootloader images will be released regularly post initial release, bringing new devices into the GarlicOS fold over time.

These will be available 30 days early for supporters before going public on GitHub, including the bootloader's source code.

The cross-platform operating system image itself will, at no point in time, timely limited or not, be part of any monetization model whatsoever.

This choice has been made to ensure that existing GarlicOS users will never be left behind on an outdated build, having free access to bugfixes and OS improvements as they land while still remaining financially viable.

1:1 support via the DMs is no longer viable (too much work, too little time)

In the past I used to provide 1:1 support via Patreon DMs, but the huge influx of people needing help, combined with the increased workloads I have to push, both on and off Patreon, makes it impossible for me to continue doing so.

Messages come in faster than I can work them off, and if I tried I'd essentially have to stop working on the parts that really matter: the projects I want to work on for all of you.

I understand that this makes working out these projects a somewhat less personal experience (which sucks, because deep down I am a community focused guy and I love interacting with all of you), but unless I can clone myself somehow I just don't know how to squeeze it all into the limited time I have available to get things done.

I highly suggest that all of you join the RetroHandhelds Discord server if you haven't already.

Its a cozy place where like-minded people like us gather to support each other and talk about our niche handhelds.

It's also where I lurk while working on all of these projects, so if you absolutely need to catch me directly for one reason or another then your chances of doing so are quite high there.

Just keep in mind: I will be working most of the time and while I will always answer if I can, expect delayed responses as I'm not constantly hovering over my inbox while I work.

And finally: A thank you to all of you!

It's been a long time since my last post, and writing this newsletter just made me realize why its been so long.

Every time it was time to write a newsletter I asked myself... where would my time be better spent? Working on whatever the current project on the to-do list was to move things forward or writing about it to keep you all in the loop?

More often than not I chose to work on the project itself, simply because I knew that putting it all into words would take hours, if not days to sound right and get the most important points across.

It took a week to write this newsletter, I wrote it in 20 minute gaps in-between work, and even now, as I'm finishing the last few paragraphs, I feel like I could spend another week just to flesh it out more and fill you in on even more details, but it is what it is.

I can't thank you all enough for being there with me for the ride and supporting me along the way, these projects wouldn't be possible without all of your generous support!

Wish me godspeed. I'll keep you guys posted.

- Black-Seraph