Why I decided to start a fresh project after contributing to and eventually maintaining NextcloudPi for the last years.
-> Skip the personal stories, I want to go right to the technical stuff
Let me lead with the story, how I got into self-hosting in general and hosting Nextcloud in specific.
In some aspects, my story will probably sound familiar to many of you who are into self hosting. I had just started my computer science studies, my server and Linux experience was negligible and I had little money, but lots of motivation and a 10 year old notebook of my parents.
First, I tried and failed for some weeks to setup a very minimalistic distribution by the name of "Slackware" - because that's what came up when running a search for "light-weight linux distro" (me, being a Linux noob, not knowing that any Linux distribution is comparatively light-weight if you need it to). But soon enough, I shifted to a pretty standard Debian stable, set up a NginX, PHP, MariaDB and,finally, Owncloud (yes, Nextcloud didn't exist back then).
text-subtitle-1
I apologize to any Slackware folk who might read this, because I really don't know much about the project and can only share my experience and impression as an absolute Linux noob back then. If you want proper information on the project, please head over to the official website.
I maintained my Owncloud server like this and then on a Banana Pi M2 for a few years and while I enjoyed the self-hosting experience, I well remember the regular fights with failed updates, ill-configured php and nginx settings, backups that hadn't worked when you needed to restore something.
When in 2016, Nextcloud was forked, I had to make a decision with which Fork I wanted to go. After some hesitation I went with Nextcloud and migrated my setup over. Not too long after, I replaced my Banana Pi with a Raspberry Pi 3 - and while I was preparing the migration, I found out about NextcloudPi and the amount of work and dread such a migration causes, probably led me to the decision to give this "managed" solution a try.
NextcloudPi is a ready-to-use image and/or install script for Nextcloud including many features, like backup & restore, dynDNS, Certificate management etc., configurable from a administration web interface.
And what can I say, I loved what Nacho had done with this project. NextcloudPi got me going from dd if=<> of=<> to setting up my Nextcloud Clients so much faster (and tbh, with better results) than I could have done on my own. And while I'm quite happy about all the things I learned during while building and managing my previous setup, I'm very thankful for the reduced maintenance effort I enjoyed from this point on.
Reduced maintenance effort, however, didn't mean I invested less time into self-hosting. At least not, once I began making my first contributions to NextcloudPi, starting with the initial version of NextcloudPi's localization. Investing some of the hours I saved in hosting effort into new functionality and bug fixes of NextcloudPi was a great opportunity and I had a lot of fun working with Nacho and am very thankful to this day for all the things I learned during this time.
However, there came the time where Nacho decided to step down from the role of maintainer after tirelessly filling it for 5 years. Very soon it was clear that I had to make the decision whether to take the project over from him, because at the time there was no other long-term contributor who knew it as well.
I gave myself a week's time to make up my mind, before I finally said "yes". The following months were a bit bumpy, as there were still many parts of NextcloudPi that I didn't know all too well, among others resulting in a critical bug in my first ever release as core maintainer.
Even before Nacho left the project, but all the more so from that point on, I therefore invested a lot of effort into the robustness of the NextcloudPi release process, expanding a lot on CI/CD, i.e. automated building and testing (for all build artifacts generated), staged rollouts, a guided and systematic approach to user testing supported by active members from the NextcloudPi community.
Still, every release carried a good amount of anxiety for me, because albeit all my testing efforts, it was just impossible to ensure that nothing was overlooked and I wouldn't break the installation of any of the thousands of users of NextcloudPi.
Soon after taking over maintenance of NextcloudPi, I realized that a lot of work would be needed to get to a point where I would feel confident about releases again. The reasons for this where manifold, but most boil down to something called:
NextcloudPi was created by Nacho 8 years ago and when I took over, it was already 5 years old.
NextcloudPi had basically started out as a bunch of bash scripts automating the actions that a server admin setting up Nextcloud would otherwise perform by hand. Mind you, that was at a time, where a level of automation like that was far from as ubiquitous as it is today, where cloud applications are shaping how we as developers, devops engineers and system admins think about software and hosting.
Containers, namely docker, weren't as popular, I at least had never heard of cloud-init or ignition files and hadn't made my first experiences with infrastructure-as-code tools like Ansible or OpenTofu yet, that allow you to approach hosting service in a similar fashion as NextcloudPi did, i.e.: describing, in code, what you want to setup and then running it reproducibly on as many systems as needed.
While bash is an obvious choice as a tool to automate administration tasks, in my opinion, NextcloudPi at some point exceeded the point where this was the right technology.
My personal (half-serious) rule of thumb is: Once you start writing libraries in bash and actively consider writing a unit test suite for it - then it's probably time to shift gears.
Today, NextcloudPi features more than 14000 lines of bash and that's more than is healthy for a code base, if you ask me.
As already mentioned, one technology that has drastically transformed how we think about running software today, are containers. Powered by some very cool features of the Linux kernel, containers allow admins, to build think about how to run each service individually. Even better, in most cases, the responsibility to setup these environments can be shifted to the developer of the service - I mean, they should know best how to run their software, right?
NextcloudPi comes from a time, where containers were less known and therefore had a different, more traditional architecture. Like it has been done for decades, every service was installed into the host system. This has the disadvantage of increasing the risk of configuration and depedency conflicts and making upgrades a way more delicate matter.
Basically, if anything in the system is upgraded, there's a chance to break something else. NextcloudPi has a lot of moving parts and optional services, so it was hardly feasible to test every possible configuration for each update.
Instead of using containers internally, Nacho added support for a NextcloudPi docker container at some point, which was a container containing the whole Nextcloud installation including all services. This made it particularly difficult to even consider using containers inside NextcloudPi. While it is not impossible containers inside docker containers, it has a lot of pitfalls and would vastly increase the configuration hell that made NextcloudPi so hard to test in the first place.
Therefore, I made the decision to abandon support for the NextcloudPi docker container in order to pave the way for internal usage of containers. My hope was that, by migrating most internal services into containers, I would be able to hugely improve testability and robustness of the system.
However, I eventually gave up on these plans, because it turned out to require huge changes to NCP's internal architecture and build system and still wouldn't solve one of the most painful issues with NextcloudPi updates:
The single thing most likely to break user installations throughout the history of NextcloudPi were likely upgrades from one major version of Debian to the next. The reason for this is, that an upgrade like this affects everything installed in the system. The last distribution upgrade script I wrote was tested extensively and confirmed working on many (test and real world) installations - just to break two weeks after release, because a new version of a package in the debian package repositories was released that caused a dependency conflict with another package.
Things like that are impossible to prevent for me as maintainer. Even if I would run automated tests, say, every hour, I had no chance to react fast enough if anything breaks to prevent the broken automation to be executed on users' installations.
Similarly risky were PHP upgrades, because they required 3rd party repostories and affected both some of the tools used in the automation scripts (notably, Nextcloud's cli occ) and the NextcloudPi admin web interface. These issues could, in theory, be much improved by using containers though.
So, as may already have become clear, I've increasingly started to doubt, whether there is a future for NextcloudPi where I'll be able to keep the maintenance effort low enough to fulfill this role sustainably.
At the same time, my own skills and knowledge about all sorts of things related to writing and running software had been growing a lot over the last years - and so have my standards as devops engineer and developer. I've been professionally managing virtual machines, containers, kubernetes and serverless deployments, cloud and on-premise software and more. And increasingly I asked myself "How would I build something like NextcloudPi if I started from scratch today?". NextcloudPi didn't get any worse over the time, but my standards and expectations increased.
And so, I finally decided to take this leap and start building Nextcloud Atomic, my new project that I hope will eventually become a worthy spiritual successor to NextcloudPi and even greatly extend on that.
Like NextcloudPi, it will be a disk image for VMs and bare metal installations that carries a custom linux distribution with everything included that is needed to run Nextcloud comfortably.
Other than NextcloudPi, it will be featuring mandatory disk encryption, atomic and fearless system and service updates, heavy use of kernel sandboxing features and containers and it will be integrating the official multi-container setup of Nextcloud, Nextcloud All-in-One, letting me benefit from all the work that is done by the AiO community and focus on the things I have to contribute: disk, certificate and backup management, updates, monitoring, NAT traversal ... - the list of planned and wished for features is long.
NextcloudPi will not be abandoned yet. I intend to bring Nextcloud Atomic to a point where it is a viable alternative for the vast majority of NextcloudPi users before I drop support for NextcloudPi. And of course, I will work on providing a robust migration path.
Thanks so much for reading and for being part of this journey!