Just Sweep It Under the Rug

 

What could go wrong?

After singing the praises of what I was able to accomplish in 50 hours when creating Slowpoke, it's time to visit the other side of the balance sheet - what did I miss for the sake of expediency (or laziness)?

I hope this post serves to enumerate why software engineering is still a valuable profession, even in the age of vibe coding.  It's possible that with the latest version of Claude, I could've completed this in 10 hours rather than 50 (although every post I read, even among the staunchest LLM proponents, reiterates the models' need for expert supervision).  But, even if I were able to complete the functional requirements in less time, making it operational and scalable still takes a lot of hard work.  It's not an exaggeration to say that offering an app to a large group requires an order of magnitude (or more) worth of effort than the original proof of concept.

Granted, some of that effort is a hydra.  The software engineering industrial complex becomes so big that parts of the machine are just there to support other parts of the machine. Keeping that in mind, much of the work that teams do, however unglamorous, is the foundation of a stable product, if not a stable internet.  The complexity of maintenance is always in the details.

So, what noble software engineering efforts did I leave out in my most recent project?

Multiple Users

There are a lot of libraries that make using social providers like Google, Facebook, LinkedIn, or Twitter easier.  Writing login logic is a tedious, error-prone task that exposes yet another way for users to leak data. So, why not let companies that already manage accounts handle all of the heavy lifting?  

In my case, I used a library called arctic to handle this.  I only enabled Google, because that's my go-to login, and I kept it in development mode.  By keeping it in development mode, I'm only allowed 100 test users, and I don't have to certify the app with Google (though I assume that the certification process isn't too onerous, given the overwhelming amount of crap on the internet).  I've also got code that verifies I'm only permitted to use my credentials to access the site.

Because this isn't technically in production mode, I frequently receive renewed requests from Google auth to log in that could otherwise be bypassed, and which would annoy users if released into the wild.

If I opened this up to multiple users, I'd need to enable better scalability to allow everyone access to either a personal section of my own Google Sheet or route them to their own sheet, which could be problematic if people change the sheet and the application code no longer recognizes the format, because there's no schema to enforce structure.

I'd also need more configuration options (and hence more features and code), since people probably want to categorize their budgets differently than I do.

Scalability

This is the big one.  I'm using Google Sheets, which has severe limitations if the amount of data grows too large.  The current APIs don't allow an application to fetch a subset of data like SQL-based databases do.  Instead, I need to fetch all of the data and manipulate it.  Given that the number of transactions I add over the course of a year is ~1000, this isn't an issue, but multiply that by 10, and the amount of time to load the data or the memory needed to display it begins to be problematic.

With the support level (free) I have for Google Sheets, the number of calls I can make to my data store would also be an issue.  The quota is enough for an individual maintaining a budget app, but it would quickly be restricted if I were to serve this app to multiple users (even as few as 10, possibly fewer).

The same applies to the level of support (free) I have with Cloudflare.  Cloudflare's free quotas are pretty generous, but not enough to run an app like this at scale (though it would probably allow around 100 users before hitting a paywall).

What's interesting is that, if I were willing to pay a little money, using Google and Cloudflare are excellent choices to scale up without needing to perform explicit work to serve my app to, at least, thousands of people.  Google's ability to scale is well-known, even among the non-techies.  Millions, if not billions, of us use their services every day (Sheets, Docs, Gmail, search) and expect it to just work.  So, if you can leverage their offerings, you can go pretty far without needing to make explicit decisions to scale your systems up, which is good because it lowers development cost and complexity.

Ditto with Cloudflare.  For the uninitiated, Cloudflare is essentially the front door for the web.  If you're browsing something that isn't served up by one of the largest companies (almost exclusively Google, Meta, Amazon, Apple, and Microsoft), you're probably accessing a site via Cloudflare.  It acts as a host that stores (caches) commonly accessed data for a site at a location near you and ensures that data is quickly and evenly distributed across the globe when you make changes.  So, they know something about scale.

In both cases, you may have to make some trade-offs that match their opinionated engineering decisions (certain libraries cause slowdowns in Cloudflare, but, for me, that only required 2 hours' of maintenance to meet their standards), but those trade-offs are likely to be far less intrusive than attempting to maintain code to make the system scalable.  At a certain point, it's worthwhile for a very large company to have its own team - or teams - maintaining bespoke scaling infrastructure, but I'll wager that tipping point is way larger than most people are expecting (maybe exclusively the Big 5 I mentioned above - or close to it).

Reliability

I didn't focus on reliability too much here for a few reasons:

• I'm using this as a prototype to see how fast I can build prototypes.  Prototypes need to work for happy path cases, but they don't need to solve for all of the edge cases that production systems need to.
• Even if this is technically production, it's production for just me, and, at least in cases like this, I'm usually more patient with my own mistakes, so I don't need to worry about reliability beforehand.
• I'm borrowing heavily on solved problems like Google's auth, Google Sheets, and Cloudflare for deployment.  That reduces the number of systems I have to test.  Yes, those systems can fail, but at some point, you have to assume that the infrastructure you're relying on is capable and predictable.  Otherwise, you'll drive yourself crazy and overengineer your solution.
• My code base is, rather surprisingly, only about 1000 lines between HTML and JavaScript.  The smaller the code base, the fewer places for bugs to hide, and, therefore, the less need to write testing infrastructure.

With all that in mind, it's still not a bad idea for me to write unit tests to help with system design and testing.  However, like I said above, it's a small codebase leveraging a lot of existing libraries and infrastructure, so I'm less concerned that I don't have the existing test coverage in place than I would if my code base were 10K LOC or greater.

The Usual Suspects

And, as happens in just about every actual app in real life, security and accessibility are an afterthought.  Security - or at least some aspects of it - is handled via the existing components I've borrowed (though that's no real guarantee of security, but it keeps me safe from a few glaring self-owns like implementing my own login logic).  

Accessibility takes a backseat because I wanted to ensure I had the functionality correct (which is the typical excuse as to why it's never addressed).  What I love about Astro, though, is that, in development, it provides a menu with an option to audit accessibility.  So far, I've only got one warning to address, which is encouraging.  I'll need to verify the site with a few other checkers, but it's nice to know the design is otherwise fundamentally accessible.

Until next time, my human and robot friends.