No, I'm not getting ready to hack off my arm in a desperate act of survival. I've now crossed the 100-hour threshold for using an AI Agent to rewrite my personal financial site. Before I'd waded into the deep end of the project, I was targeting 120 hours for site completion.
Why 120 hours? Because if I didn't finish in 120 hours, I was going to cut my arm off in frustration. Or, alternatively, it was a good benchmark for determining what amount of effort I could devote to a theoretical client without burning myself out and charging a reasonable rate. My line of thought was thus: A standard work month is 160 hours (40 hours by 4 weeks). If I budgeted 120 hours for working on the site and 40 hours for client meetings, design reviews, and rework, then I could fit the work into a neat little package of one month. Well, really two, because 160 hours across two months gives me room to adjust if my package dimensions aren't reasonable.
And, if I can reliably crank out 6 of these in a year, then I'm on my way to making a decent living. Depending on the functionality I delivered and the time needed to do it, I was thinking of charging said theoretical client $5K to $10K for my troubles. That's probably low, but everyone is treating software engineering like a commodity now, so it's a number that doesn't depress me (more on the commodity thing shortly, though), while still falling in a budget-friendly range.
Well, it turns out I've crossed over the century mark and am about to blow by my golden metric of 120 hours very shortly, with maybe a 1/3 of the project finished (so, I'm recalibrating my target to 360 hours).
You'd think this would bum me out, but no. I'm pretty happy with what I've accomplished so far, and that I'm finally polishing my procedures for interacting with a coding agent to keep my sanity and make it as productive as possible while keeping my sanity (yeah, I said it twice, but try spending half a day with a fake human every day and see how sane you feel at the end of the week).
Here's a short synopsis of what I've been able to do in about 110+ hours of design and development:
- Create a cohesive site design system with pages for transactions, budgets, accounts, and investments.
- Implement basic functionality for all of those pages.
- Hook up to a 3rd party platform to download financial transactions for bank accounts.
- Build a deployment strategy and production platform for my site.
- Implement LLM functionality that uploads investment statements and extracts relevant details with little to no rework on the user's part.
- Write testing for much of the functionality with an ultimate goal of 80% coverage.
- Significant code refactoring to massage raw AI slop into Todd-approved slop.
What's left?
- Augmenting and refining the functionality above.
- Adding more refinements for security and scalability.
- Some additional large features like net worth tracking and financial modeling.
- Accessibility and security audits.
There's still quite a bit left to accomplish, and much of it is commonly known to haunt software teams. But, the fact that I've been able to make this much progress in less than 1 work month, as one engineer, is pretty impressive.
If I step back and realize I have a good shot at creating a fully functioning personal finance application in 3-6 months with a complete company staff of 1, I'll certainly admit that I'm in awe.
"But of course," you say, "that's what all of the AI gurus have promised us. Functioning code with a few deftly worded prompts."
Well, I'd say stop believing everything TechCrunch, Axios, and the New York Times tell you when it comes to AI.
Coding agents can certainly write code faster than I can, but the quality of the code is still a major concern, as is the "reasoning" behind the coding decisions.
Here's one of the myriad examples of why JohnnyBot can't code independently:
When pulling information out of a database, it's not unusual to name the field keys using snake case (snake_case_looks_like_this). However, when passing information via JavaScript, the convention for field keys is camel case (camelCaseLooksLikeThis). There's a place in my code where I'm trying to feed code from a JavaScript source or a database source. With the exception of how the variables are named above, the field names are exactly the same (my_variable vs. myVariable), the data they're associated with is the same, and the data will be stored in the same place in the same format.
So, there are two options - recreate the read and write operations for snake case and camel case, even though the logic is nearly identical. This is an agent's modus operandi. It can parse code in milliseconds, so it doesn't care if the volume it sifts through is double (or more). But this approach is brittle, because it's not unusual for humans or agents to make an update to one of the cases, but not the other. Telling an agent to be extra careful and double-check its work isn't useful (despite all the meticulous prompts people write, impelling an agent to be an expert as though it flips some magical switch), because it often ignores critical components of its instruction set with alarming frequency.
So, the best solution for both humans and Wall-Es is to normalize the keyset, so all operations are performed in one branch rather than one of two copies. Well, then, which keyset should be converted to the other?
At first glance, it doesn't seem to make much difference, but after a little thought (and knowing enough about how variables are named due to engineering experience), it's easier to spot and alter something that looks like '_a' and convert it to 'A' than it is to find 'A' and convert it to '_a'. There are still some edge cases that fail for both - specifically for abbreviations where several letters in a row are capitalized (like myLLM vs. my_LLM), but the logic is still easier to spot and adjust for the snake to camel case, possibly by also looking at the letter just to the right of the underscore, than to determine if and when something should be uppercase. It's much more likely to have a capital letter in a snake case variable (my_Name), even if it's odd, than an underscore in camelCase variables (which would automatically make it snake case anyway. Whoa!)
Again, this is a very simple case, and one of several that I encounter in every agent session, but it's illustrative that the position of software engineer is still very much needed. Without it, you're letting your agent slather verbose code on top of verbose code until, eventually, the entire application topples over under its own weight in some variation of a Jenga! game from hell.
Until next time, my human and robot friends.
Comments
Post a Comment