Self Publish or Self Own?

Today on LinkedIn, I announced that I plan on publishing some short books about topics near and dear to my heart.  I alluded to this briefly in my last post as a means to generate potential leads for consulting and, maybe, impart some knowledge to the community at large.  I'm trying to keep my books between 25 and 75 pages, depending on the scope of the material (that's approximately 18K words at the high end).  

If you're interested, you can find more details about the endeavor in my post.  However, for this post, I'm going to post my first day's output (1060 words) verbatim.  I'm writing the book in Markdown so I can easily convert it to HTML or PDF, so some of the formatting may be unusual, although I've tried to do some light formatting to meet Blogger's standards (no I haven't).  Also, I haven't done any editing, so typos and sentences that go nowhere are a real possibility (although that's a possibility when I have done some editing).  

I haven't yet decided if I'll post the book content here with any frequency.  The main argument I have against doing so is that a lot of it is stream of consciousness right now, and the structure of the entire book may change significantly before it's finished.  As a result, I don't want to subject you, my readers, to the madness behind my creative process.

Anyway, without further ado, here's the first raw excerpt:

# Intro

## Disclaimer

Before I get too far in touting my own bona fides about my scalability knowledge, I should define the scope of my familiarity - namely, the good ol' fashion web app (or mobile app backed by server

side APIs).  A lot of this advice is a good starting point for critical or extremely large systems, but Google's systems, or high-frequency trading systems, or code that reliably needs to run a 

pacemaker have much different requirements than the vast majority of software systems out there.  I'm writing about the vast majority of software systems out there.

## Why Write This?

I've been lurking in the software engineering world for over 20 years.  I started out as what would now be classified as a full-stack engineer, writing software for personal digital assistants

(Remember those?  No, I didn't think so).  From there, I moved on to tech support and professional services for a very complex (and very fickle) payroll application.  After a couple of years 

attempting to understand the intricacies of COBOL, I landed a job as a software engineer at Orbitz, the online travel agency (OTA).  It's in the OTA space where I've spent the majority of my

career.

When I first started at Orbitz, I was very much a throw-it-over-the-wall type of engineer who didn't think much about scalability or reliability.  A few years into my career, I inherited the maintenance

of our service discovery framework (this was before the days of Consul or Kubernetes, so a bespoke service discovery application made sense for organizations).  Much like the aforementioned payroll app, 

our service discovery framework was very fickle.  Unlike the payroll app, I was the only software engineer in charge of keeping it up.  

That's not to say I didn't receive a lot of help.  Because I was now thrust deep into the operations side of software, I made fast friends with the Orbitz equivalent of a NOC and with the SRE/DevOps 

teams.  My 2+ years as maintainer of our service discovery systems was brutal, but I learned a lot.  Principally, I learned how to think about writing software from an operational perspective as well

as a functional perspective.

That education served me well.  I've since gone on to lead teams that needed to scale traffic capabilities 100x and needed to transition to an environment where much of the operational functionality

is commoditized and, in the best of circumstances, written in the Holy Grail known as *infrastructure as code*.

Well, that's a perfectly satisfactory explanation of how I got interested in scalability and reliability, but it doesn't explain why I'm writing this book.  Simply put, people still struggle with 

building scalable systems.  Even seasoned engineers.  I don't think anyone struggles with these concepts because they're not capable of understanding.  I think many times a confluence of factors

prevents an organization from building scalable systems.

*[Side note - I'll be using scalable, reliable, and performant interchangeably throughout this book.  While there are distinctions between each of the three characteristics, a system needs to 

be successful in exhibiting all of these characteristics in order to meet users' demands.  So, I can either write all of them out (I will when the distinction matters), create an annoying acronym

like SPR, or simply conflate the terms with the expectation that you understand they go together as part and parcel.  I'm opting for the latter.]*

Often, scalability is an afterthought, like security or accessibility.  It's not at the forefront of most developers' minds during feature development.  In many cases, devs don't even need to 

worry about it.  Systems are so powerful now, they can often paper over problems that wouldn't have surfaced.  And, as much as the DevOps movement has rippled through software engineering, there's 

still a bright line between development and operations no matter how many times management trots out the phrase "you build it, you run it" or a similar derivative.

Combine that with business pressures to ship, even in cases where scalability takes a back seat due to time pressures or cost ("that's tomorrow's problem!"), and the fact that most scalability

puzzles are unique, and you get a lot of scalable infrastructure built on hopes, prayers, and denial.

My intent with this book is to provide some tips that make scalability, well, more scalable.  Where possible, I'll provide concrete examples, but every system scales differently (and scales

differently at different times in its lifecycle), so I probably won't be able to provide a canonical list to get your system scalable from A-Z.  Sorry!

But I should be able to provide a clearer methodology for approaching scalability concerns, whether or not that's designing your system to be more scalable from the outset or to help identify and 

solve scalability issues after the fact.  As a (mildly) somber note to end this section - most of your scalability concerns will appear after the fact.  If you try to build too much resiliency into

your system from the outset, you'll probably overengineer it and, paradoxically, reduce your system's scalability.  There are a few simple things you can do to head off the most egregious 

repeat offenders, but remember that *every system scales differently*.  Yeah, it's scary if you think, regardless of how much you plan, you're still going to fail.  OR, you can see it as freeing if

you realize that you don't need to stay up all night attempting to summon every mode of failure that can haunt your system.

Let's get started.

## Try To Identify Every Possible Mode of Failure

Wait, didn't I just say that attempting to enumerate every possible mode of failure is, well, impossible and nothing more than an exercise in mismanaging your blood pressure?  Yeah, I did, but 

scalability is like that - there's always a sweet spot in between doing too much or too little.  Miss that sweet spot, and your system is bound to fail before too long.  Also, know that, regardless

of how much experience you have or how diligent you are, you'll miss a sweet spot.  And then another one.  And then another one.  So just relax and accept that you'll fail.  *[Sheesh, who knew

that scaling systems would also be an exercise in fatalism?  Just look at it this way - one day, the heat death of the universe will occur, and then you don't need to worry about modes of failure, 

because ALL systems will have failed.]*

...to be continued

Until next time, my human and robot friends.