Databases Taught Me Everything: Design & Documentation (Episode 2)

The Broken Record

“We need better documentation.”

“The documentation is lacking.”

“Where is the documentation?”

These statements are repeated in one form or another everywhere I have worked. It takes a lot of effort to keep documentation up to date. Especially if changes happen rapidly.

But it’s worth it.

I have experienced firsthand the immense value of putting things “on paper” before diving in and creating. Making the effort to put your ideas in writing ahead of time helps eradicate “throwaway” code and is a catalyst for successful implementation.

Design Documents: Catalyst to Success

One project in my history comes to mind. There was an existing Oracle database that was created a few years before I arrived on the scene. It was working, but the company was anticipating a healthy amount of sudden growth. We knew things had to be updated to handle the traffic.

This wasn’t a cloud-based system. There was no simple scale-up operation. It was going to take a full-on redesign of core parts of the system. One requirement was to keep everything inside the Oracle database. We could not use any external tools or processes.

A small team set to work: a junior database developer, the senior database architect, and me. The senior architect knew the system well, but he didn’t give us any shortcuts. Acting as a mentor, he wanted us to learn the system well and come up with a solution. This was initially frustrating and ended up being magnificently educational—and very appreciated.

The project didn’t begin with writing code. There were no immediate schema changes. It all began with brainstorming and documentation.

With design documentation we were able to explain what we were thinking, in detail, all without “demoing” anything. We were able to work through all the use cases without ever writing a line of code. That’s not an exaggeration.

Words (and pictures) to Reality

The design process took about four months. There was sandbox-style research in there: playing with ideas in the environment. But, again, no infrastructure changes were made until the design concept was completed and approved.

Four months may sound like a lot of time to spend in meetings and using crayons and pencils instead of coding. It is. Sometimes it is painful. Sometimes it feels unproductive.

Trust me. It’s not.

Let me assure you, it is everything but unproductive. When it came to putting the design into action, the time spent in design paid dividends in development.

When it came to putting the design into action, the time spent in design paid dividends in development. We knew exactly what we were creating. We knew exactly how it was supposed to behave. We knew exactly how to test it. And we knew exactly what we had to validate.

Once development began it was only about two months to having it fully tested and functional in production. And only 1 bug was found, 6 months after release. And that bug was caused by a very specific edge case. Again, that is not an exaggeration.

This was no small feat. Doing a full replacement of core functionality where we had extremely low tolerance for failure (both functionally and data-integrity-wise) with minimal downtime takes very careful planning.

Post-release Dividends

The design documents continued to add value well after the release. There were scads of instances where people referred to it for many purposes. Whether it was for integration or to investigate unexpected results, the documentation proved invaluable.

There were some things that were slightly outdated, but it wasn’t enough to render it invaluable or useless. And it was simple to update when old information was found.

I wish I could say I have treated every project the same. But I haven’t. There are always reasons for it, and most of them are invalid. Excuses would be a better term. And where I haven’t had the discipline to document well I found the most frustration. More code rewrites. More missed use cases. More questions afterward with hard-to-find answers. More bugs.

Breaking the Broken Record

If there were only one thing I could encourage you in, it would be this: Document everything and design meticulously. It makes all the difference in the world.

Break the broken record. As much as it depends on you, don’t give people a reason to bemoan the state of documentation. It is a valid complaint. It can be remedied. And you can be part of the solution.