🔗 Dependency management in database design

Codebase organization and design is a vital skill, and a skill I was never taught it in school and had to pick up in the industry. I suspect it wasn’t taught for two reasons:

1. Codebase design is a soft skill. Even when you can recognize a well-structured codebase, opinions differ on how to make code organized. It’s more art than science. Just look at the long list of incompatable practices on Wikipedia’s List of software development philosophies
2. The importance of codebase architecture is much more apparent with larger, more complex code bases. Most of the code written in academia may never reach the level of complexity where good organization matters.

Our project, Dolt, definitely meets that threshold where organization matters. Dolt is the first SQL database with Git-style branches; it’s revision control for your data. And we’ve put a lot of work into it: the Dolt code repository contains 762k lines of Golang code (excluding generated files), broken up into 204 different packages. We also make go-mysql-server, the SQL engine used by Dolt which itself consists of 475k lines of Golang code in 59 different packages. The code is ten years old.

That’s way more code than any one person can keep in their head. Breaking it up into packages helps, but how did we determine what the packages should be, and what code goes in what package?