Principles for being a good software engineer

You will NEVER BELIEVE these FIVE PRINCIPLES that will INSTANTLY improve your SOFTWARE DEVELOPMENT GAME

I will define software engineering for this post as writing code collaboratively to build something people will use. The target audience is people lower in software engineering confidence, as most of this is internalized as intuition after enough successful practical experience.  

Be good at logical reasoning, formalizing stuff, algorithms

I’ll start with the most obvious skill, which makes up much of what it means to be good at programming. It’s helpful to be able to, given a business problem, break this down into chunks and reason about relevant bits of logic in a systematic way. Furthermore, having a good intuition for the computational complexity of algorithms and how these are employed to solve different software problems will help you write performant code.

You can improve this significantly through practice and study, for example, learning basic CS concepts like algorithms and data structures, doing leetcode-style coding problems, and trying to formalize models or ideas you have as code. (There are diminishing returns here - you don’t need to be particularly good at competitive programming or solve IOI problems.)

Cultivate “mechanical sympathy”

I first heard the term “mechanical sympathy” in a Signals and Threads podcast episode about electronic trading that emphasized the importance of gaining an intimate understanding of the hardware you're working with. By comprehending its physical properties, functionality, and limitations, you can more effectively determine how much you can optimize your programs and evaluate the trade-offs between improving code performance and upgrading hardware.

This principle also extends to higher layers of abstraction. Even if your work primarily involves high-level programming languages like Python, it is beneficial to grasp core concepts such as memory management, network protocols, and how databases work. This foundational knowledge enables you to better compare approaches to the same problem, debug code, and understand what is and isn’t a fundamental limitation.

Write human-readable code

Code is written for humans, not machines. Sure, the machine will read the final compiled binary, but programming languages are designed to enable humans with their meat brains to make sense of stuff. The whole reason programming is feasible at scale is because of how human-friendly it has become. It’s almost a domain for the “verbally inclined” these days.

If you want to be able to maintain your project or collaborate with other programmers (which you do because your output can be scaled up with more workers), you need to make your code easy to read for humans. I don’t need to write about how here, as there are many resources on this, but I’m referring to the typical advice such as sensible naming, code splitting, and using standard design patterns. 

Persevere relentlessly in implementation, consult and deliberate on design

I see many people first learning to code get stuck with some minor bug, error message, or issue with installing a library and give up too fast without searching for a solution using all the available tools.

Today we have so many ways of finding information. Often, the best option is GPT4 or another SOTA LLM. Unless you are building something groundbreaking that involves fundamentally new technology, whatever problem you are having has a solution that can be found by simply asking an LLM, googling, prompting GitHub Copilot, etc.

You should have a very strong prior on “this problem is easy and solvable” because most day-to-day implementation issues in building software are. Note that I say implementation issues. Not design issues. An implementation issue is a problem that arises when you already know the high-level software architecture you are aiming for and are in the “actually writing the code that does the thing” stage. Examples of implementation issues are “How do I parse an Excel file in this programming language,” “How do I call this API repeatedly without having rate limiting errors,” and “How do I import this library.” You should never allow yourself to be blocked by these trivialities.

Just have a rule: "If this seems like a problem others have solved and it’s not a design problem, I will persevere and find information on the internet to solve it.” By doing so, you will learn valuable facts along the way and build up the general skill of leveraging AI and search tools. And to reiterate, use LLMs! Use all the tools! Make your life easier! The more grunt programming you outsource to tools, the more you can focus on learning fundamentally important things, such as how hardware works and core computer science concepts. 

In contrast to the point on implementation, when it comes to software design, you should not a priori assume your approach is feasible or promising. Whereas it makes sense to mostly trust LLMs, googling, etc., when it comes to low-level implementation details, as of today (June 2023), AI tools are not quite there with making optimal system design choices, particularly when it comes to optimizing for ease of development, performance, and scalability. In particular, ChatGPT suggests the “most popular / written about” approaches, which do not always correlate to the best option.

Furthermore, in the broader sense, design involves considering what you or your customers want from a piece of software and thinking about the problem with your product manager hat on. I find it helpful to talk to others about these types of problems, be that peers, coworkers, or users. Software design is the “hard problem of software” and benefits from collaborative whiteboarding, discussion, consideration, and comparison of several alternatives. I would suggest brainstorming as many high-level designs as possible that could help you reach your goal before choosing a path to pursue.

Deal with frustration

Software engineering can be frustrating, especially for those coming from a more “pure” math or science background, because you’re dealing with a bunch of suboptimal abstractions designed by humans, for humans.

You’ll likely need to wrangle a bunch of libraries and frameworks that made arbitrary design choices that you are forced to learn and deal with. Using them is still often more efficient than implementing everything from scratch, so it’s sometimes necessary to learn information such as how React works and how to use Pandas (less so now than before, given LLMs though!).

Remember that if your goal is to deliver value via building a useful tool or product, how elegant your approach is only matters to the extent that it enables future progress. My advice is, “It is normal to feel frustrated when writing software, and your ability to tolerate it can be a comparative advantage.” In addition, many frustrating things like learning frontend frameworks, deploying stuff to AWS, or figuring out how to make a plot with matplotlib will still convey generalizable patterns that make doing the next similar thing easier. But also - deal with it. You’re basically a plumber anyway.

---

The future

In some sense, I expect this content to fall into a similar category to guides such as “11 Most Important Blacksmithing Tips That Beginners Should Know” and “The Art of Commodities and Futures Hand Signals” within a few years. Because something something AI will automate the vast majority of software development. That said, a more general version of these principles will still hold:

• Think logically and systematically about the problem

• Understand how the underlying tech fundamentally works

• Be good at communicating ideas to humans

• Know which problems are easy vs. hard

• Persevere and be willing to do grunt work where necessary