The "Nate" Test

The "Nate" Test

  • By Nate Zaugg
  • President / CTO


You may have heard about "The Joel Test" before. It's a checklist of things that you should look for before joining another software team. This checklist was written more than 17 years ago! What might we change or include if we were to make a new list today?

Here is the original "Joel" test:

  1. Do you use source control?
  2. Can you build in one step?
  3. Do you make daily builds?
  4. Do you have a bug database?
  5. Do you fix bugs before writing new code?
  6. Do you have an up-do-date schedule?
  7. Do you have a spec?
  8. Do programmers have quiet work conditions?
  9. Do you use the best tools money can buy?
  10. Do you have testers?
  11. Do new candidates write code during their interview?
  12. Do you do hallway usability testing?

The Nate Test

While I agree many of these items have a timeless nature about them, I think the list could be modified (or altogether replaced) with things that are a little more important for the time. May I suggest the list may be revised to:

  1. Do you use a modern source control system, like git?
  2. Do you use an Agile methodology to build your software?
  3. Do you run a Continuous Integration (CI) server for your projects?
  4. Do you know and practice the concepts of SOLID and DDD?
  5. Is your code reusable?
  6. Is your code Unit Tested?
  7. Is it more important to do things the right way rather than do them fast?
  8. Does your team include all of the necessary roles?
    1. Does your team include at least one stakeholder?
    2. Does your project have a product manager?
    3. Does your project have a good team lead?
    4. Does your project have a good project manager?
    5. Does your project have a good tester?
  9. Does management give you everything you need (including tools, time, components, etc) to be successful?
  10. Does your company invest in training for everyone they hire?
  11. Does your company have both caves (quiet private space) and commons (open collaborative space) for their skilled workers?
  12. Do you have an effective interview process?

Do you use a modern source control system, like git?

Distributed Version Control Systems were not on the scene when this was written. In fact, I believe the post suggest CVS as the source control system. Git and other DVCS systems aren't an incremental improvement over older systems like CVS or SVN, they are a great big giant leap forward.

While discussing the advantages of git over SVN would be a pretty lengthy topic itself, the short list is this. The merge works and the workflow is better. These tools have features like fast forward merging, pull requests, squashing, and local commits. These features make a huge difference! Cherry picking just one feature (ha ha) i'd say that feature branching / pull requests are the big advantages git has. Imagine that the pesky co-worker who just can't get their stuff right before breaking the build? What if you got to choose if the code they wanted to commit was ready? That's big!

Do you use an Agile methodology to build your software?

Agile is a revolution in thinking when it comes to Software project management! While it may not work perfectly for every single project scenario, it's a great tool for the vast majority of software projects. Agile just has this way of getting everyone to pull in the same direction for good of the project. It takes a relationship where there is every incentive to try to get away with as much as possible and changes it into a relationship of belonging to the same team.

We run projects using Agile (where appropriate) and we will never do a fixed bid again! Seriously, it's not worth the headache. It doesn't matter if we doubled, tripled, or even x100 the estimate, a fixed bid project is doomed to fail. The reason is actually pretty simple. A fixed bid is assigning a known quantity to an unknown set of features. Yes, yes, I've heard it before -- we know exactly what we want. Well, it's just never the case! Besides, the requirements of a large project never pan out the way they may originally be planned. Things are simply not thought out with enough detail to be able to fix the scope enough that it would not change. With large projects, it's just impossible to keep that many details straight.

Do you run a Continuous Integration (CI) server for your projects?

While Joel suggests a daily build, I think that a build with every merge into master is even a better idea. This may happen every few minutes on a large team and this may not happen for days at a time on a small team. Some teams may even have codebases so large that this would be extremely impractical. The idea is that when a feature is ready to be pulled into master, there should be an automated system that builds, runs unit tests, and creates a package that can be deployed. Think more dev-ops and less daily builds and I think you'll get the idea.

Do you know and practice the concepts of SOLID and DDD?

Have you ever worked at a place where the answer to releasing the next version of something was to start over with a new version? Begrudged, management agrees to allow this project and undoubtedly years later this is the project that needs to be thrown away and replaced with a new version.

There could be a lot of factors that contribute that kind of atmosphere, but almost certainly they are not practicing SOLID! SOLID is a set of software principles that helps our code be more cohesive (the right pieces put with each other) and less coupled (dependent on external objects). Tight cohesion and loose coupling are the magic bullets for code reusability, and maintainability.

When you learned coding in school they didn't teach you about this stuff, but SOLID is the difference between writing code as a student/hobbyist and professional software. Remember, code is an investment and only professional code is worth investing in. If you have good architecture, and SOLID code, you are in very good shape, my friend!

Domain Driven Design (DDD) is challenging to define in a small paragraph (so maybe I'll write two or three ;) and my take on DDD is a little bit different than the author of the book of the same name. But let's look at one concept, Ubiquitous Language. UL can be very helpful for both the project manager and any future maintenance programmers. The idea is simple, use the same language between the requirements and the code. Don't attempt to "translate" requirements into something only the programmers are going to understand.

Imagine if someone could read a snippet of your code and understand it well enough that they could answer complicated questions about the concept? Having this kind of effective communication sure beats the misunderstandings that would surely come.

Is your code reusable?

Reusable code is more than just avoiding more work. It means that when you build something, it's the last time you have to build it for the enterprise (or perhaps for everyone, via open source software). Building a toolbox of code that is reusable can make you a very powerful developer indeed! If every new app you start for your company means coding another stupid login page, then reflect upon what the world would be like if you didn't have to write 1,000 login pages in your career?

Sure, you're going to have to write this code sometimes, and even when you're reusing the code it is going to need to be maintained. But changing it once for every app in your enterprise could be a huge savings. Plus, you can spend your time doing something way more interesting!

Code that is truly reusable is also well written. Keep this in mind the next time you think you're building something that cannot be reused.

Is your code Unit Tested?

When I explain to people that I write Unit Tests because I'm in too much of a hurry to avoid them, they usually give me a slightly confused look. "How can writing more code get your work done faster, that doesn't make any sense! You must be one of those Unit Testing fanboys that we never understand".

The explanation is simple. When I write a piece of code, I can usually point out a test for it in just under two minutes. Once that test is written, I can run the piece of code I wrote very quickly. I can make a change and run it again very quickly. No need to login to an application. No need to setup the UI in such a way that it's going to hit the case I've just written.

As an added benefit, let's say I refactor something large in the app. If my unit tests still run okay, I have pretty high confidence that I haven't introduced a bunch of new bugs as a result of this change. Changing code is tricky!

Is it more important to do things the right way rather than do them fast?

I once knew a developer who the business people thought was the best programmer they had ever seen! He was fast -- to be sure! The problem, of course, is that nothing he wrote was any good! Eventually it all had to be replaced and it usually comes at a much higher cost than writing it correctly in the first place.

In Software, sometimes you need to go slow now so you can go fast later. Working out the nuances of the architecture before you start building features into it is a smart thing to do. Having an enterprise architecture in mind before building systems will benefit the organization greatly.

The old story of the tortoise and the hare comes to mind, but it's not quite apt. It's more like taking the time to finish the motorcycle so you can cruise past the runners. The runner is going to get quite a head start, and comparatively your going to look like you're standing still, but it won't be too long before the motorcycle will pass up the runner -- for good!

Does your team include all of the necessary roles?

Having the right people on a team (not including the devs) is the most significant indicator of a successful project. Absent one of these roles, the project will suffer. Having experience in these roles is also very important.

  1. Does your team include at least one stakeholder? Having at least stakeholder means that there is buy-in for the project. It's one of the major factors determining success in a project. If you don't have a stakeholder attending meetings regularly then you have some major risk in your project.
  2. Does your project have a product manager? The product manager is the one who has the vision for the project. He knows how things are going to work, he knows whatever industry the app is being created for. He knows the answer to the difficult questions. Beware of a committee acting as a product owner. This almost always makes lots of problems in the project.
  3. Does your project have a good team lead? You need someone that is competent leading the effort. Otherwise your efforts may yield code that is obsolete on day 1, difficult to maintain, or worse! The project manager is going to have to place a lot of trust in this person. Make sure they are always honest in their estimations.
  4. Does your project have a good project manager? A good project manager means the developers can focus on writing code. They make sure the right questions are being asked and that the devs are doing the right things. Devs need to feel a little bit of urgency, and sometimes they get wrapped around chasing rabbits and do not know that they are stuck. A good project manager knows!
  5. Does your project have a good tester? The QA process for an organization will vary based on the cost of a bug. Your project will fall somewhere between reputation being on the line and people's lives being on the line. When it's just reputation it may be a good enough QA process to at least have someone other than the developer test to make sure a feature is complete. The key is that at the very least someone other than the person(s) that wrote the code should test it to make sure it is working.

Does management give you everything you need (including tools, time, components, etc) to be successful?

We've all had the feeling that we're being setup to fail before. It sucks! Especially since the blame usually comes our way anyway. Management that does all that they can to make sure the project is a success is the best morale booster of any other gesture -- including free lunches! Real software craftsman derive a lot of joy out of successfully launching apps.

Would it surprise you that management support is the single biggest factor to the success of failure of a project? Believe it or not, it's true! A company can have the best team of software devs but still fail to deliver software based solely on management. I've seen it! Over and over again! If the top (CXO level) of the company does not understand software and they do not offer the engineering team enough support, there will be many failures.

My advice, if you are starting at a company flat out ask how many of their past 5 software projects were successful, and whether or not they felt like they had the support of the top levels of management.

Does your company invest in training for everyone they hire?

My first job out of college was for a large software company. I remember how excited I was to start and to be able to write software full-time! I had so much enthusiasm and energy! It wasn't long after I started that I realized that although I had probably the least amount of experience of anyone on my team, I knew a heck of a lot more than many of them. The devs at this company were taken advantage of because they felt trapped.

Partly this is a problem where people don't have the interest outside of work to look into emerging technologies or new languages, frameworks, patterns, tools, etc. This is why there were still some at the company who did put in the effort on their own time to keep up in their industry.

But the company is also at fault in this scenario. I've been saying for years that the half-life of any technology in our space is only about 4 years. That means in 4 years half of the frameworks, languages, tools, etc. are obsolete! It's a lot to keep up with! Considering the alternative, it's much cheaper to train than to not.

Does your company have both caves (quiet private space) and commons (open collaborative space) for their skilled workers?

This is one of those things that I feel quite strongly about. As companies have been moving more and more to cube farms or worse (no walls -- just tables, no personal space), the average output for knowledge workers has plummeted.

It takes less than a 10 second distraction to lose 20-40 minutes of productivity. Headphones are not the answer to keeping the cube farm as they can also be distracting. The reality is that people who think and solve problems for a living need quiet! Other times they need co-workers close to collaborate quickly and efficiently. Caves and Commons are both important!

The reality is that the cube farm is never really going to go away and there are restrictions far beyond some departments ability to change, but if given the chance -- real walls are better than short walls, and short walls are better than fabric walls, and fabric walls are better than no walls at all.

Do you have an effective interview process?

Evaluating one candidate over another is tricky! We usually start by looking at a resume, which are either far too short (thanks to misguided career counselors) or too long (I don't care what job you had more than 10 years ago)! Then we pick a few that we're going to invest some time into. Perhaps it's a phone screening or a code test. Then we pick whomever did the best from those exercises.

That's all well and good, but have you stopped to consider that this process mostly helps you hire the wrong person for the job? Some companies processes are just so efficient at hiring the wrong person over the right person, that it almost seems like that's what they were trying to do.

Now I may not be the oracle when it comes to interview processes, but I do have one philosophy that I believe is helpful. Rather than evaluating the candidate on resume writing or test taking. Perhaps you should see what kind of employee you would be getting if he worked for you. On day 1, what skills would he/she have, in terms of code skills, communication skills, and soft skills. Do they show up on time? Do they stick around when it's time to get this stuff out the door and help is badly needed? Do they follow directions? Do they have the aptitude to learn new skills? Are they worth investing in? Do they stick with a company for a good amount of time? These are some of the answers I want!


I think that while many of the original points are still somewhat valid. I'd take my list now days. Having an up-to-date schedule just isn't that important when you release software often (with every new feature). Using source control is simply not enough. The modern source control tools with a modern workflow make a big difference.

If you care more about the success of projects and the quality of the software you'll be writing than if some things will be slightly painful, then you'll probably want to modernize the list.

Have other points that you think should be included? Comment below! If you're interested in working at a company who values these things, check out our Careers page.


comments powered by Disqus

Follow Us

Latest Posts

subscribe to our newsletter