A bit later I became the one in charge of doing the hiring. I followed the same basic model for awhile, then I was bitten. I had a few bad apples. I then read Smart and Gets Things Done: Joel Spolsky's Concise Guide to Finding the Best Technical Talent
, which talks about the Joel Test so I took the test from my point of view of my team. - Do you use Source Control? (YES)
- Can you make a Build in one Step? (NO)
- Do you make Daily Builds? (NO)
- Do you have a Bug Database? (YES)
- Do you fix bugs before writing new Code? (NO)
- Do you have an up-to-date schedule? (NO)
- Do you have a spec? (NO)
- Do Programmers have quiet working conditions? (NO)
- Do you have the best tools money can Buy? (YES)
- Do you have testers? (YES)
- Do new candidates write code during their Interview? (NO)
- Do you do hallway usability testings? (NO)
So our test was a 4 out of 12. I wish I remember how long ago I did that, but things have changed since then and even our YES answers have became better.
- Do you use Source Control? (YES)
- Can you make a Build in one Step? (YES)
- Do you make Daily Builds? (YES) -
- Do you have a Bug Database? (YES)
- Do you fix bugs before writing new Code? (YES)
- Do you have an up-to-date schedule? (YES)
- Do you have a spec? (YES)
- Do Programmers have quiet working conditions? (NO)
- Do you have the best tools money can Buy? (YES)
- Do you have testers? (YES)
- Do new candidates write code during their Interview? (YES)
- Do you do hallway usability testings? (YES)
New Score 11 of 12, not perfect but much better.
The quality and the output of our team has increased dramatically in the past few years. Some of that is attributable to doing the things on that list.
Do you use Source Control?
When I started we were using Visual Source Safe, we have improved in this area by switching to Subversion.
We never used branching before, now it's common to branch. We have built a one button deploy of our software using FinalBuilder. When we select the option to deploy to production which automatically tags the revision number that was used to build the software. We are currently investigating Mercurial or GIT but we have not decided on which and need to find a logical break to switch.
Can you make a Build in one Step? Do you make Daily Builds?
We purchased FinalBuilder several years ago which helped made this a reality. Our first build script was triggered as needed. It was run on an old desktop machine. We then step it up on a timer and it ran every hour, then our speed of our build decreased and we had to run it every two hours.
Our initial design was that we built and then deployed in the same script. That was a mistake, as we had to rebuild the application every time we change where we wanted to deploy For example Development, Test, QA, Production. So the scripts were changed to build and the deploy to a staging directory. Then we could deploy to any of the 4 environments. This insured that the exact same binary that was tested was moved into Production.
We then invested in FinalBuilder Server which allowed us to start the build every time someone committed.
This improved our turn around time for our testing team, but the builds where still quite long. So we took some time to analyze what was slow, fixed some problem areas of our scripts and invested in better hardware
for the build. Our builds now take 10 minutes to complete, instead of the 2 hours.
Do you have a Bug Database?
Our first bug database when I arrived was a single table Delphi application that stored a limited single description of a problem, who it was assigned to and if it had been resolved. Since then we adopted
an Bug Tracking database that another team had built. It's web based and has nearly everything we
need. We have used the TortoiseSVN bugtraq properties to Integrate our Bug database with SVN.
We also track new development in our Bug Tracking Database. This allows to to tie development
requests regardless of type to the actual code that implemented them.
Do you fix bugs before writing new Code?
We don't fix all of the bugs in the application before we write new code. However, if we want to write new code in a specific area, we find the existing bugs in that area and fix them before heading off.
Do you have an up-to-date schedule?
We try to be agile in our process. We hold daily stand-ups, and do iterations. Our planning of what occurs in each iteration is working as expected. As we have refined our process our ability to know how long
something is going to take has improved dramatically. We still are not accurate, 100% of the time but things
balance out more often that not.
Do you have a spec?
We did not have specifications for the longest time. But last year we completed a several month project to develop business rules for all areas or software covered. This is useful as the business desires to change an area they update the existing documentation. It really helps to find the dependencies and test all downstream code. It also gives direction on what is needed to the software developer. We don't spend time with UI specification, unless it's critical. Business rules is usually more than enough here.
Do Programmers have quiet working conditions?
This actually has become worse since I started, our cubical sizes have shrunk. I have no control on cubical sizes, so I have focused on what I can control. Many of the developers now where headphones to attempt to get quiet working conditions. I personally own two pairs of Audio-Technica ATHM40FS Precision Studio Headphones, one for work and the other for home. They use a 1/4" Jack so if your trying to use your 1/8" Jack you will need an adapter. It's my only way to find a quiet working environment, but they do work, I can't hear the phone ring when they are on :-)
Do you have the best tools money can Buy?
When I arrived we had Delphi 3 and Delphi 7 was was the latest version. They quickly upgraded, so I have always felt like I had the best tools. But since that time we have turned to Maintenance for all of favorite tools, so we have access to the latest tools. It also make budgeting easier.
Do you have testers?
When I first arrived we had a loose group of super users who tested as part of there responsibility. We still have these testers, but we also now have a formal team of people dedicated to the QA of the product. This really took some time to change, but the benefits are huge.
Do new candidates write code during their Interview?
This is what really was the genesis of this blog post. Although team fit/personality is part of the equation it's
not nearly as important as being able to code. If your hiring a developer on just an Interview your bound to find "Bad Apples"
Recently we hired two Contractors, we had over 20 applications and they all looked like qualified Delphi developers. We sent out a written interview. With some SQL questions, we also had two simple coding questions. They both could have been answered with little work in less than 8 lines of code. I was amazed by the number or 30+ lines of code answers.
This quickly reduced the pool of candidates below 10. We then did some actual Interviews reducing the pool of candidate even more. Then we sent them off on a 3 hour coding assignment, with full access to the Internet.
Some people familiar with the process call it brutal, but I have yet to select someone who has poor coding skills but is great in an interview again.
Do you do hallway usability testings?
We could do more to promote this, but it's common for a developer to pull someone at random to come look at something to see how they react.
In summary
This took quite a bit of time to implement, it was not an over night process. But each individual step along the path has improved our team. It's something I would encourage each development team to consider.
Do you use Source Control?
When I started we were using Visual Source Safe, we have improved in this area by switching to Subversion.
We never used branching before, now it's common to branch. We have built a one button deploy of our software using FinalBuilder. When we select the option to deploy to production which automatically tags the revision number that was used to build the software. We are currently investigating Mercurial or GIT but we have not decided on which and need to find a logical break to switch.
Can you make a Build in one Step? Do you make Daily Builds?
We purchased FinalBuilder several years ago which helped made this a reality. Our first build script was triggered as needed. It was run on an old desktop machine. We then step it up on a timer and it ran every hour, then our speed of our build decreased and we had to run it every two hours.
Our initial design was that we built and then deployed in the same script. That was a mistake, as we had to rebuild the application every time we change where we wanted to deploy For example Development, Test, QA, Production. So the scripts were changed to build and the deploy to a staging directory. Then we could deploy to any of the 4 environments. This insured that the exact same binary that was tested was moved into Production.
We then invested in FinalBuilder Server which allowed us to start the build every time someone committed.
This improved our turn around time for our testing team, but the builds where still quite long. So we took some time to analyze what was slow, fixed some problem areas of our scripts and invested in better hardware
for the build. Our builds now take 10 minutes to complete, instead of the 2 hours.
Do you have a Bug Database?
Our first bug database when I arrived was a single table Delphi application that stored a limited single description of a problem, who it was assigned to and if it had been resolved. Since then we adopted
an Bug Tracking database that another team had built. It's web based and has nearly everything we
need. We have used the TortoiseSVN bugtraq properties to Integrate our Bug database with SVN.
We also track new development in our Bug Tracking Database. This allows to to tie development
requests regardless of type to the actual code that implemented them.
Do you fix bugs before writing new Code?
We don't fix all of the bugs in the application before we write new code. However, if we want to write new code in a specific area, we find the existing bugs in that area and fix them before heading off.
Do you have an up-to-date schedule?
We try to be agile in our process. We hold daily stand-ups, and do iterations. Our planning of what occurs in each iteration is working as expected. As we have refined our process our ability to know how long
something is going to take has improved dramatically. We still are not accurate, 100% of the time but things
balance out more often that not.
Do you have a spec?
We did not have specifications for the longest time. But last year we completed a several month project to develop business rules for all areas or software covered. This is useful as the business desires to change an area they update the existing documentation. It really helps to find the dependencies and test all downstream code. It also gives direction on what is needed to the software developer. We don't spend time with UI specification, unless it's critical. Business rules is usually more than enough here.
Do Programmers have quiet working conditions?
This actually has become worse since I started, our cubical sizes have shrunk. I have no control on cubical sizes, so I have focused on what I can control. Many of the developers now where headphones to attempt to get quiet working conditions. I personally own two pairs of Audio-Technica ATHM40FS Precision Studio Headphones, one for work and the other for home. They use a 1/4" Jack so if your trying to use your 1/8" Jack you will need an adapter. It's my only way to find a quiet working environment, but they do work, I can't hear the phone ring when they are on :-)
Do you have the best tools money can Buy?
When I arrived we had Delphi 3 and Delphi 7 was was the latest version. They quickly upgraded, so I have always felt like I had the best tools. But since that time we have turned to Maintenance for all of favorite tools, so we have access to the latest tools. It also make budgeting easier.
Do you have testers?
When I first arrived we had a loose group of super users who tested as part of there responsibility. We still have these testers, but we also now have a formal team of people dedicated to the QA of the product. This really took some time to change, but the benefits are huge.
Do new candidates write code during their Interview?
This is what really was the genesis of this blog post. Although team fit/personality is part of the equation it's
not nearly as important as being able to code. If your hiring a developer on just an Interview your bound to find "Bad Apples"
Recently we hired two Contractors, we had over 20 applications and they all looked like qualified Delphi developers. We sent out a written interview. With some SQL questions, we also had two simple coding questions. They both could have been answered with little work in less than 8 lines of code. I was amazed by the number or 30+ lines of code answers.
This quickly reduced the pool of candidates below 10. We then did some actual Interviews reducing the pool of candidate even more. Then we sent them off on a 3 hour coding assignment, with full access to the Internet.
Some people familiar with the process call it brutal, but I have yet to select someone who has poor coding skills but is great in an interview again.
Do you do hallway usability testings?
We could do more to promote this, but it's common for a developer to pull someone at random to come look at something to see how they react.
In summary
This took quite a bit of time to implement, it was not an over night process. But each individual step along the path has improved our team. It's something I would encourage each development team to consider.
Hi Robert,
ReplyDeleteThanks for sharing your journey in building your team. Sadly for my team we'd score 4 out of 12 too - although I really don't know if using Visual Source Safe counts as using Source Control! Being the new guy, I don't have much say in our processes but I may try and steer my team leader towards Joel's article as a starting point. I always look forward to your blogposts. Thanks again.
Remember that this original Joel Test is from 2000. Updated versions have been discussed:
ReplyDelete- http://geekswithblogs.net/btudor/archive/2009/06/16/132842.aspx
- http://vaig.be/2010/12/joel-test-for-software-companies.html
I also thought that Joel Spolsky had created an updated version himself, but I couldn't find it...
I was not aware of other versions, thanks for the links.
ReplyDeleteI know Joel has not updated it,
In fact the New Stack Overflow Careers (Joel is the CEO), links to the original test.