As TOPP matures, we’ve been doing more deliberate QA work, spearheaded largely by Tim Coulter (yay).
But as Rob Miller noted in a recent comment on Ethan’s blog, we’ve been getting behind in QA.
As I see it, this backlog is the product of two factors: the increasingly vague schedule, and the lack of a dedicated QA staff.
To the first point: Nobody in software really likes schedules (”it’s just not ready yet!”), but I do think they help to maintain focus. As long as you don’t end up putting the schedule ahead of the product.
So, what about testing? Spolsky suggests 1 dedicated tester for every 2 or 3 developers. Does that sound high? I don’t know, but can we afford to keep growing our development staff into the dozens with zero dedicated testing staff? No. I’m convinced that’s a false economy and it’s holding us back.
Let’s look at the alternatives. I can only think of these:
- Spend less time on QA.
It would be nice to make the process faster, and it would be nice to magically turn our manual test runs into regression tests, but as Tim will surely point out, you can’t skip the manual tests. Regression tests serve an entirely different purpose. Skimping on the time-consuming manual testing is a false economy. Less time on QA means lower quality.
- Or, Spend less time on design and development.
In the short term, I think we’ll be forced to keep doing QA with the staff we have - which means time away from our “primary” responsibilities.
- Or, Hire testers.
Doesn’t sound so bad now, does it?
There’s a fourth alternative - have one person or a small group recruit, train, and encourage a team of volunteer QA folks. After all, this is the open-source world; get more eyeballs and you’ll get more leverage out of the people that you have. That’s what we’re going to do at OLPC, so if you’re thinking of the same someday, I’d love to swap notes.
Comment by Mel on September 8, 2008 at 7:06 pm