Many people ask me what I look for in a software development firm and/or how do I gauge a software development firm. For a while I had a standard list of preliminary questions that I would ask and based on their answers I would decide if I should go with them or not.

• Can I interview the developers?
• Can I look at code you have written or past projects that you have undertaken?
• Can you please provide a list of your prominent clients? Can I call them and ask about their experience?
• Which language/platform to you use to develop and why?
• How many developers work on site? Are there off site workers?
• Which software methodologies do you practice and why?
• What warranties/assurances do you provide with the end product?
• How long are the development cycles?

These questions are important and should be asked every time you meet with a new software development firm. However having done a few projects with various software development firms I realized that I was lacking one very important question.

• How do you test the software?

I didn’t look at testing because I figured they would give it to us and we would test it. Or I thought that they would obviously preform some basic testing. In most cases I was very wrong.

If the development firm has no software QA testing procedure RUN AWAY. By testing procedure I mean a product development phase where they have dedicated testers that are testing the program. This shows on pure negligence of the firms part and you should not get near them.

Any form of testing will tell you how they planned on you using the end product. If they do basic testing expect basic usage and functionality. If they write unit tests expect the program to function and work when the data is exactly perfect (as they expect the user to simulate the unit test).

What I like to hear is that they have dedicated QA people that:

1. Have access to the all of the products specs and code.
2. Know how to write in the language that the code is written in.
3. Know the architecture of the application and any frameworks it may be using. 
4. Can build scripts/programs to fuzz the functionality of the program.
5. Try their best to BREAK the program.

Additionally testing MUST have set time allocated to it. This time should be calculated just like development time ( it should be estimated as you have no idea if how many bugs there will be or when the first round of bugs are fixed a bunch new are introduced). Above all make sure you are billed for the testing time because you get what you pay for….