Testing is one of the most important, and the often overlooked activities on an outsourced IT project. How the test process is handled can make or break a outsource IT project.
Here are a few "golden rules" of outsource IT testing.
- Programmers can't test their own work. It is the same as proof reading a letter you write, you just don't see the mistakes that others find.
- All outsource IT companies has bugs, some are just not found yet. Some bugs have been found, but not worth fixing because they are minor annoyances that are hard to fix, or will very rarely be seen (we just end up calling those "features”).
- In the process of fixing bugs, even the best programmers for hire can introduce new bugs.
This is why a feature that worked in the last version may not work in a new version. Your test plan must account for this, it's a fact of life with the outsourced IT.
It does not matter how simple the change was or what the outsource IT company tells you, fixes introduce new bugs more often than not.
- The earlier a bug is found in the outsource IT project cycle, the cheaper it is to fix. Here is an example:
- John finds a missing feature while writing up his outsource IT project description (one could argue this is a bug). To fix this, he simply adds another sentence or two to his description.
That was easy!
- Two months later, John finds that one feature has a bug that makes it cumbersome to use with another feature. Since both features are already coded, changes must be made to two places in the code.
The changes will introduce new bugs and the code that was "done" will have to be changed again.
That was somewhat painful.
- Two months later, one of John's users finds another bug. Now the program is in production. There is "live data" managed by the program and the shipping department is totally dependent on the program for their operation.
Now it is risky to make the fix, down time has to be scheduled, new freelance programmers may need to be hired, and they will need time to learn the original freelancer's unique programming quirks.
Now it's really painful!
- You need to track what you test. Keep notes as you test so you will be able to refer to them when a problem occurs.
The most important thing to tell the outsourced IT programmer is what you did right before you saw a bug. This is very important in helping the outsource IT programmer locate and correct issues.
Bottom line: Test early, test often, test everything you can on every preliminary version, and keep a log of your testing.