Checklist for a new software development vendor

Aug 06, 2019

Introduction

Sometimes it is hard to choose among software development vendors, especially if you do this for the first time in your experience. All companies promise “high quality, on time, on a budget”, and they all look the same.

We propose you a checklist to score your software development vendor. Give a point each time the software development vendor responds “yes” or provides complete information on the questions below.

Before you select a vendor

The vendor shows a team of qualified engineers who will deliver the project

Usually, the software development vendor has two major departments: sales and production. It is not a rare case when the sales department closes a deal without sufficient human resources available (“sale first, find engineers later”).

The team of engineers is complete

The team must include:

• project managers;
• software engineers;
• quality assurance engineers;
• business analysts;
• chief technical officers.

Here are the possible negative outcomes if one of the roles is missing:

• a project manager is missing — missed deadlines, unmanaged risks, budget overruns;
• a software engineer is missing — no outcome at all;
• a quality assurance engineer is missing — no control over the quality of the software solution, low-quality software product;
• a business analyst is missing — the team may incorrectly understand your requirements, deliver not useful software product as a result, or a product which doesn’t fulfill your business goals;
• a chief technical officer is missing — the internal quality may be poor, it will lead to the high cost of maintenance of the software product;

The vendor shows a defined software development life-cycle.

The life cycle must include business analysis phase, architecture definition phase, software implementation phase, software test phase.

The vendor shows examples of the project documentation

The vendor must show examples of software requirements, software architecture, test plans, test reports.

The vendor shows the defined process for risk management

Among common risks:

• failure to integrate components;
• the third-party component is not reliable;
• the developer has no previous experience with the selected technology;

The project manager must manage the most probable risks accordingly. Each risk must have:

• plan A — a clear plan to prevent the risk;
• plan B — a clear plan to fix negative outcomes, caused by the risk;

If unmanaged, risks may fail the whole project.

The vendor shows the defined process for quality management of the source code

This includes defined coding standards, regular code reviews.

Why is it important? The internal quality of the software product directly affects the cost of maintenance. Maintenance includes:

• fixes of software errors;
• small updates;
• introduction of new features;

The cost of maintenance is higher for projects with poor code and architecture quality.

The vendor shows the capability to perform a different kind of tests

Ask your vendor: “Do you cover applications with automated tests”?

Unit tests, integration tests, UI tests, reliability tests, performance tests, load tests.

The vendor doesn’t mind if a third-party will have access to the project to control the quality

This gives you a way to have the second opinion about what’s going on with the project.

During the project

The vendor must submit interim software builds regularly

Usually, it is one week, two weeks, or one-month iterations.

The vendor must submit test reports along with software builds

The test report must include detailed test cases with pass/fail notes.

The vendor must report on internal code quality

Contemporary software development methods are built to show an increments in functionality, even an increment is not significant Failure to show at least some progress indicates that the software development vendor struggles with some problems. Reasons:

• lack of allocated qualified software engineers;
• software engineers are not qualified

Each new build must not break the existing functionality

If the new build of the software product breaks some existing functionality (e.g. adding to cart worked a week ago, but in the new version it is broken), then the typical reasons are:

• the software development vendor doesn’t test the application enough;
• software engineers are not qualified;

The vendor delivers the source codes of the project

This prevents vendor lock-in, in that case, if you want to change your vendor.

After the project is completed

The vendor delivers build scripts and documentation on how to build the project

The most important project acceptance rule — is that you can build and test the software product on your own. Sometimes, compiling and building the software product is a big deal.