Many small to midsize businesses do little testing or sometimes (gasp) none at all. This is understandable. A lot of companies do not start out as software companies or software isn’t their primary focus. The tools they use have been lopped together by a scrappy team or individual and their greatest strength is that their programs work well enough to serve the business needs.
But what happens when growth begins to strain the resources of your once great product? Complete rewrites are expensive and time-consuming, not something current clients will be happy about. Patching together quick fixes only exacerbates the problem. Outsourcing seems risky, especially if you have PCI DSS concerns. Have no fear! Roll up those sleeves and get ready to learn how to create a test plan to turn your spaghetti monster of code into a well-oiled machine.
Fun Note: There is actually an International Software Testing Standard - ISO/IEC/IEEE 29119 - and, despite its usefulness, it will make you sleepy.
Make a Plan
There is nothing more dangerous than diving into a project without specific ideas of what you want to accomplish. We have all been there - no well-defined goals, lots of ambition and great ideas, and six months later we have a very advanced idea of what we should have been doing instead.
The first question you should be asking is, “What problem am I trying to solve?”
Start with your immediate business needs. Assuming you are not using a freeware model of market expansion, your major expenses should be driven directly by the tech that generates the majority of your revenue. The only way to know this is through comprehensive bench-marking to understand where your resources are most readily consumed. Before you start assembling a test plan, it is important to do exploratory testing. Product knowledge is key. Documentation, contract specifications and usage analysis are invaluable in this process so if you do not have them, they should become part of your plan.
In web apps, there are three main areas that you should be focused on initially:
- System/Environment - Server, OS, Services e.g. email
- Core Logic - Back-end, DB
- Front-end - UI, UX, Data Representation
These areas likely make up the majority of your product and are responsible for the majority of resource consumption, whether that is labor, bandwidth or processing. Initial bench marking and exploratory testing in these areas is a good place to start.
Once you have identified where your issues are, you can use cost/benefit analysis to prioritize your test plan.
Priorities: Risk, Cost/Revenue, Immediacy, Stability, Flexibility (future growth), Liability (security).
Considerations: Time to Test, Human Resources, Test Environment (Test Server, Testing Software)
Analyzing risk is also an important part of prioritization. In software development the most prevalent risk is insufficient time to completion with the allotted resources and relevant deadlines. Scope creep is another major concern. These concerns can mostly be subverted through careful planning and management of the development process.
Form a Strategy
Okay, so you’ve already done a ton of work. Now what?
Based on knowledge gained in product discovery, risk assessment and established priorities, you should be ready to decide on a strategy. This is when you decide what types of testing are required and what areas are going to be tested. The types of tests that you use generally fall into one of two categories; functional testing and non-functional testing. In web development, functional testing is almost always concerned with the product itself. Examples include:
- Unit tests
- Regression tests
- Integration tests
- System tests
- Acceptance tests
- API tests
Non-functional testing is concerned with the environment it is deployed in. Examples include:
- Load tests
- Performance tests
- Stress tests
- Security tests
- Reliability tests
This non-exhaustive list is a good starting point for forming a test strategy. As you do, try to break down your strategy into granular tasks and determine who is going to complete the tasks and when they are going to do it - you will thank yourself later. This is also the time to consider any documentation and specifications you are going to integrate into your team’s workflow as it will have an impact on time-to-completion/task.
Clear expectations lead to clear results. Test objectives and criteria for completion should be well-defined for the areas you are targeting. This is where the test plan really starts to come together. Through the designated test strategies you can start breaking down the testable elements of your application according to your objectives. It helps to map out the testable areas of your application. There are many mind map programs you can use to do this. My favorite quick-use tool is sketchboard.io.
Objectives start out very general. An example may be, “Verify that users can sign in.” This can be further broken down into its logical components - Front-end form and submit, back-end authentication and authorization, call-back and redirect with session cookies, etc. The test itself may still be simply, “verify that users can sign in”, but the criteria for the test has subsidiary test cases that provide the framework for the workload in the overall plan. Attention to detail here also minimizes the amount of setup and virtualization you have to include in individual tests.
Prepare Your Environment
Testing requires people and technological resources. Allocating these elements appropriately is the key to avoiding major risks to a test plan. Identifying team members available and their strengths and weaknesses allows you to optimize the time spent on the project and build in flexibility in the inevitable case of the unexpected. Define your environment and assign roles. Some important considerations:
- Has a test server been appropriately set up for the project?
- What tools are going to be used? (Selenium, framework dependent test suite, manual verification, etc.)
- Are workstations outfitted for immediate productivity? (Environmental variables set, paths added, network accessibility, IDE’s)
- Who is going to assume responsibility for management, implementation, spec validation?
- Are appropriate communication channels set up between QA and product managers, account managers and other relevant parties?
Finalize the Plan
I highly recommend using some sort of project management software, even if it is for a personal project. Just knowing how long any given task is going to take puts a lot of power in your hands.
There are plenty of options available but the simple, generically named Planner (Linux, Windows) provides for resource management and Gantt charts with the ability to export to HTML. This allows you to break all of the tasks you defined earlier into resource groups, essentially answering who is going to do what and when they are going to do it. This may be overkill if you are an independent developer. However, I still recommend you track your time-to-completion/task so you can better budget your time in the future. More elaborate projects may benefit from paid services like Jira.
All of your work is for nothing if you cannot prove you did it. The last thing you should have in your Test Plan is a summary of the project deliverables. This includes the plan itself, any specifications or test cases you are working off of, any documentation pertinent to the plan (contracts, product specs), any documentation that you produce while testing, data/logs, and a final report on the work completed.
If you are an independent developer, you may not need all of these things. However, it is a good idea to define what you are going to give yourself that will help your productivity in the future. At a minimum, documentation that you produced, a task/time report and a brief summary highlighting anything that was unexpected or changed should be included. This provides a good foundation for future planning.
You did it!
I know what you are thinking. You are thinking, “Okay man, nice words and all but where is the bacon?” Here is an excellent example of a test plan. This, of course, is only the beginning. Ideally, you have several layers of test preparation, starting with a organizational level test plan that evolves as needed with the execution of product level and project level plans. What I have described is just a good start to integrating tests into operations with a little technical debt.
Good luck and happy testing!