Back to Homepage

Homework 6 - Verification Plans

The first software defect was reportedly discovered by Grace Hopper, a U.S. Navy Rural Admiral, while she was writing programs for the Harvard Mark II. She literally found a moth stuck between a relay in the machine, then taped it into the Mark II's log book with the explanation: "First actual case of bug being found." Now, long before that, there were defects in programs; defects are a part of developing software. We make mistakes, then we fix them.

What we don't do is ship them. No one wants to use software that doesn't do what it is intended to do. That defeats the core purpose of software: it does things reliably and fast.

What we do to prevent shipping defects is verify. But we have to do this in a way that actually decreases the likelihood that our users find defects. In every case, this means making a plan.

Because there are many ways to verify software, plans can vary. Some plans might be heavily focused on testing, whereas others might use formal verification. Others might just focus on shipping software and having users report failures. All of this depend what your team's tolerance for failure is.

Your job in this homework is to devise such a plan. Your plan should include:

• A list of all the requirements that you will verify. You don't have to verify them all (nor likely can you).
• For each requirement that you will verify, a verification plan:
• If you propose to write a tests, what exact tests will you conduct and what will count as each test passing? Will you write an automated test or run the test manually?
• If you propose to conduct reviews or inspections, how will you analyze the code? If you write a proof, what property will you prove? If you conduct a review or inspection, what aspects of the code will you inspect to verify the requirement is met?
• For all of the requirements, how will your verifications be integrated into your process? Will you run automated tests after every build? Before every commit? When will you conduct inspections and who will be involved?

The structure of your written plan is up to you. Organize it in a way that's helpful for you executing your plan.

Grading

Submit your plan to Canvas as a URL to a GitHub page.

You'll get full credit, minus any clear violations of the criteria below:

• -0.3 points. A requirement critical to the value of your software is not part of your verification plan.
• -0.2 points. A plan won't verify the stated requirement.
• -0.1 points. A plan doesn't say which requirement it's verifying.
• -0.1 points. A plan is inherently unable to be consistently replicated.
• -0.1 points. A plan is unnecessary; there is no requirement to test this.
• -0.1 points. A non-critical requirement is missing from your verification plan.
• -0.1 points. A plan is too ambiguous or underspecified. It is unclear what the tester should do.

Notice that the grading scale is pretty picky: it requires a lot of very careful, precise thought about exactly what you should verify. This is the level of precision that good verification requires.

In preparing your plan, verify its comprehensiveness by referencing the criteria above before submitting this assignment.