Rather than pay both for hosting and the ability to modify templates etc on LiveJournal, I've installed WordPress on my new domain. So if you are visiting this site after Monday, March 20, 2006, click

here.

For those of you who have it bookmarked/linked, the new address is http://adam.goucher.ca. All the content has been moved over, though some of the posts are pretty ugly as the LJ->WP conversion stripped some stuff.

And for those of you who are subscribed (all 10 of you as of this moment), I'll redirect the FeedBurner feed Wednesday, March 22, 2006 at some point in the evening to point to the new site.

• Most security people come from networking background, not software ones. This gives the bad guys who are attacking your software an unfair advantage as they are software people.


• Software people, through a lack of education, often think that security is a feature that can be added in, but security does not occur from 'magic crypto fairy dust'


• Automated code analysis tools find bugs not flaws. People are the only things that can find those


• Seven ways to make software more secure (this list I believe makes up the book he is flogging)
  
  
  
  1. Good code reviews; both automated and manual
  
  
  2. Perform architectural risk analysis
  
  
  3. Do software penetration testing
  
  
  4. White-box risk based security testing
  
  
  5. Abuse cases. If a developer says "A user won't ever do it", do it. Then giggle evilly
  
  
  6. Have explicit security requirements for your application
  
  
  7. Operational security. This is where the network security people and the software security people put everything together.
  
  

• Measure in hours - Stop thinking in terms of "man days". Think instead in terms of hours and then roll things up later into days. Management is not going to care about task x taking 80 hours. What they want to know is how many days it will take (10; or 2 single person work weeks). Testers will then report their progress against tasks in hours. This is a more defined value of time and allows people to work on different tasks during the day but still provide a very clear measurement. Then for inputting into the project plan just multiply the reported value against the "standard" work day length. Or if you are reporting via a spreadsheet template, do the calculation right in there. Note: Once you start this, do not start to watch the total number of hours a person is logging in a week and force them to meet it. Only use that number if things are ridiculously out of whack one way or another. This is similar to not measuring your testers against the number of bugs they find in a set period.


• Task independence - While not always possible, the schedule should be designed in such a way at each testing activity affects exactly one line item in the schedule. If a testing task can be applied to more than one item, how are you supposed to accurately measure the time taken? Apply the same time to each? Well, you just injected a fictitious time period into the mix. Allocate the amount? How do you decide what percentage accurately reflects the time taken. It could be easy to figure out in some cases, but might not be in others.


• Task granularity - Most testing tasks can be broken into smaller components. These smaller components should be recorded as sub-tasks which can be recorded against. This allows for clear trending and task separation. Example: It might take me 5 hours to run the automated framework against server platform x. But what if 3.5 hours of that was environmental trying to get an instance of the LDAP server I am interested in running correctly. Without a high level of granularity we cannot see that at the end of the project I spent 6 days (once we aggregated the hours reported up) fighting unrelated software. And if we had a better process in place to handle it, then either we could have pulled the schedule in a bit, or I could have done more exploratory testing.


• Sufficient detail - Yes, the project schedule is a "living document" and does not live in isolation. It should however be able to stand on its own tough. Each item should have a descriptive enough title allow the testing activity to commence without having to consult 2 or 3 documents to figure out what the heck the activity is supposed to be. Example:
  
  Bad - Upgraded Oracle LDAP Support
  
  Good - Upgraded Oracle LDAP (9.0.4) Support
  
  By just adding the version number to the task item, the value is increased significantly as you instantly can tell what the upgraded version is. Without it you need to find the version number by combing through the requirements and supported platform matrix. And both documents may be inaccurate as things change. As people are ultimately measured about how they do in the context of commitments in the schedule, it is the schedule that should be considered law.

This week's feature article on StickyMinds is a about completion requirements which is another thing I feel persons in QA should be involved in defining. Here are some check-lists to get you started on thinking along what done really means.

Requirements

• Are requirements of sufficient detail and clarity to be developed upon?


• Are requirements of sufficient detail and clarity to be tested upon?


• Is the origin of the requirement (potential customer, existing customer, market movement, etc) recorded?


• Are all outstanding questions from within the organization regarding the requirement set been addressed?

Feature Development

• Are new feature design documents accurate and check into source control?


• Are existing feature design docs that were impacted by this feature updated and in source control?


• Are the unit tests for this feature in source control?


• Has the new feature been integrated into the automatic build process?


• Have the unit tests been run by Test to verify their validity and worth? Unit tests that only check "perfect" data do not really do much to increase the quality of the code base. Likely the "perfect" condition will be one of the few the developer checks during integration testing


• Is there a bug associated with all FIXME or related comments in the code? By definition, if a developer puts something like FIXME in their new feature code, it means that there is something to fix in there. Something to fix equals bug.


• Is there no more TODO or related comments in the code? Again, by definition, if there is something left "TODO" in a feature, then the feature is not 100% complete.

Feature Testing

• Is there a new Feature Test Plan checked into source control?


• Are test plans of other affected features updated and checked into source control?


• Were new features test cases developed and checked into source control?


• Were existing test cases for other features updated as a result of this new feature?


• Is there a clear mapping between the feature test cases and the feature requirements?


• Have final test results been archived in source control?


• Has the feature been added to the automated testing solution? or Has a plan been recorded in source control regarding how the new feature could/should be integrated into the automated testing solution


• Do all modifications to the automated testing solution framework have associated unit test? Practice what you preach


• Have you tested with non-ascii data? The software market is global these days and not everyone limits their character usage to the standard ascii set

Current Product Engineering

• Have unit tests that verify the bug no long exists checked into source control?


• Have both the bug fix and unit tests been brought into the other relevant trees?


• see Feature Development items

• Money (or lack thereof) - One thing most, if not all, household recognized names have in common is the size of their bank account. In the technology realm, most of the money is made when a company goes public or is acquired. By definition, the people at the ground floor get rewarded disproportionately than the rest. But when was the last time a company went public or was acquired which was started by a QA person? Again, almost by definition they are founded by a (group of) developer(s). So you are never going to look at the annual Forbes list of wealth and see someone with 'QA/Tester' beside their name. When picking a career, what are most people going to choose? The one that tops you out at 200k/year or one that could net you millions at IPO time? I remember wanting to learn how to write C in grade 3. If I had, who knows how different my finances would look now?</i>
  
• Career Path - This is kinda Money part 2. What is the career path that someone who chooses QA/Test choosing for themselves. There are really two choices. Consult/Contract and ladder climbing. Consult/Contract is a separate bullet, so we'll concentrate on the ladder. How many CEOs of companies of recognizable size are there in the world that started in QA? Not one I would bet. C* positions tend to be filled from Finance or Operations. So realistically, the ladder tops out for someone climbing the ladder around Quality Czar. Which isn't necessarily a bad thing. But the pay cheque is missing a couple zeros in comparison.


• Domain micro-brand - A lot of QA/Test people go on their own and hop from company to company once they reach a certain level on testing philosophy development. We'll call this the domain micro-brand. These people have visibility only in the testing domain which does not help attract new people in. If you look at successful people in business, they were not there for 3 months whipping a division into shape, but often years stamping it with their identity. I would guess that there are a number of reasons for reinventing yourself in the domain micro-brand model. Money and Career Path likely factor high in the list.


• Domain immaturity - Lets face it, QA/Test (heck, programming) is in their infancy as far as a domain goes. Architects, Plumbers, Cobblers and Coopers etc all have hundreds of years to draw upon. We have, what, 50? Tops? Until the the field itself matures, the visibility of it will lag behind. So how do we do that? I'm not sure.
  
  
  
  • Some argue that we should all agree on a certain way of doing things and get a single body to certify people in it. Of course, there are those who argue that such an effort would not really do anything. In networking, the Gold Standard for awhile was the CCIE certification because to get it was damn hard. I'm not sure if it is still the same thing, but if you wanted someone who know knew networking you weeded all the resumes that did not have that. This not going to be resolved any time soon.
    
  
  
  • Part of the reason for not having certifications is that there is a lot of disagreement over what knowledge should be certified. I'm guessing though that there is a common subset that could be disseminated around. What we as QA/Testers should do is work on getting that subset developed to the point where it can be taught to students at the post-secondary level. Very few colleges or universities have courses dedicated to software engineering (version control, unit tests, design patterns etc) let alone on pure Testing Fundamentals. If we could get that taught to all 2nd or 3rd year computer science students we might get more to choose this route. More bodies == more visibility and the chance of finding a Testing Hero.
    
  
  


• Test as Stepping Stone - All too often there is a high churn rate of people in QA/Test as it is often seen/used as a stepping stone into development proper. I can't tell you how many times people have expressed shock when I say that I want to be in QA/Test for the length of my career and not move into development at some point. This is what I enjoy, am good at, and passionate (hate that term) about. Type. Compile. Swear. Type. Compile. Swear. Thanks, but thats not a work cycle I want to live in.


• Domain Fragmentation - Another working against us is how fragmented / specialized the Test world can be. Automation vs Manual. Exploratory vs Following a script. Tool A vs Tool b. Functional vs Load. The list goes on and on. Specialization creates more domain micro-brands, who start to consult/contract to get more money. See how all these build on each other?


• Personality - Lets face it, most people who enter into technology are introverts. To develop a large enough profile to be a role model/draw for new people to the field will likely take some pretty meticulous and deliberate planning (and execution of course). That is likely to result from an extrovert who just happens to be in technology. Something that statistics appear to be stacked against

• Start them Young - We need to seriously push our institutions of higher learning to include at least one course on testing to find those people who have the knack for it.


• Certification - Even though there are very valid reasons not to push certification yet, I think we need to pursue it if for no other reason that it would give some level of credibility to the profession in the eyes of organizations.


• Educate employers - If you work with someone with hiring responsibility, make sure that when they want a tester, that they do not structure their posting as "a developer who cares about quality". They won't get what they really want, a tester, and will then likely look at people who call themselves testers in a negative light.


• Educate the greater technology community - It would nice to see columns on the advances in testing methods and methodologies in the major development/technology magazines. With added exposure, will come new thoughts and blood to the profession.

• James Bach


• Cem Kaner


• Esther Derby

• From 90 Days of BzzAgent: "A small company can operate with organic and loosely-defined processes, a larger company has to bring more form and consistency to them". The context of this was in regards to HR processes, but this is also very (if not too much so) true for Quality initiatives. Often these are started too late or not given enough resources to achieve their potential. Especially if a company is the first one started by a group of developers (vs pure business people).


• SourceForge now offers Subversion access on all project (well, if the project admin has enabled it that is). More on this Version Control Systems in a later post, but for now, here is the instructions for using it.


• Let me say it again. The most effective way to shoot yourself in the foot when doing a process assessment is to not communicating the results within a short time span to all those who were involved. Not communicating results will be interpreted by people as that either management (who have the results) either does not care about the issues raised or is incompetent. Which of course may or may not be true. As soon as you have the results, publish them. Waiting 4 months after the assessment to let people know what the findings were means that momentum has been lost and next time an assessment takes place people are going to be less enthusiastic with their participation.


• Often you see in the various QA/Test forums requests for questions to use in an interview. And usually someone posts a couple. Were I devious and looking to hit the QA interviewing circuit, that would be a great way to get "inside" information on how people interview for QA positions. That aside, I want to start a list of bad questions, and perhaps a better way to get similar, but better, insight on how the candidate thinks. Remember, you are hiring them for how they think; not how they look in a suit. I don't even wear one to interviews for this reason. If they won't hire me because I didn't wear a suit, I don't want to work for them.
  
  
  
  1. Bad: Why did you choose QA/Test
     Better: What about QA/Test makes you want to come to work everyday
     Why: Why did you choose X, be it QA/Test or Programming or Marketing etc are all canned answers and will get canned responses. The better varient will hopefully allow you detect whether someone is passionate about their chosen field or just doing it for the mortgage payment.
  
  
  2. Bad: Our code is written in X, can you you program X?
     Better: Do you have any programming experience? And if so, is it X? As I mentioned when I gave you the company overview, our product is in X.
     Why: Generally, the first programming language is the hardest to learn and a lot of the skills are transferable. Having exposure to programming concepts is far more important than knowing the language the product is written for. It is not as if the test group is going to be writing unit tests -- that is of course the domain of the programmers.
  
  
  3. Bad: What are 3 strengths and weaknesses? (you get double the Badness points if you then make a little T-chart labelled "Good" and "Bad".
     Better: What would you say are your strong and weak points?
     Why: First off, this is another canned question, so candidates can be tricky and have a canned response to it. If they are really smart they will know how to spin all the negatives into a positive. But if you must ask it, not forcing them to come up with a specific number of responses for each should (hopefully) elicit a better response. You should of course get at least one weak point.
  
  

• Principle of Objective
  War: Direct every military operation toward a clearly defined, decisive and attainable objective.
  Testing: Direct every testing process at a clearly defined, decisive and attainable objective.


• Principle of Offensive
  War: Seize, retain, and exploit the initiative.
  Testing: Seize, retain and exploit the initiative in the analysis and test execution phases by pro actively taking advantage of advanced testing technologies.


• Principle of Mass
  War: Mass the effects of overwhelming combat power at a decisive place and time.
  Testing: Mass the effects of all available testing capabilities to deliver results at the decisive place and time.


• Principle of Economy of Force
  War: Employ all combat power available in the most effective way possible; allocate minimum essential combat power for secondary action.
  Testing: Employ all testing resources available in the most effective way possible; allocate minimum essential testing resources for secondary action.


• Principle of Maneuver
  War: Place the enemy in a position of disadvantage through the flexible application of combat power.
  Testing: Place the product in a position of disadvantage through the flexible application of testing resources.


• Principle of Unity of Command
  War: For every objective, seek unity of command and unity of effort
  Testing: For every analysis or execution objective, seek unity of command and unity of effort. If at all possible, centralize testing processes under the leadership of one entity and have clear lines of delegation to subordinates.


• Principle of Security
  War: Never permit the enemy to acquire unexpected advantage
  Testing: Never permit the product to acquire unexpected advantage through feature creep and unmonitored late additions to the codebase


• Principle of Surprise
  War: Strike the enemy at a time or place or in a manner for which he is unprepared.
  Testing: Strike the product in a manner for which it is unprepared.
  
• Principle of Simplicity
  War: Prepare clear, uncomplicated plans and concise orders to ensure thorough understanding.
  Testing: Prepare clear, uncomplicated plans and processes for all testing activities and provide concise guidance to all those involved in test execution for to ensure a thorough understanding.

• A bug with a long list of things to change
  
  
  
  • Pro: provides an easy checklist to change and verify against
  
  
  • Pro: don't have to read a 200 page document entirely on a computer screen
  
  
  • Con: these bugs can be large and ping-pong between Test and Doc as they change bits of the bug but not the whole thing
  
  


• A bug with an electronically marked up version of the doc
  
  
  
  • Pro: provides quick context for all changes
  
  
  • Con: every tester must have the correct software to edit the file
  
  
  • Con: have to read a 200 page document entirely on a computer screen
  
  


• A bug with a scanned copy of a a marked up version of the doc
  
  
  
  • Pro: provides quick context for all changes
  
  
  • Pro: don't have to read a 200 page document entirely on a computer screen
  
  
  • Con: need a Cool Scanner*
  
  

• Jim Thatcher's site


• The US Government's official Section 508 website


• A FAQ from the US Access Board

• Legibility - Avoid small fonts, fonts that cannot be resized and low contrast between the text and background colours. Resizing of fonts is going to be a big issue/challenge for web design in the coming years. As boomers age, there is going to be a massive number of people whose eye sight is starting to worsen. If they cannot read your content clearly, they are going to get it from another outlet. Unfortunately, a lot of web design is based around 'text -- of size x -- goes here, image goes here'. If size is really x * 5, your time making a your page pretty has been wasted.



• Odd Links - Make links obvious, using coloured, underlined text. Differentiate between visited and unvisited links. This method is the de facto standard of displaying links. Yes, you can tweak your page to display without underlines, and with the same colour as the text, but why? If presentation is really an issue, change the text colour to something that complements your existing colour palette, but leave the underline.



• Flash - "Most people equate animated content with useless content." Blame it on the bubble, but this is so true. Not only is there this perception, but then there is the issue of having the right version of the right plug-in which can be a nightmare unto itself. Also, how many people see a flash splash page or something load and immediately look for the 'skip intro' link. I do every time -- unless of course I am going to the site for the flash splash; some web development companies have very pretty splashes



• Content not web friendly - copy should be short, scannable and to the point, not just marketing fluff. But if you insist on having the marketing fluff at least include a boilerplate or such at the beginning that allows you to skip fluff if you want but still derive value from it.



• Bad site search - Invest in better software if your searches do not meet your user's needs. I am pretty sure it costs very little, if anything to integrate Google or Yahoo into your website, so the investment is likely just going to be time to make sure that all pages are spiderable and correctly added to the index. Now, this strategy doesn't really apply to intranet type scenarios where haveing all your corporate secrets in the main index would not be a good thing, but thats where products like the Google Search Appliance come in. Oh, and they look really nice too.



• Browser incompatibility - Firefox, Opera, Safari, blah, blah, blah. From a test perspective this can be a big issue as it is yet another permutation to the massive matrix of combinations you should test. Here is my formula. Take the current browser market shares for the top couple and use that as a base weighting for testing. From there, skew the percentages based upon your target market. For example, if you are a site that caters to open source or linux, you will want to up your Firefox percentages. Similarly Mac sites will want more Safari coverage. Oh for the simple days between Browser Wars I and Browser Wars II.



• Cumbersome forms - Try not to use forms, but if you must, make them as minimal as possible. Personally I think the end of forms as a means of collecting data is towards the end of it's useful life. One of the first things we taught our daughter about the internet is to always lie when they as for information. Not only is this safer, it also insulates from spam/data harvesting. If you cannot trust the information that you get from a form, the form is useless and should be removed. If you must have a form though, use the tabindex attribute for the input tag to bounce you through only the mandatory tags. I'm lazy. I don't want to have to navigate through 100 demographics fields that I'm not going to fill in. Nor do I want to move my hand all the way over to my mouse to skip by them all.



• No contact information - Giving a physical address for your organization gives it an instant boost in credibility. If you have a domain, whois will give you the business contact information which has an address. If you have a phone number, Google or any number of other reverse phone lookups will give you the address. If my mom knows how to do this, I'm guessing a large majority of the internet population does, or will soon. Even if you don't put up an address because you are working out of your livingroom or parent's basement, call it Suite 200 or something. The added benefit of this is that when you do move into a new proper office you can announce it with great fanfare and turn it into a mini marketing event.



• Frozen layouts - Pages should resize according to how they are being displayed be it bigger or smaller screen real estate or printing on A4 vs. 8.5x11 paper. Here is the perfect example. HP's main web site. Regardless of how you adjust your browser size, the content will always be the same size and tucked up in the top left corner. As the average monitor size increases, so too will the average screen resolution. Make your page dynamic, or at least dynamic enough to not fill the screen which emptiness.



• Inadequate photo enlargements - Let users see close-ups of your product. This is of course starting to be thwarted by browsers resizing images to fit their window size automagically for you. So even if you provide a monstrously sized close-up of your product, the user might still have to hover the mouse over the bottom right of the image, weight for the 'I really did want a large picture, thank-you' button and click it.

One of the traits of applications that call themselves Web 2.0 is the availability of an API which allows you to integrate it into your own service (generally called a mashup). Given the public facing nature of the API it needs to be rock-solid from a testing perspective and the easiest way to get that comfort is to do what is known as eating your own dog food. In this case it means "using the documentation that is available with the API, make your own mashup for testing purposes".

Most Web 2.0 APIs work by sending something specific to a specific URL then dealing with the returned XML appropriately. This means that the API is magically language independennt. For this reason, I would recommend testing it ina language other than the one the application was originally written for and in a language that is fast to write in (python, ruby, perl).

I had hoped to illustrate how easy it is to test an API from a local company (such as Nuvvo or BubbleShare) but neither have released APIs to their services. Instead I used FeedBurner.

The script I whipped up in an hour is below the cut or here connects to the FeedBurner server and collects the stats for a feed and displays them. It has had very little architectural thought, and limited testing, but from an example perspective I think it is sufficient to get the point across. Clearly in a full-bore test scenario you would want more features, like the rest of the API implemented for instance and checking whether the back-end servers the API is manipulating have been manipulated correctly.

One other side effect of implementing the API testing in Python (or again, Perl or Ruby) is that you can instantly release it as a module to the associated community and have native support for your product in that language, which increases your total market, which increases your revenue, which increases...

And of course, if you want to contract me to exercise your API in Python (with the aforementioned 'more thought'), email me.

/* set our database -- change adam to whatever your db is called*/
use adam
go

/* delete our procedure if it exists already */
if exists(select name from dbo.sysobjects where name = 'drop_adam_tables' and type = 'P')
	begin
		drop procedure dbo.drop_adam_tables
	end
go

/* recreate it */
create procedure dbo.drop_adam_tables
	@prefix varchar(5)
as
	declare @tbl varchar(40)
	declare @combined varchar(40)

	/* suppress the output of the inserts and drops */
	set nocount on

	/* build a temp table which has our table names */
	create table #adam_tables (
		t_name varchar(40)
	)
	insert #adam_tables (t_name) values('changed_table_name_1')
	insert #adam_tables (t_name) values('changed_table_name_n')

	/* systematically remove all our tables */
	declare tbl_cursor cursor for
		select * from #adam_tables
	open tbl_cursor

	/* fetch first row */
	fetch next from tbl_cursor into @tbl

	/* loop and remove */
	while @@fetch_status = 0
		begin
			set @combined = @prefix + '_' + @tbl
			if exists(select name from dbo.sysobjects where name = @combined and type = 'U')
				begin
					execute('drop table ' + @combined)
				end
			fetch next from tbl_cursor into @tbl
		end

	/* cleanup */
	close tbl_cursor
	deallocate tbl_cursor
go

set @combined = @prefix + '_' + @tbl

execute drop_adam_tables your_prefix

import unittest
import dummy
import xml.dom.minidom

# this is mandatory for all unit test files; every test suite (for which there is one per class) is appended here
suites = []

class custom_props(unittest.TestCase):
    def setUp(self):
        self.defaults = {"foo": "bar=true"}

    def test_missing_custom_props(self):
        """custom_props is not a mandatory item, test_dict should have default values"""
        test_xml = "foo"
        test_dict = {}
        dummy.handle_custom_properties(test_xml, test_dict, self.defaults)
        for default in self.defaults.keys():
            assert_(default in test_dict.keys(), "Missing default tag %s" % default)
            assertEqual(defaults[default], test_dict["url_tag"], "default tag %s has wrong default value" % default)

    def test_has_custom_props_with(self):
        pass

    def test_has_custom_props_without(self):
        pass

# create a test suite based upon all methods of custom_props that start with "test"
suite_custom_props = unittest.makeSuite(custom_props, "test")
# add it to the master list of suites
suites.append(suite_custom_props)

# here we create a module wide suite and call it 'all_suites'; the name is important and must be the same for each module
all_suites = unittest.TestSuite()
for suite in suites:
    all_suites.addTest(suite)

import os, os.path, sys
import unittest

# master suite
msuite = unittest.TestSuite()

# dirs that contain (potentially) interesting stuff
interestings = ["lib", "testscripts"]

for interesting in interestings:
    for root, dirs, files in os.walk(os.path.join(os.path.split(os.getcwd())[0], interesting)):
        for f in files:
            if f.startswith("test_") and f.endswith(".py"):
                test_module = os.path.split(os.path.join(root, f))
                if test_module[0] not in sys.path:
                    sys.path.append(test_module[0])
                buncha = __import__(test_module[1][:-3])

                msuite.addTest(buncha.all_suites)
                del(sys.modules[test_module[1][:-3]])

runner = unittest.TextTestRunner()
runner.run(msuite)

• you can turn any machine into a properly deployed testing machine in a matter of minutes


• you have absolute control over the environment


• you are (less likely) to run into version specific bugs on different platforms as you have the same version across all platforms</a>
  
• if you rev a version of a module you are using or patch something then you do not have to rebuild and redeploy; just sync with VCS


• anyone can deploy the environment


• does not require root/sudo permissions

• completeness - everything that should be flagged as localizable is


• branding - typically, stuff that are trademarked or are related to a specific brand are not marked as localizable


• sentence fragments - this is a big one from a contextual standpoint. Every sentence, or better still, should be one "resource". Building a sentence in fragments in the code then doing something with it almost guarantees that your translators are going to get the context wrong for some part of the sentence

• Project Test Plan - This document outlines the high-level strategy from a testing perspective. Each feature is discussed briefly as are the resources required by the project (staffing, hardware, software). The primary audience of this document is not test, but other people/groups in the organization who do not need to know the nitty-gritty details, but have input on the overall direction the testing will take. If there are any large test-support tasks happening during the duration of the project, they should also go here. A couple test-support tasks might me "use a new hardware management strategy" or "automate all new features".


• Feature Test Plan - This document is specific to a single feature and outlines the strategy for testing a specific feature and is the most important document for the individual tester as it puts them in the right frame of mind to approach the problem. Included is a detailed description of the feature and description of the approach(es) to testing it. Example: The display of CGI generated information appears in all supported browsers. Ultimately, any tester should be able to be given this document and achieve good test coverage. Nowadays, an automation section should also be included which outlines how a feature could be automated; even if it is not going to be at this time. Or even more importantly, what obstacles exist to prevent automaton which would be the start of a conversation with development.


• Feature Test Cases - Now these are where the nitty-gritty details are. I'm of the school of thought that they should be detailed enough to not allow for an confusion regarding the goal of the test, but not so detailed that a 5 year old could do them. If a feature has all it's test cases completed, then it has 100% test completion -- as we know it now. The last part is important. It could be that a new attack vector was discovered or new feature x has a cascading feature effect on only feature m which will cause you to create new test cases. This is a big topic, so I'll (try) do a separate post on test cases in the near future.

• it allows you to ramp up new members to the test group quicker than without


• it allows for business to continue as usual if someone leaves (or is hit by a bus)


• it allows you to outsource some of your testing should you chose that model at some point in the future


• it allows you to use non-testers in your testing which can be a useful method of checking usability