Hi all. Googling this question brings up valid suggestions regarding the use of automated testing for repetitive functions, using manual testing for UI tests, etc. However, I was interested in perhaps some best practices that take into account more tangible factors.
For example, is there a sweet spot for data entry values that make the decision more straight-forward? Let's say I have a process flow that requires data entries in multiple forms. For example, I need to test the 5th form in a flow, each form has 25 data entry points. Is it worth going with automated testing? What if it's the 3rd form with 10 data entry points? Looking for some best practices here or maybe your experiences with this kind of decision.
Thank you.
Discussion posts and replies are publicly visible
My personal opinion is that you should mainly focus manual testing on changes and direct regression testing (i.e. things that you immediately suspect could be impacted by a change). You should also consider manual smoke testing and probably some manual security testing.
You can't automate all the testing. By the time you think of every possible combination of inputs and script it, your test script will take more man-hours than the software did. You can only put so many tests into your test scripts, so they'd probably be limited to happy paths already designed and a few basic exception flows and invalid inputs (also nulls).You should run those primarily to load-test and do full regression tests. Even a few basic tests on everything can make it vanishingly unlikely that regression bugs go uncaught.
Performance testing I think could easily be a combination of manual and scripted. Load tests to find bottlenecks followed by repeated manual benchmarks to eke every ms out of improvements.
Generally, I like to ask myself a couple of questions:
1. How often will I need to test this feature?
2. Is the form likely to change in the future?
3. Does the cost savings of manual testing outweigh the automation efforts?
4. Have developers used good 508 practices so that the form is easy to automate?
5. Does this fit well into a bigger regression suite?