Top Resources

Stickyminds.com - Comprehensive software testing resource site associated with 'Better Software' Magazine, with articles, news, information on software testing and quality engineering, books, tools, conferences, message boards.

Cem Kaner's software testing site - Cem Kaner's site contains a large selection of his articles about software testing, legal issues, test management, and more (see the 'Publications' section of the site). Also see his 'badsoftware.com' website , a consumer and legal-issues orientation to software quality issues.

Software QA and Testing-related Organizations/Certifications/Conferences

SEI - Software Engineering Institute web site; info about SEI technical programs, publications, bibliographies, some online documents, SEI courses and training, links to related sites.

EOQ-SG - European Organization for Quality - Software Group, an independent not-for-profit organization founded in 1983. It is comprised of more than 30 national quality organizations and other institutions, enterprises and specialists.

American Society for Quality - American Society for Quality (formerly the American Society for Quality Control) web site; geared to quality issues in general, not just Software QA. ASQ is the largest quality organization in the world, with more than 100,000 members. Provides a wide variety of general quality-related certifications, as well as the CSQE (Certified Software Quality Engineer).

Association for Software Testing - A nonprofit professional organization dedicated to advancing the understanding and practice of software testing. For scholars, students, and software development practitioners. Sponsors conferences, publications, web sites, newsletter.

QAI Global Institute - QAI is a global workforce development and consulting organization addressing the education and 'Operational Excellence' needs of information technology, information technology enabled organizations and knowledge intensive organizations. 
Information on associated QAI local chapters around the world can be found via their QAI Chapters site.

Certification Information for Software QA and Test Engineers

ISTQB Certified Tester - The International Software Testing Qualifications Board, based in Belgium, was initially a part of the European Organization for Quality - Software Group. Certifications and testing are administered by ISTQB organizations in each of a number of countries around the world. Multiple certification levels are available, including: Foundation, Advanced, and Expert.

ISEB Software Testing Certifications - The British Computer Society Information Systems Examinations Board (ISEB) maintains a program of several levels of testing certifications.

CSQA/CSTE - QAI Global Institute's program for CSQA (Certified Software Quality Analyst), CSTE (Certified Software Tester), and Certified Software Project Manager (CSPM), Certified Associate in Software Quality (CASQ), and many other certifications.

 

QA and Testing-related Magazines/Publications

Tea Time With Testers Magazine - Free online testing magazine available as PDF file; web site includes back issues.

Professional Tester Magazine - Bi-monthly testing publication from Professional Tester Inc. Free access to PDF version, archive, individual articles and news and events agenda.

STQA Magazine - Software Test and QA Magazine from BZ Media. No access to articles without registering on site.

Software Quality Professional Magazine - Published by the American Society for Quality; web site includes table of contents and abstracts of all articles, and full text of selected articles.

Better Software Magazine - Web site has full text of each print issue's featured article. Published by Software Quality Engineering, Inc.

Methods and Tools - Software Methods and Tools e-newsletter web site by Martinig and Associates; regular articles are included on process improvement, testing, modeling, management, etc. Site includes current issues and past issues with full text of all articles; as well as extensive additional information and resources.

General Software QA and Testing Resources

Thoughtworks Testing Portal - Testing articles and blogs from Thoughtworks.

Google testers' blog - Public blog site for Google's testers.

The Value of Checklists and the Danger of Scripts - Presentation at CAST 2008 by Cem Kaner. Summary of the presentation available from his blog, and a link to the pdf file of the presentation.

Software Engineering: An Idea Whose Time Has Come and Gone? - An interesting 2009 article from 'IEEE Software' by Tom DeMarco in which he indicates, among other things, that his early ideas and advice regarding quantified work, project planning, and metrics for software projects - such as those in his 1982 book 'Controlling Software Projects: Management, Measurement, and Estimation', may have been wrong.

Exploring Exploratory Testing - Article by Cem Kaner and Andy Tinkham from 2003 about the exploratory testing approach to software testing; includes discussions of questioning strategies and heuristics.

Exploratory Testing Explained - Article by James Bach on exploratory testing; includes attributes of a software project and tester that impact decisions on testing approaches, exploratory testing examples, etc.

Pass vs. Fail vs. Is There a Problem Here? - Interesting blog article on testing vs checking and the issues in 'pass/fail' type testing and reporting, from 2009 in Michael Bolton's DevelopSense blog.

The Seven Basic Context-Driven Principles - Article about the 'context-driven' approach to testing on the Association for Software Testing web site, by Cem Kaner and James Bach. Lists the main principles and includes description, examples.

How to write near-perfect software - An old but still highly relevant long article in Fast Company about how software was developed for the U.S. Space Shuttle. "The group's most important creation is not the perfect software they write -- it's the process they invented that writes the perfect software."

G. Weinberg's Collection of QA Articles - Jerry Weinberg's site has a section with a collection of some of his QA-related articles; site also includes sections with articles on thinking, leadership, and software projects, and more.

Tom Gilb Web Site - Site includes a large collection of articles, papers, course slides, etc concerning risk, metrics, and other QA-related software engineering topics.

Software Negligence and Testing Coverage - Article by Cem Kaner contains an old but still very informative list of 101 types of testing coverage measures; shows the complexities in any discussion of 'testing coverage'. Selected quotes of interest from the article: "Even if you achieve complete coverage for a given population of tests (such as, all lines of code tested), you have not done complete, or even adequate, testing." and "The decision as to whether to try for 1%, 10%, 50% or 100% coverage against any given population is non-obvious. It involves tradeoffs based on thoughtful judgment."

What is a Test Architect? - Discussion re test architects at the blog site of Microsoft's Alan Page.

RBCS Testing articles - RBCS Consulting Services web site's collection of software testing articles on a wide variety of testing-related subjects.

Errors in Scientific Software - Article titled 'The T experiments: errors in scientific software' by Les Hatton; old but still alarming article from 1997.

Software Certifications and Standards impacting Embedded Software - Article from Dr Dobb's Portal from September 2006, discussing standards such as DO-178B "Software Considerations in Airborne Systems and Equipment Certification", IEC 61508 for safety-related systems, FIPS 140-2 re "Security Requirements for Cryptographic Modules", HIPPAA, etc.

'Good programmer' definitions/discussions - Since testers and developers often need to work closely together, and since many testers also do some programming, it is helpful to get some perspective on 'what is a good programmer'. Also see the discussion in the old Joel on Software forum..

WSR Consulting Group publications - Good collection of QA and Testing related articles from WSR consulting, a computer crisis/litigation consulting company. The articles have an emphasis on proper management of problem projects and engineering-customer relationships for software projects.

Practical Software and Systems Measurement - Web site with extensive information on software development metrics, sponsored by U.S. government. Site contains articles, reports, examples, and a free PC-based software tool to assiste in project-specific metrics development.

Where is the Science in Computer Science? - October 2012 article in Communications of the ACM by Vinton Cerf (VP at Google, past winner of the Turing Award, one of the acknowledged 'Fathers of the Internet', and president of ACM). In the article he states, among other things, '....Even though we design software systems and ought to have some clues about how these systems behave and perform, we generally do not have a reliable ability to anticipate the states these systems can get into, their vulnerabilities, their performance, and ability to adapt to changing conditions.' He also goes on to note our generally poor ability to predict how long it will take to find and fix bugs, or to have an idea how many new bugs will be created by fixes.

Software Estimation - December 2005 interview about software metrics and estimation from the IT Metrics and Productivity Journal .

Software Project Estimation - Good introductory article covering the basic issues of software project estimation, from the Software Productivity Centre in Vancouver.

Testing Education Articles - Collection of articles on software testing and the teaching of software testing from the Florida Institute of Technology funded by the U.S. National Science Foundation.

'Software Experts' site - Software engineering site oriented to microcontroller/embedded system environments, by Eberhard De Wille and Dana Vede. Site has sections on design, coding, refactoring, process, and a large section on testing.

'IT Infrastructure Library' - a set of best-practices guides on the management and provision of operational IT Services. From the British Office of Gevernment Commerce and the itSMF, the 'IT Service Management Forum' a UK-based organization comprised of 1000 companies and government organizations worldwide. There is a related ISO/IEC 20000 Standard against which organizations can be assesses and certified. An online organizational ITIL Service Management Self Assessment is available. There is a good summary of the ITIL approach in Wikipedia.

Big Ball of Mud - Outstanding essay on the 'de-facto standard software architecture', by Briane Foote and Joseph Yoder of the U. of Illinois at Urbana-Champaign. The 'Big Ball of Mud' architecture is defined as 'a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design....The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition.' They discuss why this architecture is so popular, advantages and disadvantages, and what can be done to improve such systems.

The Frameworks Quagmire - Old but still relevant article by Sarah A. Sheard summarizing and exploring the conflicts and complexities among various software process standards existing in the late 1990's - CMMi, SW-CMM, SDCE, Trillium, IEEE, FAA-iCMM, EIA-632, NATO-AQAP, MIL-STD-498, ISO/IEC-12207, ISO-9000, etc.

Satisfice.com Web Site - James Bach's Satisfice.com Web Site with a great collection of his articles on various aspects of software testing.

DevelopSense Web Site - Web site of Michael Bolton, who collaborates with James Bach, has a large collection of articles and blog postings with interesting perspectives on software testing.

Bret Pettichord's Web Site - Web site of Bret Pettichord with articles and links to various test and QA-related info including his Watir open source web testing framework.

BetaSoft Web Site - Wide variety of QA, testing, and automated testing discussion forums, sections for jobs and resumes, other resources.

SQATester.com - QA and Testing information, discussion forums, other resources.

Search Software Quality - TechTarget's software QA and testing site with a collection of articles, tutorials, blogs, and news.

Illustrative Risks to the Public in the Use of Computer Systems - Enormous list of software, system, and related problems compiled by Peter Neumann/SRI International. Organized by categories such as space, defense, medical, stock market, elections, insurance, cryptography, etc. Includes related book list, other information. (Also see 'Risks Digest' listed below.)

Process Improvement Case Study Featuring Reviews and Inspections - Article titled 'Process Improvement: Case Study of an Improvement Program Featuring Reviews and Inspections' in Software Quality Professional magazine.

Will Bugs Eat Up the U.S. Lead in Software? - Old but still interesting Bloomberg/Business Week (International Edition) article comparing present state of U.S. software industry to U.S. automobile industry in the 1970's, when Japan took away huge market share with better and cheaper products by adopting Deming's and Juran's quality approaches. Discusses a possible repeat with the current US software industry losing software development market share to countries such as India. Entire issue and cover story reports on software problems - discussion of past problems, problems with software engineering, and possible fixes. Also see similarInfoweek article - 'The Big Picture: Killer Apps And Dead Bodies'.

ARIANE 5 Flight 501 Failure Report by the Inquiry Board - A rare and instructive detailed public analysis of a major software failure - the 1996 launch failure of the new Ariane 5 rocket. This is the official report of the inquiry board appointed by the French National Center for Space Studies and the European Space Agency. Also see the article 'Design by Contract: The Lessons of Ariane' which includes a discussion of the code reuse issues brought to light by the Ariane 5 failure.

Eiffel FAQ - FAQ site for a programming approach, based on the ideas of Bertrand Meyer, with the goal of improving software component reusability, extendibility and reliability using assertions, preconditions, and postconditions.

Internationalization Testing - Article at Oracle/Sun's web site; includes guidelines and good check list of questions for testing an internationalized software product.

Risks Digest - Digest of the 'Forum on Risks to the Public in Computers and Related Systems'. Includes latest issue and archives covering software and system problems, vulnerabilities, disasters; based on the comp.risks newsgroup.

SEI Capability Maturity Models - SEI's CMMI web site, with info and documentation downloads on the CMMI for Development and the CMMI for Acquisition models.

Construx Software Resources - Site with many useful resources such as CxOne, a lightweight, tailorable, modular, and scalable software engineering framework, estimation info and resources, various checklists, and Steve McConnell's 'Software Survival Guide' website.

CM FAQ - Configuration Management FAQ edited by David Eaton; includes 'What is CM?', 'How should a CM system relate to process enforcement?', CM books and other resources, etc. Not updated in recent years but still a useful FAQ.

SR/Institute's Software Quality Hot List - Extensive collection of links to many QA and testing-related articles, resources, etc.

Uniform Computer Information Transactions Act (UCITA) - Text of controversial proposed U.S. laws (formerly Uniform Commercial Code Article 2B) concerning software quality. This would essentially implement new laws in all 50 states in the U.S. Additional info on UCITA controversies at Ralph Nader's CPT (Consumer Project on Technology) web site and Cem Kaner's BadSoftware.com web site. . As of 2010, only the states of Maryland and Virginia had enacted UCITA since it was first proposed in 1999. In 2009 the American Law Institute proposed an alternative version for software contracts but it resulted in similar controversy to UCITA.

FDA Medical Device Software Validation Guidelines - U.S. Food and Drug Administration's 2002 guidelines for medical device software validation.

Negotiating Testing Resources - Excellent article by Cem Kaner about testing project planning and budgeting; from a 1996 software quality conference - old but still relevant.

Software Engineering Resources - Large collection of useful information and links to many other sites and resources, all related to the SW engineering process including project planning and management, metrics, risk analysis, programming methods, OO SW engineering, testing, QA, CM. From R.S. Pressman, author of the book 'Software Engineering, A Practitioner's Approach'.

Software Test Coverage Analysis article - Article containing a good discussion of test coverage analysis from Bullseye Testing Technology, maker of "C-Cover Test Coverage Analyzer" tool.

10x Software Development - Technical Debt - Steve McConnell blog post on 'technical debt'; one of many excellent articles/posts on his Construx web site.

Managing Technical Debt in Software-Reliant Systems - Good overview article about technical debt from Software Engineering Institiute

Object-Oriented Concepts - Basics of object-oriented programming concepts, from Oracle's (formerly Sun's) Java site. Good quick intro.

CMMI (Capability Maturity Model Integration) - A suite of process improvement models for product and service development and maintenance. The suite includes the CMMI-SW model, and there is a 'staged' and 'continuous' version. Each of the CMMI models can be coordinated with other CMMI models to enable enterprise-wide process improvement. CMMI-SW builds on the previous SW-CMM model which was 'sunsetted'.

Comp.software.testing Usenet News Group - Via Google Groups web site (formerly the Deja News site), can be used to search through past postings; postings go back to 1995.

The comp.software.testing FAQ - The comp.software.testing FAQ; maintained by Raymond Rivest; resource for testing-related conferences, mailing lists, books, periodicals, organizations, and links to other sites. Not maintained in recent years but still useful.

 

Agile and XP Testing Resources

Manifesto for Agile Software Development - The origin of the 'Agile' approach and the twelve guiding principles of agile software development.

Agile Testing Articles - Large collection of articles related to Agile testing at Agile Alliance web site.

Agile Methodologies - Martin Fowler's online discussion of 'agile' methodologies (XP, Scrum, Crystal, FDD, DSDM, etc.) includes summaries of various approaches as well as reference information, and factors to consider in choosing these approaches.

Perils and Pitfalls of Agile Adoption - Article by Matt Heuser at InformIT site, includes discussion of risks such as that agile methods are easy to misunderstand, that it's easy to think you're doing Agile right, and be wrong, and that agile methods make value (or lack of value) visible.

Agile Testing - What is it? Can it work? - PDF version of an article by Bret Pettichord that summarizes considerations and issues in testing in agile environments.

An Uncomfortable Truth about Agile Testing - Article by Jeff Patton on the StickyMinds site about some of the potential difficulties of testing on an Agile project.

XP Resources - Large collection of resources from Ron Jeffries about 'Extreme Programming' including a discussion of how QA fits into the XP approach, XP Magazine archives with articles such as 'Test-First Design', 'Incremental Requirements', 'Extreme Programming and the CMM', and more. Also see 'The Rules and Practices of Extreme Programming ' at the www.extremeprogramming.org web site.

XP in a Safety-Critical Environment - Interesting article by Mary and Tom Poppendieck concerning the applicability of XP practices in safety-critical software development.

Test Automation Resources

Seven Steps to Test Automation Success - Good introductory article on how to approach automated testing; by Bret Pettichord.

NoVaTAIG Test Automation Resources - Test automation presentations, articles and resources from the Northern Virginia Test Automation Interest Group monthly meeting summaries ('Recent past meetings' link on site main page).

Architectures of Test Automation - Long article on test automation by Cem Kaner, includes discussions on GUI regression testing, maintainability, a classification scheme for test automation, an automation evaluation scheme, 'test automation' vs 'computer assisted testing' and more.

Test Automation Snake Oil - Old but still relevant article by James Bach about how to approach test automation.

Test Tools for Free - Short article by Danny Faught discussing some free test tools, along with the basics of 'freeware', and information on a testing freeware newsletter. 

Effective Performance Testing articles - Extensive collection of how-to and other information on performance testing at Scott Barber's web site.

Evaluating and Choosing the Right Tool - Elisabeth Hendrickson describes a five-step process for comparing, evaluating, and choosing the right test tool; from the Stickyminds.com web site.

Java GUI Testing - Short discussion of automated Java GUI testing issues, includes interesting discussion of methods of identifying a component in a GUI hierarchy for use in developing automated test scripts.

 

Mobile Testing Resources

Test Your Mobile Web Apps with WebDriver - From the Open Source at Google blog - discussion of how to write automated tests to test a site when viewed from an Android or iOS browser. The WebDriver web testing framework includes a touch API that allows a test to interact with a web page through finger taps, flicks, finger scrolls, and long presses. It can rotate the display and provides an API to interact with HTML5 features such as local storage, session storage and application cache.

Mobile Wireless Test Automation - Site with a collection of information by Julian Harty on practical experiences in automating aspects of software testing for mobile wireless applications; also has information on effective mobile manual testing. Includes testing techniques for iPhone, Android, SMS; and information on common tools, app testing over WiFi, more. Also has links to some presentations and tutorials and lists of other resources.

Mobile Application Security Testing - Whitepaper from Foundstone/McAfee about mobile app security testing.

Mobile App security Articles - Articles and whitepapers on mobile app security from Denim Group's web site.

MobiForge - Testing - MobiForge mobile development community web site section on mobile software testing.

Mobile Application Testing blog - Mobile app testing blog by Anurag Khode

Mobile Web Development Resources - Mobile web development resources including some testing resources and standards resources, etc.

 

Web QA and Testing Resources

Why Load Testing Ajax is Hard - Article by Patrick Lightbody on the Ajaxian blog site from December 2008 on the challenges of load testing sites incorporating Ajax.

Web Site Testing Checklist - More of a web site development checklist, but it is extensive and still useful as a way to generate ideas for testing a web site.

Performance Testing Guidance for Web Applications - Online 18-chapter guide for an end-to-end approach for implementing web performance testing. Part of the Performance Testing Guidance Project web site. Covers: managing and conducting performance testing in both Agile and structured environments; load testing, stress testing, and other types of performance related testing; identifying objectives, designing tests, executing tests, analyzing results, and reporting. Published 2007 but still useful.

Web Site Performance Testing - A collection of useful information on various aspects of performance testing, from Scott Barber's web site. Topics include: "Pinpointing and Exploiting Specific Performance Bottlenecks", "Common Performance Testing Challenges", "How Fast is Fast Enough", and "Introduction to Performance Testing". Although some of the information is not specifically oriented to web performance testing, it is still highly applicable.

Load Testing Of Web Sites - Article from IEEE Internet Computing about web load testing; useful overview from 2002.

Keynote Systems Resources Page - Useful collection of articles and information on web site performance testing. Also see the related site performance indiceswhich lists a variety of business, consumer, government, and other web sites along with their 'performance index'.

Evaluating Web Sites for Accessibility - Article on the World Wide Web Consortium web site's 'Web Accessibility Initiative' section on how to assess and test web sites for accessibility issues.

Handling and Avoiding Web Page Errors - Three part series from Microsoft site; covers sources of common Web page errors, how to handle run-time script errors, and techniques for avoiding preventable errors. Old but still useful.

 

Web Security Testing Resources

StopBadware.org - Web site security guidelines and information from the StopBadware site, based at Harvard University?s Berkman Center for Internet & Society.

Computer Audit FAQ - Good introductory information from IsecT Ltd. on 'Computer Audit', which refers to the analysis of computer systems and networks by examining the effectiveness of their technical and procedural controls (information security control systems) to minimise risks. Also has links to other resources, and some articles such as 'Strategic Approach to Information Security Management'

SANS Security Resources - Web site of SANS (SysAdmin, Audit, Network, and Security Institute), a cooperative research and education organization for sysadmins, security professionals, and network administrators for sharing lessons learned and solutions. Includes and Instrusion Detection FAQ; more than 1500 white papers on security; webcasts; security trends, top security risks, and much more are freely available.

Common Attack Pattern and Enumeration - CAPEC is a publicly available, community-developed list of common attack patterns (descriptions of common methods for exploiting software systems), with a comprehensive schema and classification taxonomy. By MITRE Corporation.

W3 Security Resources - Large collection of information and resources on web security, including an FAQ, hosted by the W3C Consortium (the folks who set web standards/protocols, etc.)

Microsoft Security Site - Microsoft's web site for discussion of security issues for MS products, including their web server products. Includes security self-assessment information, home and business security information, etc.

Security Focus.Com - Site from Symantec for news, forums, resources, vulnerability info, conference info, tools, etc. related to computer security including web and internet security issues. Search vulnerability database by keywords, date, vendor, version, etc.

Computer Emergency Response Team site - CERT's internet security web site; includes web server security information; hosted by the Software Engineering Institute at Carnegie Mellon University.

 

Web Usability Resources

Prioritizing Web Usability - PDF chapter from 2006 book on Web Usability by Jakob Nielsen and Hoa Loranger.

Usability.gov - Web site with a large collection of web usability resources, information, and guidelines. Although the site was developed by the U.S. federal government for use by various federal agencies, the site is a resource available to anyone.

User Interface Engineering - Web site of User Interface Engineering Inc., founded by Jared M. Spool. Many articles on web site and product usability, such as 'Web Application Form Design', 'Seven Common Usability Testing Mistakes', '5 Things to Know about Users', and more.

Useit.com - Jakob Nielsen's web usability website with such articles as 'How Users Read on the Web", 'Costs of User Testing', and 'Differences between Print Design and Web Design'.

UIWizards.com resources list - Jeff Johnson's UIWizards.com listing of usability design and testing resources