Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) [1] to write tests in a number of popular programming languages, including C#, Java, Groovy, Perl, PHP, Python and Ruby. The tests can then be run against most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms.

[edit] History

Selenium was originally developed by Jason Huggins in 2004, who was later joined by other programmers and testers at ThoughtWorks. It is open-source software, released under the Apache 2.0 license, and can be downloaded and used without charge.

The name comes from a joke made by Huggins in an email, mocking a competitor named Mercury, saying that you can cure mercury poisoning by taking Selenium supplements. The others that received the email took the name and ran with it.[2]

The latest side project is Selenium Grid, which provides a hub allowing the running of multiple Selenium tests concurrently on any number of local or remote systems, thus minimizing test execution time.

[edit] Selenium components

[edit] Selenium IDE

Selenium IDE is a complete integrated development environment (IDE) for Selenium tests. It is implemented as a Firefox extension, and allows recording, editing, and debugging tests. It was previously known as Selenium Recorder. Selenium-IDE was originally created by Shinya Kasatani and donated to the Selenium project in 2006.[citation needed]

Scripts may be automatically recorded and edited manually providing autocompletion support and the ability to move commands around quickly.

Scripts are recorded in Selenese, a special test scripting language for Selenium. Selenese provides commands for performing actions in a browser (click a link, select an option), and for retrieving data from the resulting pages.

Features:

• Record and playback
• Intelligent field selection will use IDs, names, or XPath as needed
• Autocomplete for all common Selenium commands
• Walk through tests
• Debug and set breakpoints
• Save tests as Selenese, Ruby scripts, or other formats
• Support for Selenium user-extensions.js file
• Option to automatically assert the title of every page

[edit] Selenium Client API

As an alternative to writing tests in Selenese, tests can also be written in various programming languages. These tests then communicate with Selenium by calling methods in the Selenium Client API. Selenium currently provides client APIs for Java, C#, Ruby and Python.

With Selenium 2, a new Client API was introduced (with WebDriver as its central component). However, the old API (using class Selenium) is still supported.

[edit] Selenium Remote Control

Selenium Remote Control (RC) is a server, written in Java, that accepts commands for the browser via HTTP. RC makes it possible to write automated tests for a web application in any programming language, which allows for better integration of Selenium in existing unit test frameworks. To make writing tests easier, Selenium project currently provides client drivers for PHP, Python, Ruby, .NET, Perl and Java. The Java driver can also be used with JavaScript (via the Rhino engine). A new instance of selenium RC server is needed to launch html test case - which means that the port should be different for each parallel run.[citation needed] However for Java/PHP test case only one Selenium RC instance needs to be running continuously.[citation needed]

Selenium Remote Control was a refactoring of Driven Selenium or Selenium B designed by Paul Hammant, credited with Jason as co-creator of Selenium. The original version directly launched a process for the browser in question, from the test language of Java, .Net, Python or Ruby. The wire protocol (confusingly called 'Selenese' in its day) was reimplemented in each language port. After the refacor by Dan Fabulich, and Nelson Sproul (with help from Pat Lightbody) there was an intermediate daemon process between the driving test script, and the browser. The benefits included the ability to drive remote browsers, and the reduced need to port every line of code to an increasingly growing set of languages. Selenium Remote Control completely took over from the Driven Selenium code-line in 2006. The browser pattern for 'Driven'/'B' and 'RC' was response/request, which subsequently became known as Comet.

With the release of Selenium 2, Selenium RC has been officially deprecated in favor of Selenium WebDriver.

[edit] Selenium WebDriver

Selenium WebDriver is the successor to Selenium RC. Selenium WebDriver accepts commands (sent in Selenese, or via a Client API) and sends them to a browser. This is implemented through a browser-specific browser driver, which sends commands to a browser, and retrieves results. Most browser drivers actually launch and access a browser application (such as Firefox or Internet Explorer); there is also a HtmlUnit browser driver, which simulates a browser using HtmlUnit.

Unlike in Selenium 1, where the Selenium RC server was necessary to run tests, Selenium WebDriver does not need a special server to execute tests. Instead, the WebDriver directly starts a browser instance and controls it. However, Selenium Grid can be used with WebDriver to execute tests on remote systems (see below).

As of early 2012, Simon Stewart of Google (inventor of WebDriver) and David Burns of Mozilla are negotiating with the W3C to make WebDriver an internet standard. As such, Selenium-Webdriver (Selenium 2.0) aims to be the reference implementation of the WebDriver standard in various programming languages. Currently Selenium-WebDriver is fully implemented and supported in Python, Ruby, Java, and C#.

In practice, this means that the Selenium 2.0 API has significantly fewer calls than does the Selenium 1.0 API. Where Selenium 1.0 attempted to provide a rich interface for many different browser operations, Selenium 2.0 aims to provide a basic set of building blocks from which developers can create their own Domain Specific Language. One such DSL already exists: the Watir project in the Ruby language has a rich history of good design. Watir-webdriver implements the Watir API as a wrapper for Selenium-Webdriver in Ruby. Watir-webdriver is created entirely automatically, based on the WebDriver specification and the HTML specification.

[edit] Selenium Grid

Selenium Grid is a server that allows tests to use web browser instances running on remote machines. With Selenium Grid, one server acts as the hub. Tests contact the hub to obtain access to browser instances. The hub has a list of servers that provide access to browser instances (WebDriver nodes), and lets tests use these instances. Selenium Grid allows to run tests in parallel on multiple machines, and to manage different browser versions and browser configurations centrally (instead of in each individual test).

[edit] See also

[edit] References

1. ^ Selenium Commands – "Selenese" Selenium Documentation, retrieved September 9, 2011
2. ^ Krill, Paul (April 6, 2011). "Open source Selenium web app test suite to support iPhone and Android". InfoWorld. http://news.techworld.com/applications/3272444/open-source-selenium-web-app-test-suite-to-support-iphone-and-android/. Retrieved May 9, 2012. "Selenium was so named because Huggins, dissatisfied with testing tools on the market, was seeking a name that would position the product as an alternative to Mercury Interactive QuickTest Professional commercial testing software. The name, Selenium, was selected because selenium mineral supplements serve as a cure for mercury poisoning, Huggins explained." 

[edit] External links

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. It may involve chemical tests, physical tests, or performance tests.

In systems engineering it may involve black-box testing performed on a system (for example: a piece of software, lots of manufactured mechanical parts, or batches of chemical products) prior to its delivery.[1]

Software developers often distinguish acceptance testing by the system provider from acceptance testing by the customer (the user or client) prior to accepting transfer of ownership. In the case of software, acceptance testing performed by the customer is known as user acceptance testing (UAT), end-user testing, site (acceptance) testing, or field (acceptance) testing.

A smoke test is used as an acceptance test prior to introducing a build to the main testing process.

[edit] Overview

Testing generally involves running a suite of tests on the completed system. Each individual test, known as a case, exercises a particular operating condition of the user's environment or feature of the system, and will result in a pass or fail, or boolean, outcome. There is generally no degree of success or failure. The test environment is usually designed to be identical, or as close as possible, to the anticipated user's environment, including extremes of such. These test cases must each be accompanied by test case input data or a formal description of the operational activities (or both) to be performed—intended to thoroughly exercise the specific case—and a formal description of the expected results.

Acceptance Tests/Criteria (in Agile Software Development) are usually created by business customers and expressed in a business domain language. These are high-level tests to test the completeness of a user story or stories 'played' during any sprint/iteration. These tests are created ideally through collaboration between business customers, business analysts, testers and developers, however the business customers (product owners) are the primary owners of these tests. As the user stories pass their acceptance criteria, the business owners can be sure of the fact that the developers are progressing in the right direction about how the application was envisaged to work and so it's essential that these tests include both business logic tests as well as UI validation elements (if need be).

Acceptance test cards are ideally created during sprint planning or iteration planning meeting, before development begins so that the developers have a clear idea of what to develop. Sometimes (due to bad planning!) acceptance tests may span multiple stories (that are not implemented in the same sprint) and there are different ways to test them out during actual sprints. One popular technique is to mock external interfaces or data to mimic other stories which might not be played out during an iteration (as those stories may have been relatively lower business priority). A user story is not considered complete until the acceptance tests have passed.

[edit] Process

The acceptance test suite is run against the supplied input data or using an acceptance test script to direct the testers. Then the results obtained are compared with the expected results. If there is a correct match for every case, the test suite is said to pass. If not, the system may either be rejected or accepted on conditions previously agreed between the sponsor and the manufacturer.

The objective is to provide confidence that the delivered system meets the business requirements of both sponsors and users. The acceptance phase may also act as the final quality gateway, where any quality defects not previously detected may be uncovered.

A principal purpose of acceptance testing is that, once completed successfully, and provided certain additional (contractually agreed) acceptance criteria are met, the sponsors will then sign off on the system as satisfying the contract (previously agreed between sponsor and manufacturer), and deliver final payment.

[edit] User acceptance testing

User Acceptance Testing (UAT) is a process to obtain confirmation that a system meets mutually agreed-upon requirements. A Subject Matter Expert (SME), preferably the owner or client of the object under test, provides such confirmation after trial or review. In software development, UAT is one of the final stages of a project and often occurs before a client or customer accepts the new system.

Users of the system perform these tests, which developers derive from the client's contract or the user requirements specification.

Test-designers draw up formal tests and devise a range of severity levels. Ideally the designer of the user acceptance tests should not be the creator of the formal integration and system test cases for the same system. The UAT acts as a final verification of the required business function and proper functioning of the system, emulating real-world usage conditions on behalf of the paying client or a specific large customer. If the software works as intended and without issues during normal use, one can reasonably extrapolate the same level of stability in production.

User tests, which are usually performed by clients or end-users, do not normally focus on identifying simple problems such as spelling errors and cosmetic problems, nor showstopper defects, such as software crashes; testers and developers previously identify and fix these issues during earlier unit testing, integration testing, and system testing phases.

The results of these tests give confidence to the clients as to how the system will perform in production. There may also be legal or contractual requirements for acceptance of the system.

[edit] Q-UAT - Quantified User Acceptance Testing

Quantified User Acceptance Testing (Q-UAT or, more simply, the "Quantified Approach") is a revised Business Acceptance Testing process which aims to provide a smarter and faster alternative to the traditional UAT phase.[citation needed] Depth-testing is carried out against business requirements only at specific planned points in the application or service under test. A reliance on better quality code-delivery from the development/build phase is assumed and a complete understanding of the appropriate business process is a pre-requisite. This methodology - if carried out correctly - results in a quick turnaround against plan, a decreased number of test scenarios which are more complex and wider in breadth than traditional UAT and ultimately the equivalent confidence-level attained via a shorter delivery-window, allowing products/changes to come to market quicker.

The Q-UAT approach depends on a "gated" three-dimensional model. The key concepts are:

1. Linear Testing (LT, the 1st dimension)
2. Recursive Testing (RT, the 2nd dimension)
3. Adaptive Testing (AT, the 3rd dimension).

The four[which?] "gates" which conjoin and support the 3-dimensional model act as quality safeguards and include contemporary testing concepts such as:

• Internal Consistency Checks (ICS)
• Major Systems/Services Checks (MSC)
• Realtime/Reactive Regression (RTR).

The Quantified Approach was shaped by the former "guerilla" method of acceptance testing which was itself a response to testing phases which proved too costly to be sustainable for many small/medium-scale projects.

[edit] Acceptance testing in Extreme Programming

Acceptance testing is a term used in agile software development methodologies, particularly Extreme Programming, referring to the functional testing of a user story by the software development team during the implementation phase.

The customer specifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, whatever it takes to ensure the functionality works. Acceptance tests are black box system tests. Each acceptance test represents some expected result from the system. Customers are responsible for verifying the correctness of the acceptance tests and reviewing test scores to decide which failed tests are of highest priority. Acceptance tests are also used as regression tests prior to a production release. A user story is not considered complete until it has passed its acceptance tests. This means that new acceptance tests must be created for each iteration or the development team will report zero progress.[2]

[edit] Types of acceptance testing

Typical types of acceptance testing include the following

User acceptance testing

This may include factory acceptance testing, i.e. the testing done by factory users before the factory is moved to its own site, after which site acceptance testing may be performed by the users at the site.

Operational Acceptance Testing (OAT)

Also known as operational readiness testing, this refers to the checking done to a system to ensure that processes and procedures are in place to allow the system to be used and maintained. This may include checks done to back-up facilities, procedures for disaster recovery, training for end users, maintenance procedures, and security procedures.

Contract and regulation acceptance testing

In contract acceptance testing, a system is tested against acceptance criteria as documented in a contract, before the system is accepted. In regulation acceptance testing, a system is tested to ensure it meets governmental, legal and safety standards.

Alpha and beta testing

Alpha testing takes place at developers' sites, and involves testing of the operational system by internal staff, before it is released to external customers. Beta testing takes place at customers' sites, and involves testing by a group of customers who use the system at their own locations and provide feedback, before the system is released to other customers. The latter is often called “field testing”.

[edit] List of development to production (testing) environments

• Development Environment
• Development Testing Environment
• Testing Environment
• Development Integration Testing
• Development System Testing
• System Integration Testing
• User Acceptance Testing
• Production Environment

[edit] List of acceptance-testing frameworks

[edit] See also

[edit] References

[edit] External links

In software development and product management, a user story is one or more sentences in the everyday or business language of the end user or user of a system that captures what a user does or needs to do as part of his or her job function. User stories are used with Agile software development methodologies as the basis for defining the functions a business system must provide, and to facilitate requirements management. It captures the 'who', 'what' and 'why' of a requirement in a simple, concise way, often limited in detail by what can be hand-written on a small paper notecard. User stories are written by or for the business user as that user's primary way to influence the functionality of the system being developed. User stories may also be written by developers to express non-functional requirements (security, performance, quality, etc.)[1], though primarily it is the task of a product manager to ensure user stories are captured.

User stories are a quick way of handling customer requirements without having to create formalized requirement documents and without performing administrative tasks related to maintaining them. The intention of the user story is to be able to respond faster and with less overhead to rapidly changing real-world requirements.

A user story is an informal statement of the requirement as long as the correspondence of acceptance testing procedures is lacking. Before a user story is to be implemented, an appropriate acceptance procedure must be written by the customer to ensure by testing or otherwise determine whether the goals of the user story have been fulfilled. Some formalization finally happens when the developer accepts the user story and the acceptance procedure as a work specific order.

[edit] Creating user stories

When the time has come for creating user stories, one of the developers (or the product owner in Scrum) gets together with a customer representative. The customer is responsible for formulating the user stories. The developer may use a series of questions to get the customer going, such as asking if some particular functionality is desired, but must be careful not to dominate the idea creation process.

As the customer conceives the user stories, they are written down on a note card (e.g. 3x5 inches or 8x13 cm) with a name and a description which the customer has formulated. If the developer and customer find that the user story is lacking in some way (too large, complicated, imprecise), it is rewritten until it is satisfactory often using the INVEST guidelines from the Scrum project management framework. However, it is stressed in Extreme Programming (XP) that user stories are not to be definite once they have been written down. Requirements tend to change during the development period, which is handled by not carving them in stone.

User stories generally follow the following template:

"As a <role>, I want <goal/desire> so that <benefit>"

but the shorter version is commonly used as well:

"As a <role>, I want <goal/desire>"

Some practitioners place the benefit at the front, in order to keep the benefit at the forefront:

"In order to <receive benefit> as a <role>, I want <goal/desire>"

[edit] Examples

As a user, I want to search for my customers by their first and last names.

As a non-administrative user, 
I want to modify my own schedules but not the schedules of other users.

As a mobile application tester, 
I want to test my test cases and report results to my management.

Starting Application
The application begins by bringing up the last document the user was working with.

As a user closing the application, 
I want to be prompted to save if I have made any change in my data since the last save.

Closing Application
Upon closing the application, the user is prompted to save (when ANYTHING has changed in data
since the last save!).

alternatively...

As a user closing the application,
I want to be prompted to save anything that has changed since the last save
so that I can preserve useful work and discard erroneous work.

The consultant will enter expenses on an expense form. The consultant will enter items
on the form like expense type, description, amount, and any comments regarding the expense.
At any time the consultant can do any of the following options.
(1) Once this is completed the consultant will “Submit”. If the expense is under fifty (<50),
the expense will go directly to the system for processes.
(2) In the event the consultant has not finished entering the expense, the consultant may
want to “Save for later”. This instance should then be displayed on a list (queue) for
consultant with the status of “Incomplete”.
(3) In the event the consultant decides to clear the data and close the form the consultant
will “Cancel and exit”. This instance will not be saved anywhere.

As a central part of many agile development methodologies, such as in XP's planning game, user stories define what is to be built in the software project. User stories are prioritized by the customer to indicate which are most important for the system and will be broken down in tasks and estimated by the developers.

When user stories are about to be implemented the developers should have the possibility to talk to the customer about it. The short stories may be difficult to interpret, may require some background knowledge or the requirements may have changed since the story was written.

Every user story must at some point have one or more acceptance tests attached, allowing the developer to test when the user story is done and also allowing the customer to validate it. Without a precise formulation of the requirements, prolonged nonconstructive arguments may arise when the product is to be delivered.

[edit] Benefits

XP and other agile methodologies favor face-to-face communication over comprehensive documentation and quick adaptation to change instead of fixation on the problem. User stories achieve this by:

• Being very short. They represent small chunks of business value that can be implemented in a period of days to weeks.
• Allowing developer and the client representative to discuss requirements throughout the project lifetime
• Needing very little maintenance
• Only being considered at the time of use
• Maintaining a close customer contact
• Allowing projects to be broken into small increments
• Being suited to projects where the requirements are volatile or poorly understood. Iterations of discovery drive the refinement process.
• Making it easier to estimate development effort
• Require close customer contact throughout the project so that the most valued parts of the software get implemented.

[edit] Limitations

Some of the limitations of user stories in agile methodologies:

• They can be difficult to scale to large projects.
• They are regarded as conversation starters.

[edit] User stories and use cases

While both user stories and use cases serve the purpose to capture specific user requirements in terms of interactions between the user and the system, there are major differences between them.

User Stories

Use Cases

Provide a small-scale and easy-to-use presentation of information. Are generally formulated in the everyday language of the user and contain little detail, thus remaining open to interpretation. They should help the reader understand what the software should accomplish. Must be accompanied by Acceptance Testing procedures (acceptance criteria) for clarification of behavior where stories appear ambiguous.

Describe a process and its steps in detail, and may be worded in terms of a formal model. A use case is intended to provide sufficient detail for it to be understood on its own. A use case has been described as “a generalized description of a set of interactions between the system and one or more actors, where an actor is either a user or another system”.[2] May be delivered in a stand-alone document.

[edit] See also

[edit] References

• Daniel H. Steinberg and Daniel W. Palmer: Extreme Software Engineering, Pearson Education, Inc., ISBN 0-13-047381-2
• Mike Cohn, "User Stories Applied", 2004, Addison Wesley, ISBN 0-321-20568-5
• Mike Cohn: Agile Estimating and Planning, 2006, Prentice Hall, ISBN 0-13-147941-5

1. ^ Davies, Rachel. "Non-Functional Requirements: Do User Stories Really Help?". http://www.methodsandtools.com/archive/archive.php?id=113. Retrieved 12 May 2011. 
2. ^ Advantages of User Stories for Requirements

[edit] External links