...

Writing my first executable specification

Writing an executable specification for a Calculator program

Consider your customer wants to create a calculator.

...

If you hit the Execute button on the top of the page, you will get the result of your test.

How it works?

Once you click on the Execution button,  calls the actual software containing the code for the Calculator. It sends the parameters x (equal to 6) and y (equal to 3) to the Calculator to obtain the resulting sum of those two numbers. The "?" symbol next to the word "sum" is a key-sign, that indicates to  that a test must be performed.

In that particular example,  will query the calculator application and compare the value returned by the system with the value specified by the business expert (in the "sum?" column). Since the result returned by the system is the same as the expected result, the cell is colored in green by . The business expert is now confident that the Calculator is able to add two numbers.

Adding another example

Once the Calculator is developed, the business expert may think about new examples that he would like to test. The business expert adds another row in the Rule For table and executes the test (no coding is needed here).

...

Adding new specifications to our Calculator

The business expert wants to improve the Calculator: The Calculator should be able to perform product and quotient of two numbers.
The new rules can be written in a way similar to the "sum" specification. In case of expected errors, the keyword error may be used in order to test those special cases.

Rule Validation (Rule For)

Definition

Specific Keywords for expected values

 offers a list of useful keywords to support the Business Expert.

Empty cells	When a test cell is left blank,  only shows the returned value
error	When you expect an error, specify it in the cell to test that particular behavior

Coloring

 will visually show the test result by coloring each testing cell:

...

...

java.lang.ArithmeticException: / by zero
info.novatec.testit.livingdoc.fixture.calculator.Calculator.quotient(Calculator.java:63)

Writing a Rule For Specification

When do we use the RuleForInterpreter?

The RuleForInterpreter is used to express concrete and measurable business rules. The business rules are often related to all kind of calculations.

When a table of rule is executed,  compares the values returned by the system under development against the expected values defined by the Business Expert.

First row: Identification of the Rule For

In a Rule For table, the first row specifies the name of the interpreter and the name of the rules to be tested.
The business expert should be clear in the identification of the set of rules.

...

• Example Mortgage : A Mortgage management system with the following specifications:
  "Whenever the down payment on the property is less than 25% of the sale price, the buyer has to pay a premium for insuring the mortgage. The premium fee amount is 2.5% of the financed amount (which is sale price - down payment)."
  For that particular calculation we could identify the set of rules as:

  rule for

  insurance premium fee calculation

Second row: the Header

The second row is called the header row and serves to distinguish between the given values and the expected values. Given values serve as inputs to the system, whereas expected values serve as comparison values against values that are actually returned by the system. When a column header ends with special characters ? or (), it denotes an expected value.

...

• Example Mortgage : The given values for that example are the sale price and the down payment. The expected value is the premium fee.

  rule for	insurance premium fee calculation
  sale price	down payment	premium fee?

Third and following rows: Examples

The remaining rows capture the examples. They are the executable test rows.

Final expression of the Division example

Final expression of the Mortgage example

Specification: "Whenever the down payment on the property is less than 25% of the sale price, the buyer has to pay a premium for insuring the mortgage. The premium fee amounts is 2.5% of the financed amount (which is sale price - down payment)."

...

Execution of specifications

Based on those executable specifications, the developers are now ready to code the functionality and the fixture (the fixture is the link between the system under development and the executable specification). Once this is done, the specification can be executed by clicking on the Execute button on the top of the page.

...

List Validation (List of, Set of, Superset of, Subset of)

Definition

Specific keywords

None

Coloring

 will visually indicate the test result by coloring the rows:

...

Particular behavior for the superset of interpreters:
Since the SupersetOfInterpreter accepts that the specification may contain rows that are not contained in the SUD,  will not color the rows, which are contained in the specification but not returned by the SUD.

Writing a List specification

When do we use the CollectionInterpreters

The CollectionInterpreters are used to express any kind of group, list or set of values.

...

First row: Identification of the Collection Interpreter

As for all other interpreters, the first row of any CollectionInterpreter specifies the name of the interpreter and the name of the collection to be tested.

• Example - Provinces of Canada: A simple example would be a system containing the Canadian provinces name and code. In that particular case, we could define the collection as Canada Province Codes :

  list of

  Canada Province Codes

• Example - Phone Book : A phone book application able to manage the first name, last name and the phone number of people. The ListOfInterpreter could be expressed as :

  list of

  phone book entries

Other rows: A set of elements

From the third row, we specify the set of values to test.

• Example - Provinces of Canada :

  list of	Canada Province Codes
  Name	Code
  ALBERTA	AB
  BRITISH COLUMBIA	AB

• Example - Phone Book :

  list of	phone book entries
  FirstName	LastName	Phone Number
  Fred	Flintstone	(123) 456-7890
  Barney	Rubble	(123) 321-7666
  Great	Gazoo	(123) 989-4455

Execution of specification

Once the developers have coded the functionality and the fixture(the fixture is the link between the system under development and the executable specification), the specification can be executed by clicking on the Execute button on the top of the page.

...

Since the SupersetOfInterpreter accepts that the specification may contain rows that are not contained in the SUD,  will not color the rows, which are contained in the specification but not returned by the SUD.

Executable specification for the example - Provinces of Canada

Executable specification for the example - Phone Book

Creating the collection of data
We will use the DoWithInterpreter to create our personal phone book.

...

Workflow validation

Definition

Specific Keywords

 offers a list of useful keywords to support the Business Expert. Those keywords are placed at the beginning of an action row.

coloring

 will visually show the test result by coloring each testing cell:

Standard form (without keyword)	Only the Action description will be colored.
Accept	Only the cell containing the keyword Accept will be colored.
Check	The cell containing the expected value will be colored. In the case of a failure,  will show the expected and the returned values.
Reject	Only the cell containing the keyword Reject will be colored.
Display	A new cell at the end of the row will be colored containing the returned value.

Writing a Do With specification

When do we use the DoWithInterpreter

The DoWithInterpreter is used to express interactions with the system under development that must be performed in a particular order. This form of specification provides information about the business flow.

When a sequence of action is executed,  confirms that each action has successfully been performed.

First row: Identification of the Do With

As for all other interpreters, the first row of the DoWithInterpreter specifies the name of the interpreter and the name of the sequence of actions to be tested. What makes the DoWithInterpreter particular is that it only has to be defined once for all the sequences of actions expressed in a page. Obviously, the DoWithInterpreter must be defined before any sequence of actions.

Example of a bank Account management system

Example context :  The system under development must be able to manage two different types of bank account. The system should allow to execute the usual transactions within an account.

...

or more simply as

Following rows: Actions & Verifications

The second and the following rows are used to express specific actions or verifications. The business expert should express the action in human readable format rather than using computer like language.

Example: In our bank example, the first action would be to create a bank account.

If we want to take the account number as a parameter, we would have:

...

Example: Verify the balance of an account

To verify the success of a particular action, we write the keyword "Check" prior to the first part of the action description.
In our example, there would be two parameters: the account number and the amount of balance.

Final expression of our example

Our first business rule says that a new account should have a balance of 0.00 dollars.

...

Execution of specification

Based on those executable specifications, the developers are now ready to code the functionality and the fixture (the fixture is the link between the system under development and the executable specification). Once this is done, the specification can be executed by clicking on the Execute button on the top of the page.

...

Scenario specification

Definition

Coloring

 will visually show the test result by coloring a complete row or words inside the row:

Writing a Scenario specification

When do we use the ScenarioInterpreter

The ScenarioInterpreter is used to express interactions with the system under development that must be performed in a particular order. This form of specification provides information about the business flow.

When a sequence of action is executed,  confirms that each action has successfully been performed.

First row: Identification of the Scenario

As for all other interpreters, the first row of the ScenarioInterpreter specifies the name of the interpreter and the name of the sequence of actions to be tested. What makes the ScenarioInterpreter particular is that it only has to be defined once for all the sequences of actions expressed in a page. Obviously, the ScenarioInterpreter must be defined before any sequence of actions.

Example of a bank Account management system

Example context :  The system under development must be able to manage two different types of bank account. The system should allow to execute the usual transactions within an account.

...

or more simply as

Following rows: Actions & Verifications

The second and following rows are used to express specific actions or verifications. The business expert should express the action in human readable format rather than using computer like language.

Example: In our bank example, the first action would be to create a bank account .

If we want to take the account number as a parameter, we would have:

...

Example: Verify the balance of an account

In our example, there would be two parameters: the account number and the amount of balance.

Final expression of our example

Our first business rule says that a new account should have a balance of 0.00 dollars.

...

Execution of specification

Based on those executable specifications, the developers are now ready to code the functionality and the fixture (the fixture is the link between the system under development and the executable specification). Once this is done, the specification can be executed by clicking on the Execute button on the top of the page.

...

6. Context definition (Setup)

Definition

Coloring

 will visually show the test result by coloring each testing cell:

Writing a Setup specification

When do we use the SetUpInterpreter

The SetUpInterpreter is used to simplify the creation of a particular state for the system under development. Once the state is created, we can focus on the business process to test.

When a setup table is executed,  enters data in the system under development to create the desired state.

First row: Identification of the Set Up

As for all other interpreters, the first row of the setup table specifies the name of the interpreter and the name of the desired state.

Example of a bank Account management system

Example context : A customer should be allowed to transfer money from his or her accounts.

...

Second row: the Header

The second row is called the header row and serves to identify the data to be inserted in the system under development.

Example: In our bank example, the header row could take the following form.

Following rows: Data

The remaining rows capture the data, which is meant to be inserted. They are the executable rows.

Final expression of our example

...

Execution of specification

Based on those executable specifications, the developers are now ready to code the functionality and the fixture (the fixture is the link between the system under development and the executable specification). Once this is done, the specification can be executed by clicking on the Execute button on the top of the page.

...

Advanced

Do With - Table, Bullet List or Number List example

 supports the expression of rules in table format, bullet list format or number list format. The following examples are equivalent and will yield the same results when executed.

It is important to note that the table and bullet list examples are not executable, since the import info.novatec.testit.livingdoc.samples.fixture.bank and do with bank rules cannot be specified twice in the same page. Even though it is possible to change between the two methods in the same page, you must keep in mind that the tests are executed in the same system under development. So it is not necessary to re-specify the same rule in the page.

Table

Bullet List

• import
• info.novatec.testit.livingdoc.samples.application.bank

...

• do with bank
• open checking account 12345-67890 under the name of Spongebob _Squarepants
• deposit $100.00 in account 12345-67890
• check that balance of account 12345-67890 is $100.00

Number List

• import
• info.novatec.testit.livingdoc.samples.application.bank

...

1. do with bank
2. open checking account 12345-67890 under the name of Spongebob _Squarepants
3. deposit $100.00 in account 12345-67890
4. check that balance of account 12345-67890 is $100.00

Object-Graph Navigation Language (OGNL)

 has an extension to supports OGNL expressions. Here are some examples, using the Calculator system:

...

To use this extension, you need to specify the fixture factory as info.novatec.testit.livingdoc.extensions.ognl.OgnlSupport and add the livingdoc-extensions-ognl-x.x.jar and ognl-x.x.x.jar to the classpath of the SUT.

Information Tag

Information tags

When a document is executable,  will try to interpret all the supported forms of the document body.
This could lead to some issues, if we need to insert information as Tables or Bullet lists, for example.

...

Commented forms

Comment tag

If necessary, we can decide that a test shouldn't be executed. Instead of deleting the test from the document, we can comment it. This will force  to skip the test.

The test should be preceded by the tag Comment.

Here are some examples:

Test using the bullet list form.

• Comment
• My bulletlist test to skip

Test using the Table form.

We won't see any coloration of the above table or bullet list after execution of this document.

...