Testing A Class - Two Simple Examples

The following examples demonstrates how to perform fully automatic testing on two classes: one without Design by Contract comments and one with these comments.

Example 1: Testing a Class Without Design by Contract Comments

This example demonstrates how Jtest tests a single class file that does not contain Design by Contract comments.

1. Go to Jtest's Class Testing UI. (This UI opens by default when you launch Jtest).
2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.
3. Browse to Simple.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.
4. Click the Start button in the tool bar.

Jtest will perform static analysis, then automatically create and execute white-box test cases designed to test the class's construction. A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. Errors found will be reported in the Errors Found panel.

Static Analysis Violations

The following coding standard violations will be reported in the Static Analysis Violations branch of the Errors Found panel.

To see more information about a violation, expand the violation's branch. For example, expand the violation of the PB.TLS rule.

This violation reveals that the developer inadvertently wrote case10 instead of case 10. If the class is not fixed, it will give incorrect results when it is passed the value 10. To view the source code of the class (with the line containing the error highlighted), double-click the node containing the error's file/line information.

Uncaught Runtime Exceptions

Now let's look at the uncaught runtime exception that Jtest's white-box test cases found. The Errors Found panel will list the following uncaught runtime exception under the Uncaught Runtime Exceptions branch.

This error message reveals that there is some input for which the class will throw an uncaught runtime exception at runtime. This could cause the application running this class to crash.

To see a stack trace like the one the Java virtual machine would give if this uncaught runtime exception were thrown, expand this branch.

To see an example usage of this class that would lead to the reported uncaught runtime exception, expand the Test Case Input branch.

This error message reveals that the startsWith method is implemented incorrectly. The method should return false for the argument "" and "0" instead of throwing a runtime exception. If the error is not fixed, any application using this class will eventually crash or give incorrect results.

To view the source code of the class (with the problematic line of the stack trace highlighted), double-click the node containing the exception's file/line information.

To see a sample of the test cases that Jtest automatically created, click the View button to open the View Test Cases window. In the View Test Cases window, Control-right-click the Automatic Test Cases node, then choose Expand Children from the shortcut menu.

Regression Testing

Jtest doesn't display any regression errors on the first run through a class because it is impossible to detect a regression error the first time a class is tested. Regression testing checks that class outcomes don't change, so it always needs a first run for reference.

To see how regression testing works, introduce an error into Simple.java and run it again.

1. Copy Simple_re.java into Simple.java (both classes are located in <jtest_install_dir>/examples/eval).
2. Recompile Simple.java.
3. Retest Simple.class.

Now, along with the other errors, Jtest reports the following regression errors in the Errors Found panel:

Expand the error messages to see the inputs for which these regression errors occur. The first error tells us that the code has changed and that the method "add" is now returning 3 instead of 0 for the input 0, 0. The second error reveals that the method "add" is now returning 17 instead of 14 for the input 7,7.

Example 2: Testing a Class With Design by Contract Comments

This example demonstrates how Jtest tests a single class file that contains Design by Contract-format specification information.

1. Go to Jtest's Class Testing UI. (This UI opens by default when you launch Jtest).
2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.
3. Browse to Example.class (in <jtest_install_dir>/examples/eval) using the Browse button in the Class Name panel.
4. Click the Start button in the tool bar.

Jtest will perform both static and dynamic on the class. Because the code's specification is incorporated into the code using Design by Contract comment tags, Jtest can fully automate black-box (functionality) testing as well as white-box (construction) testing. Jtest will automatically create and execute black-box test cases that verify the functionality described in the class's postcondition contract. It will also create and execute white-box test cases that check how the class handles a wide range of inputs.

A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. This test uncovers one Design by Contract violation, one uncaught runtime exception, and one static analysis violation.

Design by Contract Violations

The following Design by Contact violation will be reported in the Design by Contract Violations branch of the Errors Found panel.

This violation indicates that one of the class's methods did not function as described in the specification. To see more information about this violation, expand the violation's branch.

To open the source code of the class in an editor, right-click the Source button, then choose Edit Source from the shortcut menu.

The source file reveals that the code's @post contract (postcondition) requires the method to return the value of a+b. However, as Jtest revealed, it does not function as specified: the method actually returns the value of a-b. If this were your own class, you would now fix the problem, recompile the class, then retest it to check if your modifications fixed the problem.

To see a sample of the test cases that Jtest automatically created to test this class's functionality, click the View button to open the View Test Cases window. In the View Test Cases window, Control-right-click the Automatic Test Cases node, then choose Expand Children from the shortcut menu.

Uncaught Runtime Exceptions

Next, go to the uncaught runtime exception found (located in the Uncaught Runtime Exceptions branch of the Errors Found panel) and expand its branches.

This error message shows that a NegativeArraySizeException occurs when a negative index is used as an index to an array. This is an expected exception. If this were your code, you would want to document this exception in your code by adding the following Design by Contract Javadoc comment above the method:

 /** @exception java.lang.NegativeArraySizeException */

By adding this comment, you make the code easier to maintain. Someone looking at the code later on will immediately know that the method is throwing an exception because the code is supposed to throw an exception, not because the code has a bug. In addition, you tell Jtest to suppress future occurrences of this exception.