As promised in my earlier entry from today, here's my quick-and-dirty
tutorial on unit testing in PHP using phpt.
First off, phpt test files, from what I can see, were created as part of the
PHP-QA effort. While I cannot find a link
within the PHP-QA site, they have
a page detailing phpt test files, and this page shows all the
sections of a phpt test file, though they do not necessarily show examples
of each.
Also, you might find this
International PHP Magazine article informative; in it Aaron Wormus gives a brief
tutorial on them, as well as some ways to use phpt tests with PHPUnit.
Finally, before I jump in, I want to note: I am not an expert on unit
testing. However, the idea behind unit tests is straightforward: keep your
code simple and modular, and test each little bit (or module) for all types
of input and output. If the code you're testing is a function or class
method, test all permutations of arguments that could be passed to it, and
all possible return values.
Okay, let's jump in!
Overview
The basic format of a phpt test file looks like this:
--TEST--
test name
--FILE--
<?php
// your PHP code goes here
?>
--EXPECT--
Expected output from the PHP code
As you can see, the file is broken into several sections, each beginning
with a --TITLE--. --TEST-- is the name of the test; this could be a function
name, a class name, a class method name, or some free text. Try and make it
meaningful. --FILE-- is the PHP code that will be executed, and --EXPECT--
is the expected output from this PHP code. The test passes if the output
from the PHP code matches what's expected.
There are some other sections you can use as well; I've used the --SKIPIF--
section type to test for which version of PHP is present (Cgiapp2 is
PHP5-only, for instance); if the condition is met, then the test is skipped.
You may also specify --EXPECTF-- or --EXPECTREGEX-- instead of --EXPECT--,
but I found that in most cases, I could control the output from my code such
that neither of those was necessary.
Tips for Writing Tests
First off, my sole experience with phpt tests is testing Cgiapp and Cgiapp2,
which are classes; these tips may not make sense in other situations.
Second, tips are highlighted in bold.
What I found is that you should create one test file per method.
(Generally speaking, that is; I have encountered a few situations where I
needed multiple files, primarily when testing code that uses header().) In
that test file, you then want to test:
- Method Arguments
- Method return value(s)
This means that you'll need to write code for a number of situations. After
writing a few tests, I discovered that it becomes hard to debug if you do
not include informational output in your test code. Create informational
output about what's being tested:
<?php
echo "Test 1: single string argument\n";
?>
These statements are invaluable when a test fails; you can then see what you
were testing at a glance.
If you're using trigger_error() or PEAR_Error in your code (you are, aren't
you?), include an error handler in your test code so you can trap
these and convert them to messages you can format and control.
Supposedly, the --GET-- and --POST-- sections allow you to specify the
variables present in those arrays for the purpose of your tests. However,
this only works on CGI versions of PHP... and, if you're like me, you're
using the CLI SAPI. The easy workaround is to simply build your $_GET and
$_POST arrays in the --FILE-- section.
The same is true for $_SESSION. However, the $_SESSION array will
be present if you specify session_start(); it will simply be empty.
If you need to include a file, include it relative to the test directory. To
determine what that directory is (don't assume it's '.'), use the
construct dirname(__FILE__):
require_once dirname(__FILE__) . '/setup.php.inc';
Running Tests
Once you have a test file, simply execute pear run-tests
testFile.phpt (substituting your test file's name, of course). If you
wish to run several tests at once from several files, you may include each
file's name as an argument. If you want to run all tests in a directory,
simply execute pear run-tests without any arguments.
When tests are run, you will see information on the screen. If a test fails,
the name of the test file and the test name are given.
Debugging a Failed Test
Eventually, a test will fail. It may be that you wrote it incorrectly, or
that you actually have a bug in your code. The question is, how do phpt
tests help you figure out which?
When tests are run on a file, the file is split on its sections. The
--FILE-- section is actually written to a file named after the test file,
but with the .php extension. The --EXPECT-- section is written to a file
with the .exp section; output is piped to a file with the .out section; and
a log of what transpires is written to a file with the .log extension.
Finally, if the test fails, a .diff file is created containing the diff
between the .exp and .out files. For example, if we have a test file named
testFile.phpt, and it fails tests, we'll now have the following
files:
run-tests.log
testFile.diff
testFile.exp
testFile.log
testFile.out
testFile.php
testFile.phpt
Your first stop should be the .diff file. At a glance, you will be able to
see, for instance, if a PHP error occurred. I discovered in several of my
tests that I'd missed semicolons or braces in my test code when I saw syntax
error warnings pop up in these diffs.
If the .diff doesn't explain the differences enough for you, pop open your
.exp and .out files. I use VIM, and I
typically execute a :vsplit so I can load these files up side by side
and compare them. In doing so, I can visually see where the output starts to
differ from the expected. (Several times I discovered typos in my expected,
which meant the tests ran fine after I fixed the typo.)
Remember how I said earlier to create informational output about what's
being tested? This is where it comes into play. What I found is that
output that reads like:
.
.
Bad argument passed
something
is simply harder to understand than:
Test 1: current directory as argument
.
Test 2: no argument passed
.
Test 3: object as argument
Bad argument passed
Test 4: 'something' as argument
something
In the above example, if what was expected for test 2 was something else, I
now know exactly which test in my test file failed -- and that helps me
determine where I might need to go to fix it in my code.
Summary
Tips for Writing Tests
- Create one test file per method
- Create informational output about what's being tested
- Include an error handler in your test code, if errors are being
triggered
- Build your $_GET and $_POST arrays in the --FILE-- section; it's
more portable than --GET-- and --POST--
- use the construct dirname(__FILE__)
Running Tests
- pear run-tests testFile.phpt
- pear run-tests testFile1.phpt testFile2.phpt
- pear run-tests
Debugging a Failed Test
- Examine the .diff file; look for PHP errors in your test code
- Compare the .exp and .out files side-by-side:
- Check for typos in your expected output
- Check informational output to determine which part of the test
failed
Where to go from here
Obviously, the only way to fully understand testing is to do it. There are
plenty of resources on unit testing available; the c2 wiki has some good resources, and
many books cover the subject (The Pragmatic Programmer comes to
mind).
I've read arguments that you should test first the interface. This means
that you don't throw unexpected arguments at a function/method. Later, after
the code matures, you either add tests for the unexpected arguments, or you
add tests for bugs that have been reported. The PHP-QA site recommends
having a test file for the method, but then also having test files that
address specific bugs; I have yet to go that far with testing.
Finally, I have read in a number of resources that true Unit Testing should
start before you start programming. While I understand this
principle to a degree, I also find that as I code, I discover intricacies in
the problem that I could not have anticipated earlier... and the solutions
to those intricacies are often new methods. To that end, I feel that writing
tests should happen after the first draft of code. Doing so provides the
first interface with the code, and also helps code cleanup and bug hunting
before application testing begins. However, this is my humble opinion only.
Happy testing!
