LogoPhly, boy, phly
the weblog and site of Matthew Weier O'Phinney

Thursday, September 11. 2008

Setting up your Zend_Test test suites

Now that Zend_Test has shipped, developers are of course asking, "How do I setup my test suite?" Fortunately, after some discussion with my colleagues and a little experimenting on my one, I can answer that now.

PHPUnit offers a variety of methods for setting up test suites, some trivial and some complex. The Zend Framework test suite, for instance, goes for a more complex route, adding component-level suites that require a fair amount of initial setup, but which allow us fairly fine-grained control.

However, testing and test automation should be easy and the complex approach is overkill for most of our applications. Fortunately, PHPUnit offers some other methods that make doing so relatively simple. The easiest method is to use an XML configuration file.

As an example, consider the following:


<phpunit>
    <testsuite name="My Test Suite">
        <directory>./</directory>
    </testsuite>

    <filter>
        <whitelist>
            <directory suffix=".php">../library/</directory>
            <directory suffix=".php">../application/</directory>
            <exclude>
                <directory suffix=".phtml">../application/</directory>
            </exclude>
        </whitelist>
    </filter>

    <logging>
        <log type="coverage-html" target="./log/report" charset="UTF-8"
            yui="true" highlight="true"
            lowUpperBound="50" highLowerBound="80"/>

        <log type="testdox-html" target="./log/testdox.html" />
    </logging>
</phpunit>
 

First thing to note, relative paths are relative to the configuration file. This allows you to run your tests from anywhere in your tests tree, Second, providing a directory directive to the testsuite directive scans for all files ending in "Test.php" in that directory, meaning you don't have to keep a list of your test cases manually. It's a great way to automate the suite. Third, the filter directive allows us to determine what classes to include and/or exclude from coverage reports. Finally, the logging directive lets us specify what kinds of logs to create and where.

Drop the above into "tests/phpunit.xml" in your application, and you can start writing test cases and running the suite immediately, using the following command:


% phpunit --configuration phpunit.xml

I like to group my test cases by type. I have controllers, models, and often library code, and need to keep the tests organized both on the filesystem as well as for running the actual tests. There are two things I do to facilitate this.

First, I create directories. For instance, I have the following hierarchy in my test suite:


tests/
    phpunit.xml
    TestHelper.php
    controllers/
        IndexControllerTest.php (contains IndexControllerTest)
        ErrorControllerTest.php (contains ErrorControllerTest)
        ...
    models/
        PasteTest.php           (contains PasteTest)
        DbTable/
            PasteTest.php       (contains DbTable_PasteTest)
        ...
    My/
        Form/
            Element/
                SimpleTextareaTest.php

"controllers/" contains my controllers, "models/" contains my models. If I were developing a modular application, I'd have something like "blog/controllers/" instead. Library code is given the same hierarchy as is found in my "library/" directory.

Second, I use docblock annotations to group my tests. I add the following to my class-level docblock in my controller test cases:


/**
 * @group Controllers
 */

 

Models get the annotation "@group Models", etc. This allows me to run individual sets of tests on demand:


% phpunit --configuration phpunit.xml --group Controllers

You can specify multiple @group annotations, which means you can separate tests into modules, issue report identifiers, etc; additionally, you can add the annotations to individual test methods themselves to have really fine-grained test running capabilities.

Astute readers will have noticed the "TestHelper.php" file in that directory listing earlier, and will be wondering what that's all about.

A test suite needs some environmental information, just like your application does. It may need a default database adapter, altered include_paths, autoloading set up, and more. Here's what my TestHelper.php looks like:


<?php
/*
 * Start output buffering
 */

ob_start();

/*
 * Set error reporting to the level to which code must comply.
 */

error_reporting( E_ALL | E_STRICT );

/*
 * Set default timezone
 */

date_default_timezone_set('GMT');

/*
 * Testing environment
 */

define('APPLICATION_ENV', 'testing');

/*
 * Determine the root, library, tests, and models directories
 */

$root        = realpath(dirname(__FILE__) . '/../');
$library     = $root . '/library';
$tests       = $root . '/tests';
$models      = $root . '/application/models';
$controllers = $root . '/application/controllers';

/*
 * Prepend the library/, tests/, and models/ directories to the
 * include_path. This allows the tests to run out of the box.
 */

$path = array(
    $models,
    $library,
    $tests,
    get_include_path()
);
set_include_path(implode(PATH_SEPARATOR, $path));

/**
 * Register autoloader
 */

require_once 'Zend/Loader.php';
Zend_Loader::registerAutoload();

/**
 * Store application root in registry
 */

Zend_Registry::set('testRoot', $root);
Zend_Registry::set('testBootstrap', $root . '/application/bootstrap.php');

/*
 * Unset global variables that are no longer needed.
 */

unset($root, $library, $models, $controllers, $tests, $path);
 

The above ensures that my APPLICATION_ENV constant is set appropriately, that error reporting is appropriate for tests (i.e., I want to see all errors), and that autoloading is enabled. Additionally, I place a couple items in my registry -- the bootstrap and test root directory.

In each test case file, I then do a require_once on this file. In future versions of PHPUnit, you'll be able to specify a bootstrap file in your configuration XML that gets pulled in for each test case, and you'll be able to even further automate your testing environment setup.

Hopefully this will get you started with your application testing; what are you waiting for?

Posted by Matthew Weier O'Phinney in PHP at 15:00 | Comments (31) | Trackback (1)
Defined tags for this entry: best practices, mvc, oop, php, testing, zend framework
Related entries by tags:
Zend Framework 1.7.0 Released
Vimgrep and Vim Project
Pastebin app updates
ZendCon08 Wrapup
git-svn Tip: don't use core.autocrlf

Trackbacks
Trackback specific URI for this entry

Testsuite für Zend Framework Tests aufbauen
Matthew hat in seinem Blog einen Beitrag geschrieben, wie man Zend_Test und eine Testsuite aufsetzen kann. Der Beitrag beschreibt den Aufbau eine XML Konfigurationsdatei für PHPUnit, stellt ein Beispiel für die Verzeichnisstruktur für eine Testsuite...
Weblog: Ralfs Zend Framework und PHP Blog
Tracked: Sep 12, 01:30

Comments
Display comments as (Linear | Threaded)

Unit testing is a great idea. However in the case of Zend_Framework, as for today, it's far too complicated to be put in place. I'm an experienced php developper and I've spent those last 2 days trying to make this work with no results so far (either because of phpunit bugs, either because it won't work with ZF, etc.). So, for young php developpers, unit testing is far out of reach - at least with ZF.

There's a lot of work to be done in making things simpler ; good simple tutorials are needed to democratize this technology.
#1 Zed on 2008-09-12 05:57 (Reply)
I'm sorry you're having issues getting testing to work. I realize that I live and breath tests at this point, so much of what I do seems simple to *me*, but may not be to others.

If you are having issues with Zend Framework components, the best places to get assistance are the mailing lists (fw-mvc and fw-general, in particular) and IRC (#zftalk on freenode). As soon as you hit a stumbling block, I suggest using those resources -- so that you can waste minutes or hours, not days.

You say you need a good simple tutorial -- what in this particular tutorial could be made easier for you, so that I can improve the experience for beginners?
#1.1 Matthew Weier O'Phinney (Link) on 2008-09-12 08:14 (Reply)
Thanks for your reply and for the links. I'll check this out immediately.

A serie of very short tutorials would be great : short texts (ex : 1 page - Word format -, not more) with immediate results everyone can try.
Ex :
1. Test a simple ZF project with one controller, in order to setup the framework. Steps : Create the new ZF project. Add 5 lines of code to the controller. Test the controller (something usefull).
Where do I put the test files in my project ? Do I have to use phpunit in command line ?

2nd tutorial. Discover the basic test mecanisms
3. Discover ZF tests - very simple tests.
4. ... more concrete exemples

Etc. What novice testers need are complete steps & code in order to make it fully operationnal. We souldn't even have to understand how it work in order to play with it ;-)

Thanks for your concern on this matter !
#1.1.1 Zed on 2008-09-12 09:09 (Reply)
Thanks for the feedback! I'll see what I can come up with. Keep an eye on my blog and the ZF site in the coming weeks.

By the way, I'd change (1) from "Add 5 lines of code... Test..." to "Write a test specifying the expected behavior... Write the code to match the expectation..." :-)
#1.1.1.1 Matthew Weier O'Phinney (Link) on 2008-09-12 09:36 (Reply)
Great !
Oh by the way I found another post of a user that has pretty much the same problems I had :
http://www.zfforums.com/zend-framework-general-discussions-1/installation-configuration-3/zend_test-how-set-up-1569.html

In fact, ZF really misses good documentation exemples. Like we can find in php.net in the user comments. It would really be nice to have more concrete exemples in order to make it work.

I posted an issue about Zend_Form (captcha validation needed once) earlier and you replied with an exemple. That's exactly what we miss in order to make ZF a great framework. Good and practical exemples.

Z.
#1.1.1.1.1 Zed on 2008-09-12 10:42 (Reply)
Thanks for the link; I'll check that out soon.

BTW, we've debated adding comments to our manual, and are looking into the legal issues (if we incorporate any comments into the source at any point, we'd need to be sure of the intellectual property origin). I think we'll be able to accomodate it in some fashion before long.

As for "That's exactly what we miss in order to make ZF a great framework," I think you'll get a number of people who claim it already is. :-) But I agree -- we need more examples and use cases. Feel free to sign up as a contributor to help the documentation!
#1.1.1.1.1.1 Matthew Weier O'Phinney (Link) on 2008-09-12 13:12 (Reply)
No, you're right, ZF is a great framework, I didn't mean to underappreciate it's value when I was talking about how to make it better !
Concerning the legal issues, I'm replying to you on your personnal email.

Th.
#1.1.1.1.1.1.1 Zed on 2008-09-13 06:15 (Reply)
What I'd very much like to see is a tutorial on creating unit tests for existing projects. I have a fairly complex project spanning different modules, making heavy use of the actionstack helper and such. I'm coming across a lot of problems trying to create unit test coverage, so a tutorial on creating unit tests for existing projects would be very nice :-) (Not sure if it's reasonable or not)
#1.1.1.2 Vincent (Link) on 2008-09-12 10:48 (Reply)
You're saying that relative paths are relative to the configuration file. That doesn't seem to be true (at least on Windows) running it from the command line. Instead it seems to be relative to HOMEPATH of the user.
#2 Peter on 2008-09-12 10:25 (Reply)
Interesting to note -- I'm on *nix systems, and the behavior is as I described. I'll make a note that this isn't true on Windows.
#2.1 Matthew Weier O'Phinney (Link) on 2008-09-12 13:09 (Reply)
Here's a little example of something working :
Here's my setup :

My base directory structure :
/usr/local/www/application/controllers
/usr/local/www/tests/

So, I'm trying to test my UserController.
I wrote a file UserControllerTest.php that I put in /usr/local/www/tests ; here's it's content :



This is a very basic setup, I've not managed to go further for now, but it seems it's working, so I thought I'd post this for the orther guys out there discovering unit tests with ZF.
#3 Zed on 2008-09-13 09:37 (Reply)
Hello,
I have recently Used Zend_test . I use startMVC in my bootstrap file.
The problem is when i run my tests which asserts the controllers in which i set the layout , It gives me Debug warning like this for Action Helpers :
Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/HeadLink.php) [function.fopen]: failed to open stream: No such file or directory
what could be the problem. Please Help me do this.
Regards,
Ramin
#4 Ramin on 2008-10-10 15:39 (Reply)
First off, that's not an action helper, but a view helper. Second, are you seeing those errors in your logs, or somewhere else?

Zend_View, in current released versions, calls Zend_Loader::isReadable(), which in turn calls @fopen() with the third parameter, which tells it to look on the include_path. This is more efficient than manually looping over the include_path. Note that the fopen call has error suppression; this is to *prevent* such errors as you're seeing from being displayed. However, they will still be reported in your *logs*. Hence my questions.

BTW, with 1.7.0, any class using the PluginLoader, which includes Zend_View, will no longer use isReadable(), but instead rewrite the include_path briefly. From benchmarks I've done, this has around a 12.5% performance boost on real-world applications. 1.7.0 is right around the corner...
#4.1 Matthew Weier O'Phinney (Link) on 2008-10-10 16:54 (Reply)
Dear Matthew,
These warnings come in zend Studio Console Panel in tests written for the actions where i setLayout .
as i have used headTitle , headScript, HeadLink, and ..
The exact console output in zend studio is :

Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/HeadLink.php) [function.fopen]: failed to open stream: No such file or directory
Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/HeadStyle.php) [function.fopen]: failed to open stream: No such file or directory
Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/Partial.php) [function.fopen]: failed to open stream: No such file or directory
Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/Doctype.php) [function.fopen]: failed to open stream: No such file or directory
Debug Warning: /trunk/library/Zend/Loader.php line 160 - fopen(d:/Projects/UKStudy_test/trunk/application/modules/admin/views\helpers/Layout.php) [function.fopen]: failed to open stream: No such file or directory

which annoys me. When i resetMVCInstanse those warnings get away But the action with the Layout set in it wont assert

This is my Bootstrap file :
* Setting default time zone
*/
date_default_timezone_set('GMT');

set_include_path('.' . PATH_SEPARATOR . 'd:/Projects/UKStudy_test/trunk/library/' .PATH_SEPARATOR.'d:/Projects/UKStudy_test/trunk/Test/'. PATH_SEPARATOR . 'd:/Projects/UKStudy_test/trunk/application/modules/admin/models/' . PATH_SEPARATOR . get_include_path());
require_once('Zend/Loader.php');
Zend_Loader::registerAutoload();

$registry = Zend_Registry::getInstance();
$config = new Zend_Config_Ini('d:/Projects/UKStudy_test/trunk/application/config.ini','general');
$db = Zend_db::factory($config->db);
$registry->set('config',$config);
$registry->set('db',$db);
Zend_Db_Table::setDefaultAdapter($db);

// Setup controller
//setting Module Directory and its Controllers
$controller = Zend_Controller_Front::getInstance();
$controller->throwExceptions(false); // should be turned on in development time
//$controller->addModuleDirectory('d:/Projects/UKStudy_test/trunk/application/modules');

// bootstrap layouts
Zend_Layout::startMvc(array(
'layoutPath' => 'd:/Projects/UKStudy_test/trunk/application/layouts',
'layout' => 'Ukstudy'
//TODO: fetch layout and theme from db
));

//adding fireBug Writer
$logger = new Zend_Log();
$writer = new Zend_Log_Writer_Firebug();
$logger->addWriter($writer);
$registry->set('logger',$logger);
//Starting Zend session so it May not be sent after Headers
Zend_Session::start();
// run!
//removing Dispatch for tests
//$controller->dispatch();
#4.1.1 Ramin on 2008-10-11 04:46 (Reply)
Please contact me via my contact form or one of the ZF mailing lists -- this is too difficult to troubleshoot via comments.
#4.1.1.1 Matthew Weier O'Phinney (Link) on 2008-10-11 10:26 (Reply)
Dear matthew which mailing list should i join so i could talk about this !?
core ?!
General ?!
MVC ?
Documentations ?!
Regards
#4.1.1.1.1 ramin on 2008-10-12 07:33 (Reply)
Either fw-general or fw-mvc.
#4.1.1.1.1.1 Matthew Weier O'Phinney (Link) on 2008-10-12 10:38 (Reply)
Hello there,
I may want to ask how Zend_test can work with Exceptions we throw in our code ?
is there any type of assertion which asserts the Exception Message ?!
#5 Ramin on 2008-10-13 09:32 (Reply)
You have a few options here. First, if you are using the ErrorHandler plugin properly, you'll get to the ErrorController's errorAction() -- so you can test to see if you got that. Second, once you have, either test for content (if you render exception messages in the view script), or pull the "error_handler" parameter from the request object, and test against it's "exception" property.
#5.1 Matthew Weier O'Phinney (Link) on 2008-10-13 10:26 (Reply)
Dear Matthew ,
is there any method in zend_test or PHPunit by which you can put dependancies on tests ?
for example i have a cut and paste methods which i can test individually.
but as it shows, the paste is dependant to cut. i mean testing pasteAction does not satisfy if if dont test the cut.
Is there any method or command by which you can say forexample :
test PasteAction if testCutAction was correct ?!
Regards,
Ramin
#6 Ramin on 2008-10-16 08:16 (Reply)
No, because PHPUnit itself does not yet support this.
#6.1 Matthew Weier O'Phinney (Link) on 2008-10-16 09:23 (Reply)
Thanks for the tutorial.

I am not sure i totally understand how to use the TestHelper.php file. It seems like this is a replacement for your bootstrap.php file, is this correct?

For example, in the Zend_Test documentation there are several methods described for bootstrapping before each *ControllerTest is run. In one instance it recommends moving much of the bootstrap setup to a plugin, and then calling the a function from "setUp()" that adds the plugin to the front controller.

If we use the TestHelper.php files as described above, it seems we can ignore setting up the bootstrap in the "setUp()" function.

Is that right?

thanks,

Elliot
#7 Elliot Cohen on 2008-11-02 22:52 (Reply)
Actually, no. The TestHelper.php file is for bootstrapping your PHPUnit environment, not the ZF application environment. Typically, you need to setup a different include_path when testing, and may need to specify particular globals and/or settings that are test environment specific. Take a look at the ZF test suite's bootstrap to get an idea of what this file usually does.

Your ZF bootstrap is needed in addition to this, as it will setup your ZF application -- configuration, database connections, view settings, etc.
#7.1 Matthew Weier O'Phinney (Link) on 2008-11-03 06:43 (Reply)
Sorry Matthew but I also don't really get this TestHelper file, Im still starting my suite using phpunit --configuration phpunit.xml

Ive changed all the paths in the TestHelper but I'm getting errors telling me it cannot find the relevant libraries which tells me Autoload is failing. My dir structure is a little morer complex than your example

app/
models
modules/
default/
controllers/
intranet/
controllers/

Can you explain to me when the TestHelper gets called and maybe what I am doing wrong?
#7.2 Chris Lock on 2008-11-04 11:47 (Reply)
Stick a require_once in your actual test class files pointing to the location of TestHelper.php.
#7.2.1 Matthew Weier O'Phinney (Link) on 2008-11-04 13:08 (Reply)
Thanks Matthew, I figured this out after some much needed sleep, thanks for the reply nonetheless. All I have to do now is figure out why it's not bootstrapping ;-)
#7.2.1.1 Chris Lock on 2008-11-05 04:49 (Reply)
Thanks for the great starter article. Could you post an example of your test pages? Possibly IndexControllerTest.php ?
#8 Jesse on 2008-11-06 14:47 (Reply)
Actually, my pastebin demo uses Zend_Test, and its sources are on github: http:://github.com/weierophinney/pastebin
#8.1 Matthew Weier O'Phinney (Link) on 2008-11-06 15:38 (Reply)
Can you give me some insight on how to debug the Invalid controller specified error? I'm getting the error as my first test runs and tries to dispatch to the default index controller.

The application works fine through the web, but my command line execution is failing with this Invalid controller specified error.

Thanks for any help, I know this is obscure.
#9 Jesse on 2008-11-10 13:50 (Reply)
Can you open this conversation up on one of the ZF mailing lists? That way you can paste the test case you're attempting to use, and perhaps also get help from others besides myself.
#9.1 Matthew Weier O'Phinney (Link) on 2008-11-16 12:46 (Reply)
Thanks for the recommendation.
#9.1.1 Jesse on 2008-11-16 23:42 (Reply)

Add Comment

Standard emoticons like :-) and ;-) are converted to images.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

 
 
  • Home
  • Resume
  • Blog
  • Phly PEAR Channel
  • Contact Me
  • About this site

ZCE

Zend Education Advisory Board Member

Add to Technorati Favorites

Calendar

Back November '08 Forward
Mon Tue Wed Thu Fri Sat Sun
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

Quicksearch

Links

  • PHLY - PHp LibrarY
  • Paul M. Jones
  • Mike Naberezny
  • Shahar Evron
  • Planet PHP
  • Zend Where I now work
  • Garden.org Where I once worked

Archives

November 2008
October 2008
September 2008
Recent...
Older...

Categories

XML Linux
XML Personal
XML Aikido
XML Family
XML Programming
XML Dojo
XML Perl
XML PHP

All categories

Syndicate This Blog

XML RSS 0.91 feed
XML RSS 1.0 feed
XML RSS 2.0 feed
ATOM/XML ATOM 0.3 feed
ATOM/XML ATOM 1.0 feed
XML RSS 2.0 Comments

Show tagged entries

xml best practices
xml books
xml conferences
xml dojo
xml dpc08
xml file_fortune
xml linux
xml mvc
xml oop
xml pear
xml personal
xml php
xml phpworks08
xml programming
xml ubuntu
xml vim
xml webinar
xml zendcon
xml zendcon08
xml zend framework
© 2004 - present, Matthew Weier O'Phinney
matthew-web <at> weierophinney.net