perl -e 'use File::Find; finddepth(\&wanted,"."); sub wanted { if( $_ eq "Makefile" ) { system("echo $File::Find::name; make -k test"); }} '
OK, I admit it is a rather bloated one liner.
Quick and Dirty Test Suite
I remain an advocate of a quick and dirty test suite structure that is assembled in a directory tree, with tests in directories marked by a Makefile (with a test target), or a shell script, or...
This is an easy way of integrating tests that come from different test suites, with different assumptions. It is easier than having to write an adapter from your-favorite-Perl-Test::Unit-test-suite to your favorite Javascript-test-suite, and vice versa, ad infinitum. Actually, the directory structure, and the convention that there is a Makefile with a test target is an adapter - but a fairly low level, universal, adapter.
I usually accompany something like this one-liner with a script that greps for common indications of test failure, such as the string 'TEST FAILED'. I also like positive indications of success, such as 'TEST PASSED'. I like being able to create a green bar widget.
I usually accompany something like this one-liner with a script that greps for common indications of test failure, such as the string 'TEST FAILED'. I also like positive indications of success, such as 'TEST PASSED'. I like being able to create a green bar widget.
I am not a big fan of Makefiles as a scripting solution - but they are fairly ubiquitous. A Makefile with "make test" is more standard than a shell script called something like 'run-test.sh'.
The above one-liner does not make any provision for setup - environment variables, etc. I assume that is either done before invocation, or is encapsulated by "make test".
CON: stopping recursive traversal
The above one-liner does not make allowance for Makefiles where 'make test' knows how to recurse into subdirectories. Indeed, the above one-liner is how some of my quick and dirty test suites recurse over submodule tests. Such recursive Makefiles used in conjunction with the above one-liner may result in tests being run more than once.
The above one-liner lacks an important feature: stopping. Or, rather, it has the bad feature that it walks the entire tree - it does not have the ability to stop when certain conditions are met, such as a directory named 'Foo.do-not-follow'.
It is important to have the stopping feature, because it is quite likely that some foreign, imported, test suite also uses the same convention, of Makefiles with 'Make test' targets. But may require some setup. In such a case, it may be desirable to have:
- .../ImportedTestSuite-adapter
- Makefile - a makefile suitable for my quick and dirty test suite, with 'make test' target. Sets up for the stuff below
- ImportedTestSuite
- the foreign, imported, test suite, which should NOT be run directly by my quick and dirty test driver; ideally, its subtree should NOT be walked
- Makefile - a makefile for the foreign, imported, test suite, which should NOT be run by my quick and dirty test suite
I sometimes implement stopping with a naming convention, like
- .../ImportedTestSuite-ADAPTER
- run 'make test' in this directory, but do not traverse further
- .../ImportedTestSuite-adapter/Makefile.MY-TEST-SUITE
- look for this specially named Makefile, but do not traverse further
- .../ImportedTestSuite-adapter/ImportedTestSuite.SPECIAL-NAME
- do not run 'make test', do not traverse further
- CON: I like keep a test suites own naming scheme
- .../ImportedTestSuite-adapter/IMPORT/test-suite's-own-name
- PRO: allows using a foreign test suite's own name for itself
- COM: the import directory is often almost empty, containing only a single subdirectory. (Although this may change if it is one of a family of test suites, or if you keep multiple versions.)
Naming conventions are easy, but applying them to a directory can produce very long and clumsy pathnames, and sparse subdirectories. The pathname length can be a problem on OSes with relatively small pathname length limits, like Windows or POSIX.
Using special files at places in the directory, like
- .../ImportedTestSuite-adapter/Makefile.MY-TEST-SUITE
- .../ImportedTestSuite-adapter/MY-TEST-SUITE.CONF
- with an embedded directive
places less pressure on pathname length. But it requires controlling the traversal. Whereas aa naming convention inside a directory name is trivial to filter out.
