The content of this blog is my personal opinion only. Although I am an employee - currently of Nvidia, in the past of other companies such as Iagination Technologies, MIPS, Intellectual Ventures, Intel, AMD, Motorola, and Gould - I reveal this only so that the reader may account for any possible bias I may have towards my employer's products. The statements I make here in no way represent my employer's position, nor am I authorized to speak on behalf of my employer. In fact, this posting may not even represent my personal opinion, since occasionally I play devil's advocate.

See http://docs.google.com/View?id=dcxddbtr_23cg5thdfj for photo credits.

Friday, August 19, 2016

Beyond Getopt: Doc + Test > CodeGen

For decades I have been automating generating code for boilerplate like command line parsing (getopt) and generating documentation for that code.  E.g. circa 1985 in my first job I wrote tools to generate manpages from what looks very much like Perl POD (Plain Old Documentation).

I admit to joining the "test-first" and "test-driven" bandwagon later. Grudgingly at first - Bob Colwell at Intel/P6 said "if it doesn't have an automated test, it doesn't exist".   With growing enthusiasm as I discovered Agile and refactoring circa 1996.

Yet for all of my love of automation (i.e. my laziness), I was never quite happy with getopt style functions, whether in C, C++, Perl, etc.  Often the standard libraries did not parse options the way I want:  E.g. typically I want options to be parsed strictly left to right, so that later options can override earlier options, so that you can create configurations and then customize. Except, e.g., when I want to disallow. E.g. often I want to have expression parsing as part of the options, so that you can say CPUSIM -cachesize=linesize*sets*ways -sets=4 ...  E.g. I want to be able to have the same or related options parsers for command line, environment variables, and config files.   E.g. I want modular, composable, options parsing - at Intel we called this "knobs" parsing, to the merriment of pommies - so that I can plug a cache model into the L2 position, and instantly get all of the appropriate options for that new L2 cache.  And so on.

Plus, of course, I always want to automate the creation of docs and help messages, so that they are guaranteed to be kept in sync wth the parsing code.

Many getopts libraries only automate the code generation.

Some provide help and documentation - usually perfunctory.   E.g. when you create the option, you may also provide a string saying what the option is for.


Today I tried something a bit different: I unified the documentation and the tests for command line arguments.    The simple examples were tests, automated of course.   More complicated tests were interleaved with the simple examples. Much more complicated outlined.

Rather like a literate programming approach.   Later, I googled refs such as that attached below.

I am quite pleased with this approach.  


In an ideal world one might automate all three: getopt codegen, help docs, and tests.  

Although one then worries about common mode errors and single points of failure or untestedness.

But it turns out that it is not that hard adding the getopt code.  Especially if too complicated, to many validation checks for regular getopt.   It is the creation of the documentation and the tests that is a pain.  If the tests are automated, the codegen is straightforward.


See related:

Getopt::Euclid - search.cpan.org:

Combining Literate Programming and Functional Unit Tests to Augment the Workflow and Documentation in Software Development Projects - DBIS EPub: "Combining Literate Programming and Functional Unit Tests"