Disclaimer

The content of this blog is my personal opinion only. Although I am an employee - currently of Imagination Technologies's MIPS group, in the past of other companies such as Intellectual Ventures, Intel, AMD, Motorola, and Gould - I reveal this only so that the reader may account for any possible bias I may have towards my employer's products. The statements I make here in no way represent my employer's position, nor am I authorized to speak on behalf of my employer. In fact, this posting may not even represent my personal opinion, since occasionally I play devil's advocate.

See http://docs.google.com/View?id=dcxddbtr_23cg5thdfj for photo credits.

Tuesday, December 22, 2015

Perl one-liner to run all tests in a directory tree

This is a Perl one-liner to run all tests in a directory tree:
perl -e 'use File::Find; finddepth(\&wanted,"."); sub wanted { if( $_ eq "Makefile" ) { system("echo $File::Find::name; make -k test"); }} '
OK, I admit it is a rather bloated one liner.

Quick and Dirty Test Suite

I remain an advocate of a quick and dirty test suite structure that is assembled in a directory tree, with tests in directories marked by a Makefile (with a test target), or a shell script, or...

This is an easy way of integrating tests that come from different test suites, with different assumptions.  It is easier than having to write an adapter from your-favorite-Perl-Test::Unit-test-suite to your favorite Javascript-test-suite, and vice versa, ad infinitum.   Actually, the directory structure, and the convention that there is a Makefile with a test target is an adapter - but a fairly low level, universal, adapter.

I usually accompany something like this one-liner with a script that greps for common indications of test failure, such as the string 'TEST FAILED'.  I also like positive indications of success, such as 'TEST PASSED'.  I like being able to create a green bar widget.

I am not a big fan of Makefiles as a scripting solution - but they are fairly ubiquitous.  A Makefile with "make test" is more standard than a shell script called something like 'run-test.sh'.

The above one-liner does not make any provision for setup - environment variables, etc.  I assume that is either done before invocation, or is encapsulated by "make test".

CON: stopping recursive traversal

The above one-liner does not make allowance for Makefiles where 'make test' knows how to recurse into subdirectories.  Indeed, the above one-liner is how some of my quick and dirty test suites recurse over submodule tests.  Such recursive Makefiles used in conjunction with the above one-liner may result in tests being run more than once.

The above one-liner lacks an important feature: stopping.  Or, rather, it has the bad feature that it walks the entire tree - it does not have the ability to stop when certain conditions are met, such as a directory named 'Foo.do-not-follow'.

It is important to have the stopping feature, because it is quite likely that some foreign, imported, test suite also uses the same convention, of Makefiles with 'Make test' targets.  But may require some setup.   In such a case, it may be desirable to have:
  • .../ImportedTestSuite-adapter
    • Makefile - a makefile suitable for my quick and dirty test suite, with 'make test' target.  Sets up for the stuff below
    • ImportedTestSuite
      • the foreign, imported, test suite, which should NOT be run directly by my quick and dirty test driver; ideally, its subtree should NOT be walked
      • Makefile - a makefile for the foreign, imported, test suite, which should NOT be run by my quick and dirty test suite
I sometimes implement stopping with a naming convention, like
  • .../ImportedTestSuite-ADAPTER
    • run 'make test' in this directory, but do not traverse further
  • .../ImportedTestSuite-adapter/Makefile.MY-TEST-SUITE
    • look for this specially named Makefile, but do not traverse further
  • .../ImportedTestSuite-adapter/ImportedTestSuite.SPECIAL-NAME
    • do not run 'make test', do not traverse further
    • CON: I like keep a test suites own naming scheme
  • .../ImportedTestSuite-adapter/IMPORT/test-suite's-own-name
    • PRO: allows using a foreign test suite's own name for itself
    • COM: the import directory is often almost empty, containing only a single subdirectory.  (Although this may change if it is one of a family of test suites, or if you keep multiple versions.)
Naming conventions are easy, but applying them to a directory can produce very long and clumsy pathnames, and sparse subdirectories.  The pathname length can be a problem on OSes with relatively small pathname length limits, like Windows or POSIX.

Using special files at places in the directory, like 
  • .../ImportedTestSuite-adapter/Makefile.MY-TEST-SUITE
  • .../ImportedTestSuite-adapter/MY-TEST-SUITE.CONF
    • with an embedded directive
places less pressure on pathname length.   But it requires controlling the traversal.  Whereas aa naming convention inside a directory name is trivial to filter out.




Friday, December 11, 2015

I dislike Perl's lib/Module.pm AND lib/Module/SubModule.pm

I dislike how Perl requires modules, and the submodules which are really "internal" to them, to live in two places in the filesystem - even if those places are right next to each other.

E.g. the module's top level code in some lib/Module.pm

And the code for any submodules that are really part of the module in the directory tree under lib/Module, e.g. lib/Module/SubModule.pm

So you cannot simply operate on one filesusem object for the module: e.g.g you cannot do:

mv lib/Module some-other-lib-path/Module

Instead, you must do
mv lib/Module.pm some-other-lib-path/Module.pm
mv lib/Module.pm some-other-lib-path/Module.pm 
A minor difference, two commands rather than one, something that can odten be patched over by regexps.

But an important difference: I would go so far as to say that the representation of Perl modules (classes) in the filesystem is not object oriented.  I would say that one of the key characteristics of an object, in this class treating the code for a class as an object, is that it behaves like it is a single object, unless explicitly opened up to look inside.

---+ Bad Influence on Perl CPAN class structure

I think this decision, to have lib/Module.pm and lib/Module/SubModule.pm, has had a bad, or at least confusing, influence on CPAN's module structure.

Sometimes a CPAN module Foo::Bar (lib/Foo/Bar.pm) is actually a module completely unrelated[*] to module Foo (lib/Foo.pm). More confusing if module Foo actually has some internal modules Foo::Fu (lib/Foo/Fu.pm).

(Note *: OK, not "completely unrelated".  How about "unrelated wrt code structure", or "related only by topic, but not actual code".)

Then there is no localization of Foo in the filesystem.   Some parts of lib/Foo are part of module Foo, and some are not.   And not everthing in module Foo is under lib/Foo.

I.e. the Perl CPAN filesystem structure sometimes reflects module structure, and sometimes it just reflects theme.

---+ Kluges

For this reason, I often make my modules appear as Foo::Foo, i.e. lib/Foo/Foo.pm.

But this can become tiresome.  Tiresome. Repetitious.  And it does not prevent somebody else from defining Foo::Bar, and wanting to live in the same directory tree (if not in separate PATH elements).

So I might try Topic::Long_Module_Name_Unlikely_To_Conflict::Short_Module_Name

i.e.
      lib
           /TopicLong_Module_Name_Unlikely_To_Conflict
               /Short_Module_Name.pm
               /Short_Module_Name/Sub_Modules...

e.g.  lib/Foo/AG_Foobar/Foo.pm

Not ideal.

Similarly, I might use a level of indirection to group internal submodules

Foo::internal::Submodules
vs
Foo::Unrelated_Submodules

Again, not ideal.    

---+ What is really needed

Modules should correspond to subtrees of the filesystem.

e.g. lib/Foo.pm, if no furter structure.

lib/Foo.pmdir/Foo.pm  if submodules such as lib/Foo.pmdir/Sub.pm

with it being an error to have both lib/Foo.pm and lib/Foo.pmdir exist.

Or, if you will, require the indirection even if no further structure: lib/Foo.pmdir/Foo.pm, and never lib/Foo.pm

The main file might be called lib/Foo/main.pm.  But I rather like lib/Foo/Foo.pm, or lib/Foo.pmdir/Foo.pm





---+ References

use - perldoc.perl.org: <>

require - perldoc.perl.org: <The require function will actually look for the "Foo/Bar.pm" file in the directories specified in the @INC array.>>



'via Blog this'

Finally got emacs compilation-error-regexp working for perl CPAN Test::Unit::TestCase

Summary: Finally got emacs compilation-error-regexp working for perl CPAN Test::Unit::TestCase



Not only did I need to add a regexp,
but I also needed to disable a compilation-mode-font-lock-keywords pattern
- apparently there may be some phase ordering.

This woukd be more reliably if there were the ability to return ALL possible substrings matching a regexp,
as opposed to being maximally or minimally greedy.

    ;; adding patterns to compilation-error-regexp-alist and/or compilation-error-regexp-alist
    ;; to try to get a perl CPAN Test::Unit error message, like

    ;; There were 3 failures:
    ;; 1) TestCases_for_FrameMaker_MIF_AG.pm:875 - test_another_error(TestCases_for_AG_FrameMaker_MIF)
    ;; expected '', got 'deliberate mismatch'
    ;;
    ;; 2) TestCases_for_FrameMaker_MIF_AG.pm:879 - test_yet_another_error(TestCases_for_AG_FrameMaker_MIF)
    ;; expected '', got 'deliberate mismatch'
    ;;
    ;; 3) TestCases_for_FrameMaker_MIF_AG.pm:898 - test_special_stuff__format_tuple_to_MIF_string__Cell_context__WIP(TestCases_for_AG_FrameMaker_MIF)

    (require 'compile)

    (add-to-list 'compilation-error-regexp-alist
'("\nThere were [0-9]+ failures:\n1) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_"
  1 2)
    (add-to-list 'compilation-error-regexp-alist
'("\n[0-9]+) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_"
  1 2)
      )
    (if nil  ;; the regexp below causes compilation-error-regexp-alist to stop working completely - no matching
      (add-to-list 'compilation-error-regexp-alist
'("\nThere were [0-9]+ failures:\n1) .*\\(\\w\\|\\W)*\n[0-9]+) .*\\(test_\\)"
  2)
)
      )


Diffs:


    === modified file '.emacs'
    *** .emacs 2015-12-12 00:18:53 +0000
    --- .emacs 2015-12-12 00:26:30 +0000
    ***************
    *** 4129,4139 ****
     ;; perl CPAN Test::Unit
     ;; 1) TestCases_for_FrameMaker_MIF_AG.pm:763 - test_special_stuff_to_insert_wip(TestCases_for_AG_FrameMaker_MIF)
     '(("\\([a-zA-Z0-9_.-.---]+:[0-9]+\\)"
    !      ;;(0 'compilation-line-face nil t)
    !      ;; (0 'ag-test-result-unexpected) ;; works (modulo filename matching)
    !      ;;(1 'ag-test-result-unexpected) ;; works (modulo filename regexp
    !      (1 compilation-line-face nil t)
    !      ))
     '(("expected\s+.*got\s+.*"
 (0 'ag-test-result-unexpected)
 ))
    --- 4129,4139 ----
     ;; perl CPAN Test::Unit
     ;; 1) TestCases_for_FrameMaker_MIF_AG.pm:763 - test_special_stuff_to_insert_wip(TestCases_for_AG_FrameMaker_MIF)
     '(("\\([a-zA-Z0-9_.-.---]+:[0-9]+\\)"
    ! ;;      ;;(0 'compilation-line-face nil t)
    ! ;;      ;; (0 'ag-test-result-unexpected) ;; works (modulo filename matching)
    ! ;;      ;;(1 'ag-test-result-unexpected) ;; works (modulo filename regexp
    ! ;;      (1 compilation-line-face nil t)
    ! ;;      ))
     '(("expected\s+.*got\s+.*"
 (0 'ag-test-result-unexpected)
 ))
    ***************
    *** 4628,4637 ****
      (add-to-list 'compilation-error-regexp-alist
 '("\nThere were [0-9]+ failures:\n1) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_"
    1 2)
)
    ! (if nil
(add-to-list 'compilation-error-regexp-alist
    !     '("\nThere were [0-9]+ failures:\n1) .*\\(\\w\\|\\W)*\n[0-9]+) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_) "
    !        2 3)
 )
)
    --- 4628,4640 ----
      (add-to-list 'compilation-error-regexp-alist
 '("\nThere were [0-9]+ failures:\n1) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_"
    1 2)
    + (add-to-list 'compilation-error-regexp-alist
    +     '("\n[0-9]+) \\([a-zA-Z0-9_.-.---]+\\):\\([0-9]+\\) - test_"
    +        1 2)
)
    ! (if nil  ;; the regexp below causes compilation-error-regexp-alist to stop working completely - no matching
(add-to-list 'compilation-error-regexp-alist
    !     '("\nThere were [0-9]+ failures:\n1) .*\\(\\w\\|\\W)*\n[0-9]+) .*\\(test_\\)"
    !        2)
 )
)



Friday, December 04, 2015

p4 status - misleading Perforce command name

So, I want to run a command in a script that asks Perforce if a file exists in the depot.



I wonder what such a command might be callewd. Let's try "p4 status".



Strange... google "p4 status".



!!!! "p4 status"  `Open files for add, delete, and/or edit in order to reconcile a workspace with changes made outside of Perforce.1





This is stupid.  In English, in 99.99% of programming systems, "status" is a read-only query.


Wednesday, December 02, 2015

.ignore files for version control systems

It is good to have .ignore files at the root of a project's repo. (bzr, hg, git, but not Perforce)

It is good to have global ignores - as the bzr manuals say "ignored files which are not project specific, but more user specific. Rather than add these ignores to every project, bzr supports a global ignore file ~/.bazaar/ignore

In addition to "per-user global", it is also good to have "global across an installation". Git has this, but not bzr.

How to Get a Refund For an iPhone, iPad, or Mac App From Apple

How to Get a Refund For an iPhone, iPad, or Mac App From Apple:



'via Blog this'



Many iPhone apps do not work as expected.



But getting refunds for them is more painful than on Android.



Another reason to prefer Android - if they can stop the security bleeding.