Disclaimer

The content of this blog is my personal opinion only. Although I am an employee - currently of Nvidia, in the past of other companies such as Iagination Technologies, MIPS, Intellectual Ventures, Intel, AMD, Motorola, and Gould - I reveal this only so that the reader may account for any possible bias I may have towards my employer's products. The statements I make here in no way represent my employer's position, nor am I authorized to speak on behalf of my employer. In fact, this posting may not even represent my personal opinion, since occasionally I play devil's advocate.

See http://docs.google.com/View?id=dcxddbtr_23cg5thdfj for photo credits.

Friday, July 10, 2009

Yahoo Geocities closing - Don't Trust Free Websites

Yahoo Geocities is closing down.

For several years this was the home of my resume, and other stuff: http://www.geocities.com/andrew_f_glew.

Yahoo gives you the option of paying 5$/month for webhosting, or saving your files. Yahoo requires you to save the files one at a time, clicking Save As on a menu. There is no ability to download all of your files as a tarball or other archive. There is no WebDav access, or other filesystem-like access. I.e. there is no easy path to automatically transferring a large website.

Fortunately, I did not have many files on Yahoo.

Even more fortunately, I abandoned Yahoo Geocities a few months ago, and moved all of my stuff to Google Docs. http://docs.google.com/Doc?id=dcxddbtr_6dvpxg2cj&hl=en

However, there is no guarantee that Google Docs will remain free, or even available, forever.

Google Docs doesn't appear to have an easy way of downloading all files. Who was it that provided such a facility? One of the wiki sites that closed down?

Coincidentally, I have been shopping for a hopefully more permanent home for me and my files on the web. I've been looking at Amazon S3 and ECC. Yes, I am willing to pay.

Amazon ECC is good for virtual hosting. However, while I like being root, I would also like an ordinary user account on some UNIX system. I would like somebody else to be responsible for keeping up to date on security patches. Ideally I would like to run a wiki on such an account, but I want the wiki to run, not even as me, but as a Mini-Me, with even less privilege.

I guess I want it all: the convenience of having someone else sysadmin, but with the ability to run a small number of personal web services in deprivileged accounts.

For that matter, I'd like the hosting system to run a non-standard OS like FreeBSD, and a non-standard instruction set. ARM? Power? Is it security through obscurity?

As for my free websites:

* I learned that I still hasd a Yahoo mail account from long ago. Once again, no bulk download, unless I upgrade to pay, at which point I get POP.

* Google Docs - no automated access

* Google Mail - I have IMAP and POP access. I think I better start backing it up better.

Backing it up to S3? Wherever.

Just like there is a period of recent history that was lost, because information stopped being recorded on paper and started being recorded on quickly obsolete digital formats such as tape, floppies, etc., now we are traversing the stage where history is lost because of service providers closing down. One can only hope that we will come out the other end with storage as a commodity, with standard procedures for migration.

Sunday, July 05, 2009

Gilding the Lily

By the way, I recognize that in the preceding posts I risk the sort of gilding the lily that so many software teams do when they define many, many, different comment keywords.


    E.g. http://www.cs.clemson.edu/~mark/330/colwell/p6-coding-standards.html.
    This is relatively small - only 4 primary keywords, with 3 secondary.
    Other example standardsare much larger.

    TBD
    - This "flag" is used to indicate items that require later definition. It stands for To Be Defined (or Determined). The ensuing comment should provide more particulars.
    NYI
    - This "flag" is used to indicate items that have been defined and are now awaiting implementation. It stands for Not Yet Implemented. The ensuing comment should provide more particulars.
    MACHDEP
    - This "flag" is used to indicate the existence of an explicit machine dependency in the code. Again, the ensuing comment should provide more particulars.
    BUG:
    - This "flag" is used to indicate the existence of a bug of some form. It should be followed immediately in the comment (on the same line) with one of the keywords incomplete, untested, or wrong, to indicate its type along with a more descriptive comment if appropriate. If none of the keywords is applicable, some other descriptive word should be used along with a more extensive comment.



The most comonly used such keyword is probably
  • TBD or TODO
and other common keywords include
  • NYI: not yet implemented
  • BUG:
  • NOTE:
But frequency of use rapidly goes down. E,g. although I approve of the concept of MACHDEP, I hardly ever used it during the P6 project, and I had totally forgotten about it.


Comments such as these work best when automated.

E.g., although I hate doxygen, doxygen comments get automatically extracted, and are hence more likely to be seen and corrected.

E.g. ditto Perldoc. Again, I find perldoc makes code harder to read.

Preparing this blog, I found
http://www.icsharpcode.net/TechNotes/Commenting20020413.pdf
"The Fine Art of Commenting",
Bernhard Spuida, 2002.

Spuida mentions several such systems, including Microsoft's C# .NET system,
where XML tags are embedded in comments.
As indicated by Spuida's title "Senior Word Wrangler",
this motivation seems to be mainly about preparing documentation.

It was fun to re-find the Indian Hill Coding Standards
http://www.psgd.org/paul/docs/cstyle/cstyle.htm

Every few years I seem to go through a spate of reading of coding standards.
I think every programmer should be familiar with several, many, different coding standards.
So that he or she can pick parts that work well,
for projects that need those parts.

Final words:
Brian Marick's lynx.
Hypertext embedded within the code itself.
My own hope of wiki files in the source trees,
and/or linked to from the code itself.
Programming languages using extensible notations, such as XML,
allowing arbitrary structured annotations.

Test passed/failed

More wrt test monitoring.

I concluded the last post with (slightly extended):
TEST RUN: test1
TEST START: test2
TEST CHECK OKAY: test2 check1
TEST STARTED: test2.1
TEST PASSED: test2.1
TEST FINISHED: test2.1
TEST END: test2
Implying 2 top level tests, test1 and test2. Test1 is a "monad", reported by TEST RUN ooutside of START/END. Test2 is bracketed by START/END, and contains subtest 2.1.

When I started testing seriously, I thought that all tests could be classified passed/failed. That is always a worthwhile goal. If it could be accomplished automatically, it might suggest what we might express in pseudo-XML as:

<test name="test1" result="passed"/>
<test name="test2">
<test-check result="ok" test_name="test1" check_name="check1"/>
...
</test name="test2" result="passed">

My pseudo-XML allows attributes on the close. Without this, one might just expeft a TEST PASSED message immediately before the close.

However, over the years I have learned that things are not always so clear cut. While it is always best to write completely automated tests that clearly pass or fail ...

Sometimes you write tests and just run them, but do not automatically determine pass or fail.

Sometimes manual inspection of the output is required.

Sometimes you just want to say that you have run the test, but you have not yet automated the checking of the results... and sometimes, given real-world schedule pressure, you never get around to automating the checking. In such cases, IMHO it is better to say


TEST STARTED: foo
TEST TBD: have not yet automated results checking yet
TEST ENDED: foo


than it would be to just omit the test.

Oftentimes, the fact that a test has compiled and run tells you something. Or, rather: if the test fails to compile or run it tells you that you definitely have a problem.

Sometimes you can automate part of a test, but needmanual inspection for other parts. In this case, I think reporting "TEST PASSED" is dangerously misleading:


TEST STARTED: foo
TEST PASSED
TEST ENDED: foo


or, better


TEST STARTED: foo
TEST PASSED
TEST TBD: foo: need manual inspection of rest of test output
TEST ENDED: foo


I think that "TEST PASSED" tends to imply that the entire test has passed. If you say "TEST PASSED" without a label test name, it tends to imply that the enclosing test has passed.

Better to say


TEST STARTED: foo
TEST PASSED: sub-test bar of test foo
TEST TBD: foo: need manual inspection of rest of test output
TEST ENDED: foo


I have recently started using other phrases, such as "TEST CHECK"


TEST STARTED: foo
TEST CHECK OKAY: foo check1
TEST PASSED: sub-test bar of test foo
TEST TBD: foo: need manual inspection of rest of test output
TEST ENDED: foo


Q: what is the difference between a TEST PASSED: subtest and a TEST CHECK OKAY (or TEST CHECK PASSED)? Not much: mainly, the name tends to imply something about importance. Saying that a test or subtest passed seems to imply that something freestanding has passed. A check within a test seems naturally less important.

This is along the lines of assertions. Some XUnit tests count all assertions passed. While this can be useful - particularly if some edit accidentally removes thousands of assertions - I myself have found that the number of assertions gives a false measure of test effort.

It may be that I am conflating "test" with "test scenario". A "test scenario" or "test case" may be subject to thousands of assertions. Particularly if the asserts are in the infrastructure. But I really want to count test cases and scenarios.

Here's one reason why I try to distinguish #tests passed from #checks performed:
  • my test monitor performs consistency checks such as tests_passed = test_cases, tests_started = tests_ended, etc.
What I really want is things like
  • Number of tests that had positive indication of complete success - tests passed. (Or, at least, success as complete as any test can indicate.)
  • Number of tests that had postive indication of a failure or error.
  • Similarly, warnings.
  • Number of tests that had no positive indication - a monad "TEST RUN" message was sen, or perhaps a TEST START/END pair, but no positive indication.
  • Number of tests where failure can be inferred - e.g. TEST START without a corresponding test end.

Test monitoring keywords

Playing with my text-test-summarizer-monitor, a little Perl/Tcl script that looks at the output of tests, filtering things likes tests started/ended, passed/failed, looking for various common indications of problems. Throws up a Tcl widget. The closest thing I have to a green bar.

Here's an annoyance: my test may look like

TEST END: test1
TEST START: test2
TEST CHECK OKAY: test2 check1
TEST END: test2


Oftentimes I like to announce "TEST STARTED" and "TEST ENDED". (I just had to extend my scripts to handle both START and STARTED. This is useful in case the test crashes in the middle, and you never get to the test end.

However, occasionally the test infrastructure does not allow this. Occasionally I just say, once, at the end "I ran this test, and it passed". That's what I mean above by the "TEST END: test1" without the corresponding TEST START.

In XML, this would be simple:


<test name="test1"/>
<test name="test2">
<test-check result="ok" test_name="test1" check_name="check1"/>
</test name="test2">


Note:
  1. I added an attribute to the closing, </test name="test2">. Although not part of standard XML, occasionally this has helped me. I call this, therefore, pseudo-XML
  2. Note that test is available in both and ... form
Note that this is pretty verbose. Although human readable, I would not call it human friendly.
Because the pseudo-XML is not so human friendly, I often prefer to print messages such as


TEST END: test1
TEST START: test2
TEST CHECK OKAY: test2 check1
TEST END: test2


But here I run into terminology: I don't have a natural way of having a message, that is not confused with START/END.

First: I am reasonably happy with brackets {BEGIN,START}/{END,FINISH},
and {STARTED}/{ENDED,FINISHED}. English grammar, how inconsistent.
I want to tolerate all of these, since I have used all from time to time,
and regularly find a test suite falsely broken if I assume that I have consistently used START and not STARTED.

(I'm going to arbitrarily reject TEST COMPLETED. Too verbose. Especially the dual TEST INITIATED. At least, I'll eject until it seems I encounter it a lot.)

But a TEST END without a TEST START is too confusing. I need an English phrase that doesn't need a corresponding start. Let's try a few:


  • TEST PASSED: test name.
      with the corresponding
    • TEST FAILED: test name

    • However, there might be some confusion because I definitely want to use TEST PASSED/FAILED withing TEST START/END. See below.

  • TEST RESULT: test name
      Similarly, there might be some confusion because I might want to use TEST RESULT within TEST START/END.

  • TEST RUN: test name
      Nice because of the possible dual TEST NOT RUN



I think that I am arriving at the conclusion that any of the above, outside a TEST START/END, make sense, and should be considered equivalent to &lttest .../>

I am not currently checking for the proper nesting of tests, but I could be.

I think that it would be good to have a count of top level tests, either TEST START/END brackets or TEST outside such brackets, but ignoring stuff within the brackets.


Giving me

TEST RUN: test1
TEST START: test2
TEST CHECK OKAY: test2 check1
TEST END: test2

Thursday, July 02, 2009

Hyper Fan Mode in the Morning

Oh, how I have come to dread the sound of fans on my laptop in the morning!:

Walk to my office. Dock my HP 8530W laptop. Power on.

And hear the dread sound of fans going full bore. Laptop hung.

I dread this sound, since I know that it will take 10-15 minutes to get o a place where I can start work.

It is curious that both my much maligned Toshiba M400 tablet, and my employer provided HP 8530W laptop, have the same "ultra-fan" mode symptom of death. Since they have different OSes - Vista in the former, XP in the latter - it suggests that the problem is not the OS per se, but some common component. Since they have different manufacturers, it suggests that it is not manufacturer specific. Probably some common software, such as a Microsoft power management utility common to both OS versions.

Wednesday, July 01, 2009

"Moded" Keys in FVWM

I am highly encouraged to use the FVWM2 X Window Manager at work on Linux.
Not exactly forced, but other options are a hassle.

Finally got around to providing some key bindings.

In particular, this hack to enter a "mode" where a window's size is changed by repeated key presses
(1) define a menu with hotkeys calling a function
(2) have the function perform the action (e.g. narrowing the window), and then re-invoke the menu with the hotkeys.

I haven't seen it described in the FVWM2 FAQ, although I got the idea from a FAQ suggestion wrt using hotkeys.



########################################################################################

## Andy Glew stuff
AddToMenu MainMenu
+ "Glew" Popup Glew
AddToMenu Window_Ops
+ "Glew" Popup Glew
+ "Ident" Module FvmwIdent

Key F1 A SC Menu Glew
Mouse 1 A SC Menu Glew



#----------------------------------------------------------------------------------------------------


AddToMenu Glew "Glew" Title
+ "&RaiseLower" RaiseLower
+ "Resi&ze" Resize
+ "&PlaceAgain" PlaceAgain
+ "" Nop
+ "Maximize (toggle)" Maximize
+ "Maximize" Maximize true
+ "&UnMaximize" Maximize false
+ "Maximize 1850px1100p" Maximize true 1850p 1100p
+ "Resize 1850p 1100p" Resize 1850p 1100p
+ "" Nop
+ "MainMenu" Menu MainMenu
+ "Window_Ops" Menu Window_Ops
+ "Glew_Window_Sizes" Menu Glew_Window_Sizes
+ "&Keyboard commands" Menu Glew_Window_Manager_Keyboard_Commands_Menu
+ " Glew_Window_Sizing_From_Keyboard" Menu Glew_Window_Sizing_From_Keyboard
+ " Glew_Window_Moving_From_Keyboard_Menu" Menu Glew_Window_Moving_From_Keyboard_Menu
+ "" Nop
+ "FvwmConsole" Module FvwmConsole -fg black -bg green3
+ "Restart Fvwm2" Restart fvwm2
+ "" Nop
+ "Ident" Module FvwmIdent

AddToMenu Glew_Window_Sizes "Glew_Window_Sizes" Title
+ "1000x660 toshiba 3505" Maximize true 1000p 700p
+ "1000x700 largest compaq evo n115" Maximize true 1000p 700p
+ "1024x768" Maximize true 1024p 768p
+ "1260x956 Largest Thinkpad T42p" Maximize true 2900p 956p
+ "1350x1000 Largest Thinkpad T43p" Maximize true 1350p 1000p
+ "1350x980" Maximize true 1350p 980p
+ "1400x1050" Maximize true 1400p 1050p
+ "1500x1100" Maximize true 1500p 1100p
+ "1550x1100 largest sony grx580" Maximize true 1500p 1100p
+ "1550x1100 largest thinkpad a21p" Maximize true 1500p 1100p
+ "1600x1200" Maximize true 1600p 1200p
+ "1800x1100" Maximize true 1800p 1100p
+ "1850x1100" Maximize true 1850p 1100p
+ "1920x1200" Maximize true 1920p 1200p
+ "2700x1000" Maximize true 2700p 1000p
+ "2900x1120 2 monitors at work" Maximize true 2900p 1120p
+ "2900x950" Maximize true 2900p 950p

########################################################################################

AddToMenu Glew_Window_Manager_Keyboard_Commands_Menu "Glew Window Manager Keyboard Commands Menu" Title
#+ "Glew_Cursor_Moving_From_Keyboard" Menu Glew_Cursor_Moving_From_Keyboard_Menu
+ "Glew_Window_&Sizing_From_Keyboard" Menu Glew_Window_Sizing_From_Keyboard
+ "Glew_Window_&Moving_From_Keyboard_Menu" Menu Glew_Window_Moving_From_Keyboard_Menu


########################################################################################

# Hack to enter a "mode" where a window's size is changed by repeated key presses
# (1) define a menu with hotkeys calling a function
# (2) have the function perform the action (e.g. narrowing the window),
# and then re-invoke the menu with the hotkeys.

# Using the "c" client specific units, since they work well with emacs,
# even though they are NOT exactly characters.
# TBD: make use pixels.

DestroyMenu Glew_Window_Sizing_From_Keyboard
AddToMenu Glew_Window_Sizing_From_Keyboard "Glew_Window_Sizing_From_Keyboard" Titles
+ "&Wider" Function Glew_Window_Sizing_From_Keyboard_Function w+1c w+0c
+ "&Narrower" Function Glew_Window_Sizing_From_Keyboard_Function w-1c w-0c
+ "&Taller" Function Glew_Window_Sizing_From_Keyboard_Function w+0c w+1c
+ "&Shorter" Function Glew_Window_Sizing_From_Keyboard_Function w-0c w-1c

DestroyFunc Glew_Window_Sizing_From_Keyboard_Function
AddToFunc Glew_Window_Sizing_From_Keyboard_Function
+ I Resize $0 $1
+ I Menu Glew_Window_Sizing_From_Keyboard

########################################################################################

# Similar hack to move windows from keyboard

SetEnv Stepsize_for_Glew_Window_Moving_From_Keyboard_Menu 10p

DestroyMenu Glew_Window_Moving_From_Keyboard_Menu
AddToMenu Glew_Window_Moving_From_Keyboard_Menu "Glew_Window_Moving_From_Keyboard_Menu" Titles
+ "&Right" Function Glew_Window_Moving_From_Keyboard_Function w+$[Stepsize_for_Glew_Window_Moving_From_Keyboard] w+0p
+ "&Left" Function Glew_Window_Moving_From_Keyboard_Function w-$[Stepsize_for_Glew_Window_Moving_From_Keyboard] w-0p
+ "&Down" Function Glew_Window_Moving_From_Keyboard_Function w+0p w+$[Stepsize_for_Glew_Window_Moving_From_Keyboard]
+ "&Up" Function Glew_Window_Moving_From_Keyboard_Function w-0p w-$[Stepsize_for_Glew_Window_Moving_From_Keyboard]

DestroyFunc Glew_Window_Moving_From_Keyboard_Function
AddToFunc Glew_Window_Moving_From_Keyboard_Function
+ I Move $0 $1 warp
+ I Menu Glew_Window_Moving_From_Keyboard_Menu

## Vain attempt to make stepsize adjustable.
## TBD: fix
#
#AddToMenu Glew_Window_Moving_From_Keyboard_Menu "Glew_Window_Moving_From_Keyboard_Menu" Titles
#+ "&faster" Faster_Stepsize_for_Glew_Window_Moving_From_Keyboard
#+ "&slower" Slower_Stepsize_for_Glew_Window_Moving_From_Keyboard
#
#DestroyFunc Faster_Stepsize_for_Glew_Window_Moving_From_Keyboard
#AddToFunc Faster_Stepsize_for_Glew_Window_Moving_From_Keyboard
#+ I SetEnv Stepsize_for_Glew_Window_Moving_From_Keyboard 10p
#+ I Menu Glew_Window_Moving_From_Keyboard_Menu
#
#DestroyFunc Slower_Stepsize_for_Glew_Window_Moving_From_Keyboard
#AddToFunc Slower_Stepsize_for_Glew_Window_Moving_From_Keyboard
#+ I SetEnv Stepsize_for_Glew_Window_Moving_From_Keyboard 1p
#+ I Menu Glew_Window_Moving_From_Keyboard_Menu

########################################################################################

#BROKEN: # Similar hack to move cursor from keyboard
#BROKEN: # DOES NOT WORK, since menus affect cursor position
#BROKEN:
#BROKEN: SetEnv Stepsize_for_Glew_Cursor_Moving_From_Keyboard_Menu 10p
#BROKEN: # TBD: arrange to change
#BROKEN:
#BROKEN: DestroyMenu Glew_Cursor_Moving_From_Keyboard_Menu
#BROKEN: AddToMenu Glew_Cursor_Moving_From_Keyboard_Menu "Glew_Cursor_Moving_From_Keyboard_Menu" Titles
#BROKEN: + "&Right" Function Glew_Cursor_Moving_From_Keyboard_Function +$[Stepsize_for_Glew_Cursor_Moving_From_Keyboard] +0p
#BROKEN: + "&Left" Function Glew_Cursor_Moving_From_Keyboard_Function -$[Stepsize_for_Glew_Cursor_Moving_From_Keyboard] -0p
#BROKEN: + "&Down" Function Glew_Cursor_Moving_From_Keyboard_Function +0p +$[Stepsize_for_Glew_Cursor_Moving_From_Keyboard]
#BROKEN: + "&Up" Function Glew_Cursor_Moving_From_Keyboard_Function -0p -$[Stepsize_for_Glew_Cursor_Moving_From_Keyboard]
#BROKEN:
#BROKEN: DestroyFunc Glew_Cursor_Moving_From_Keyboard_Function
#BROKEN: AddToFunc Glew_Cursor_Moving_From_Keyboard_Function
#BROKEN: + I CursorMove $0 $1
#BROKEN: + I Menu Glew_Cursor_Moving_From_Keyboard_Menu

#----------------------------------------------------------------------------------------

# From the fvwm2 FAQ
#7.9 Moving the mouse/focus/page with the keyboard.
#
# Try these key bindings for mouse movement:
#
# # shift- to move a few pixels
# Key Left A S CursorMove -1 0
# Key Right A S CursorMove +1 +0
# Key Up A S CursorMove +0 -1
# Key Down A S CursorMove +0 +1

# Glew version of these bindings

# - I wish I had a modifier reserved for the X/VNC/FVMW2 window manager
# but since I am running Emacs meta is usually taken
# and since I am running under VNC on Windows the Windows key is taken
# (Moral: needone key per layer of Window Manager)

# so, anyway, the closest thing I have to a free modifier is meta-arrow

# shift- to move a few pixels
Key Left A M CursorMove -1 0
Key Right A M CursorMove +1 +0
Key Up A M CursorMove +0 -1
Key Down A M CursorMove +0 +1

Friday, June 26, 2009

git subtree

Mike Haertel forwarded me a link to git subtree.

It goes a long way towards what I want. It doesn't have the intentional branching - what it means for a project to be on the same branch as a subproject library it is using. But it goes a long way, perhaps most of the way, towards what I want.