Version Control Checkout and Sharing of *Partially Populated* Subtrees

I'm on the version control tool warpath again.

I'm a damned good example of why VC tools need subprojects, nested trees, whatever... Never you mind Keith Packard (Mr X Consortium) saying that I shouldn't bother.

At the moment I am messing around with
a) at least 2 diverged CVS trees for my home directory, one on my laptop, one on Linux.
b) a simulator, which uses BitKeeper
c) my own main project, which uses CVS (fortunately, one site only)
d) various other tools at work, using CVS and SVN
e) my TWiki site at work, which uses RCS (although I can make it look like CVS if necessary)
f) friends and coworkers who use GIT

My contributions to b), c), d), and e) make extensive use of stuff from my personal library. But, these other projects do not want to import my whole personal library - rather, they want to import just the modules they need, and their dependences.

Part of my personal source tree is structured as follows:

~/src:

include
" debug.h
" lprintf.h
" MathConst.h
" ag.h
" debug.h
" lfile.h
" libag.h
" simplelock.h
" strfamily.h*
libag
" assert
" bitpattern
" bitvector
" builtin_initialization
" class_name
" code_location
" compile " component
" " SimpleScalar-libag-component
" error
" error_stream
" File_Slurp
" file_slurp
" fmt
" getopt
" getter_setter
" indented_ostream
" Inherit
" interval
" IO_converter
" libuarch
" " cache-array
" " " examples
" " memory_image
" " multilevel_cache
" " SimpleScalar-libag-libuarch
" Memory_Object
" misc
" new-libag-ideas
" " quantiles
" number
" old-stuff-trying-to-reuse
" regd_ptr
" smart_ptr
" stack_trace
" stdio-stream
" test
" thisprintf
" to_bitstring
" to_string
" typename
" types
" unindent
" xml

You get the picture: lots of little modules, each in their own directories or directory trees.

Way back when I started, I put multiple header files in the same directory, like include/debug.h, include/lfile.h. I also created "utility" files like ag.h and libag.h. Over time I learned that it was best to put each module, each header file, in a directory of its own. Such a directory gives you a place to place tests, etc. There is a pattern of repeating the directory name and the include name #include "libag/test/test.h"

Above I have excerpted "header-only" libraries. I also have some C/C+ libraries that need to be compiled, but I have learned that "header only" lubraries, that only need to be #included, are a lot more convenient to use.

Some of my libraries have interdependencies. For example, libag/test/test.hh includes ../code_location/codelocation.hh.
First thing to note: the use of includer relative paths. If I simply copy (cp -R) libag/{test,code_location}, and #include "import/libag/test/test.hh", everything works.

Second, and most important thing to note: a typical user of some (but not all) of my libraries might want a pruned subtree:

import/libag
" test
" code_location

Importing all of libag is just clutter.

I.e. they do not want to import just complete subtrees.

They want to import a pruned subtree - stuff that is in the parent directory, libag, as well as subdirectories such as libag/test and libag/code_location.

I no longer put actual library functions at such parent-root directories. (Alright, I still have some files there, libag/stralloc.h, mainly dating back before I decided to use the "directory per module" pattern.) But I still have metadata there - libag/README.libag, explaining library organization. I also have tools like SConstruct files and files that guide test jig runners, so that I can simply say "make test" or "scons test" in libag, and all the tests run. (My test runner does NOT require a list of all the subdirectories.)
I would like to keep some of that metadata in libag when it is checked out for use by other projects.
I.e. I want all of the file contents of libag, but only a subset of the subdirectories.
I say again: I want a PRUNED SUBTREE of the source.

---

Really, what I want is a version control that makes it easy to share small modules. Not heavyweight subprojects, a la git. Not projects that I have to define in advance, like git or CVSROOT/modules. I want really lightweight modules. Directory subtrees seem to be a very natural way of specifying submodules.
I.e. at the very least I want to be able to do "vcs co libag/test libag/code_location".
Unfortunately, this least thing that I want to do is barely supported by any of the modern version control tools (git, hg, bzr). cvs, oronically, supports it fairly well.

But I want to do better than this least.

Ultimately, I want automatic dependency tracking. I want to be able to say "vcs co libag/test", and have something automatically recognize that I need to checdk out libag/code-location as well. Perl CPAN does it - I want it everywhere.

But I also want to be able to check out libag, with local files, and a pruned set of subdirectories.

---

OK, say I have gotten that far: I can check out just the modules I want.

Next, I want to be able to work in a single workspace - and check things into multiple version control repositories.
E.g. I want to be able to use CVS to check changes made to my libraries back into my personal epositories.
But the other projects that are importing my stuff also want to check things into their repositories. E.g. keiko/imprt/libag/test/test.hh gets BitKeepered, as well as CVS'ed.

Years ago, I gave the CVS community patches to allow this: I allowed there to be multiple CVS metadata directories, e.g. CVS1 and CVS2, and I allowed the user to specify which to use, CVS1 to check into repository 1, CVS2 for repository 2.

I've done similar kluges with BitKeeper and Subversion. E.g. I have checked CVS subdirectories, containg CVS/Root, CVS/Repository, and CVS/Entries files into BitKeeper - so that a bk clone or bk pull can always be kept in synch with CVS.
It is hard to go the other way: distributed version control systems such as git and BitKeeper, that clone an entire repository, *can* be checked into other VC systems, but don't behave nicely, wrt disk space and other issues. The problem: they do not clearly separate the concept of repository and workspace and links between the two.
AMD's "buildlist" methodology - basically a list of all files, and their RCS/CVS versions - lent itself well to this.


---

Reading through some of the interminable "Bzr versus Git versus Hg" comparisons, I see people asking "Who cares about history? You only need the last few years of changes."
Well, I care about history. I have been accumulating code snippets, library functions, and tools for more than 20 years. Some may date back to 1980. I probably have SCCS (!) files dating back to 1985-1987, copied and/or checked into RCS, and then CVS. There are some big gaps in the record, corresponding to periods where changes I made to my libraries were owned by companies, not me --- but I have a long history. Not very dense - some of these tools may sit unaffected for years, e.g. I am just now brushing off PerlSQL - but long lived.
But why care about this history? Well, I *have* on several occasions had to back up to 10 year old versions of some of my library functions, when the flavors of UNIX/Linux changed under me.

More Toshiba TabletPC Woes - the Continuing Story

I still don't have a working Toshiba M400 TabletPC - more than 4 months after I purchased it. I purchased it in part to do my taxes and accounting on, but instead I had to use my wife's HP Pavilion tx1420us Entertainment Notebook PC - another Vista TabletPC.

I swear, I will never again buy a machine site unseen over the Internet!!!! And I will return the machine the moment there is a problem, within the return deadline. Unfortunately, I bought this machine for Xmas, and did not open it up within the requisite 15 days.

I am tempted to say that I will never again buy a Toshiba - but I had a great experience with my last Toshiba TabletPC, a Portege 3505.

I really wish that I could return this new Toshiba TabletPC. Preferably with a refund. Or replace it with a new comparable model of comparable price - although after this experience I am shy of the top end Toshiba/Core 2 Duos. Or even replaced with a working PC of the same model.

I hope that Toshiba will eventually get it all working --- but even so, I have not had any use of it for 4 months, and I would be buying a different model. (E.g. delay repairing a new computer, and the value of the repaired computer plummets.)

I probably just have a lemon. I hope it gets fixed by this trip to the repair depot, as opposed to sending it to the local repair shop.

---

The latest installment: Toshiba authorized return to depot for service. But the shipping container did not arrive before I left for spring break vacation, and there was a 10 day timeout. Reset by a phone call.

One thing that is becoming obvious: I don't have time to deal with this. With work, plus personal computer work using my wife's working computer (taxes, etc.) the times I can pay attention to this Toshiba are quantized, at particular weekend days or timeslots.

---

New fillip: after reinstalling the OS, per Toshiba's recommendation, I now am asked for a power-on password. I am unsure whether it is a hard disk password or a BIOS password, or what.

No matter - I don't think I set such a password. I wonder if the power or keyboard problems I suspect are plaguing this machine are causing this password manifestation?

I will just have to rely on the depot service guys zorching the whole thing.

Outlook Rules & ActiveSync Deadlock

It appears that running Outlook rules at the same time as Activesync is synchronizing Outlook/Exchange to your Microsoft SmartPhone causes a deadlock. (Or, at least, a screen that says "25 minutes remaining" for half an hour.)

Cozumel Resorts

I feel obliged to post this, sharing info and all that.

Just had a great spring break vacation in Cozumel. My first, or maybe second, in a warm place.

Stayed at Melia Cozumel Resort, north end, west coast. Pleasant, all inclusive.

Nice swimming off Melia beach, although many rocks to stub feet on. 1 lobster, 1 crab, under said rocks.

Nicest aspect of the Melia, for me, was that it was next to the El Cozumeleno resort. A pier or jetty in front of the El Cozumeleno created a sheltered area of clear water with great snorkeling, whereas the water in front of the Melia was cloudy, caused by small surf on beach.

Net: if you like beach, Melia is fine. If you like snorkeling, El Cozumeleno better. But the short distance between the two resorts makes it almsot moot - just a few houses.

A similar note for people who, like me, like snorkeling and SCUBA: the Hotel Park Royal, at the south end of the island, is right across the street from the Aqua World dive outfit.

Ruby Lee of HP did NOT inspire my work on P6 SIMD / MMX

Minor annoyance.

I was clicking around Ruby Lee's website, looking for graduate students to hire for a summer internship, when I saw this:

• But that's not to mention a third innovation that Lee brought to the industry. She also led the way in creating a set of multimedia instructions that build directly into the core native language of the computer the ability to handle multimedia of all types -- images, voice, animation. _I_n_t_e_l_ later adopted the idea, which consumers saw when the company began advertising that its chips had multimedia-accelerating "MMX" technology. Now essentially all computers use the technique, allowing multimedia to be an everyday part of computing.

(1) This could be interpreted as saying that Ruby Lee takes credit for inventing SIMD instruction sets, leading to _I_n_t_e_l_ implementing MMX. Or (2) that she was one of many pioneers in that field.

Statement (2) is probably true.

Statement (1) is false. I was there, at the start of MMX, and earlier, when we were calling it P6 SIMD. I have several personal claims to fame in this regards: (a) I am responsible for MMX being 64 bits rather than 32 bits; (b) I am responsible for MMX being placed in the floating point register file, as a way to get around OS context switch issues; (c) I was told to destroy the slides for P6 SIMD, after P6 decided not to do it, but instead I sent the data to Israel, which helped kick off MMX; (d) after the i750 video chip was cancelled, driving back in a car with Bob Dreyer, we decided that we had to do something like P6 SIMD or MMX, because the special video chip was gone. This led to the FP register state. (e) I handed out pieces of swiss cheese at the wrapup dinner for MMX, protesting the irregularity of the ISA. I should have had those pieces of cheese bronzed.

With 100% certainty, I can say that I was never aware of anything Ruby Lee or HP were doing during the development of MMX.

HP was so far out of my field of vision that I don't know when HP shipped multimedia. I vaguely remember email from Konrad Lai about it about a year before MMX shipped - as P6 was being wrapped up (without MMX), as P55C was locked and loaded(with MMX), and as P6C (with MMX) was ready to follow P6 swiftly out the door.

Why were we later, when I suspect that we actually started first? Mainly, the usual: the company is a big slow dinosaur (although capable of reacting really quickly when under threat).

Fred Pollack was the manager who encouraged me to work on P6 SIMD. If Fred knew about HP's work, maybe HP deserves credit. However, Ruby's NSF bio says she worked on it 1992-1996 - P6 SIMD may actually have started earlier than HP's work on SIMD multimedia.

I personally credit the i750 video processor guys for the SIMD instruction set architecture. All I did was show how to integrate it into a general purpose out-of-order processor. And protest the stupid restrictions that were placed on it - we should have had all combinations of {8,16,32,64} {unsigned saturation, saturation, no saturation} {+,-,*,<<,>>}, etc. as well as less regular combinations such as unsigned+signed giving unsigned, unsigned+signed giving signed.

---

This isn't intended to be a criticism of Ruby. Puffery is to be expected in university brag letters, etc.

Am I paranoid, or do I just have a REALLY slow PC at work?

For quite some time now I have been complaining about the slowness of my laptop PC at work - especially when compared to similar computers I use outside work.

The slowness often manifests itself when I power up from standby, hibernate, or completely off.

I often blame the slowness on virus scanners, etc.

But I wonder: could my laptop be occupied by a rootkit?

Note that I don't say "malware such as a rootkit". Possibly it could be a rootkit placed by bad guys.

But possibly also it could be a rootkit or other monitoring software placed by my employer's IT Security people. This comes to mind because of patent applications 20070083739 Processor with branch predictor and 20070083735 Hierarchical processor, invented by me, assigned to and prepared by Centaurus Data LLC and related companies.

http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=11&f=G&l=50&co1=AND&d=PG01&s1=glew.IN.&OS=IN/glew&RS=IN/glew

http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=12&f=G&l=50&co1=AND&d=PG01&s1=glew.IN.&OS=IN/glew&RS=IN/glew

Am I paranoid? Or do I just have a really slow system? And are virus scanners and other cruft really wasting as much time on other people's conputers as they seem to be on mine?

SmartPhone blog disappears?

I was about to post about how happy I am with my new SmartPhone - and, indeed, I am very happy. It does everything I bought it for:

• phone
• synchronze with my calendar, so I don't have to use my PC to figure out what meeting I ned to go to.
  
• Slow PC. Bad virusscanner
• E.g. my SmartPhone beeps me awake at 6am, I roll over, and look at whether I have a 7am, 8m, 9am, or whatever meeting. I.e. my SmartPhone has made it safe to sleep in?
• pen input with Transcriber
• wifi web browsing a bonus
  

But... I entered a nice blog entry using pen input. Scrolled down, hit the "post" button. Nothing. Nor using the scroll button.

Evetually my post evaporated. So much for it being saved automatically.

So, not everything is hunk dory. But I am still happier with my SmartPhone, my AT&T Tilt (HTC 8925) than I am with any laptop or tablet PC I have bought or used recently.