Quoting is the source of much evil (or the lack thereof)

Quotification, or the lack thereof, is the source of much evil.  SQL injection type errors, etc. 

I've written already about a modest proposal, to apply a simple taint bit to bytes saying "This is data, not language syntax."

I hesitate to expose my own bugs, but what the heck:

Today I just had a bug that looked like

bash: syntax error near unexpected token `('

terminating .bashrc too early, before all path setup was done.

as I tried to make my UNIX and cygwin environments converge.

Bug was insufficient quoting in

eval $pathvar=$elem:"$pathval"

in some code manipulating the PATH. Had problems with filenames containing special characters like space and ()s.  Such fiulenames ae relatively uncommon on *IX, but common on Windows, and Cygwin is both.

Fixed by adding quotes

eval $pathvar="'$elem'":"'$pathval'"

At least I hope it's fixed. What if pathvar has a special character? Nah...


Part of the trouble with quotification is that you have to get it just right.  Too many quotes are bad, just like too few.

I might like to do

eval SAFE(SAFE(SAFE($pathvar)))=SAFE(SAFE(SAFE($elem)))::SAFE(SAFE(SAFE($pathval)))

which might really be considered

eval $pathvar=$elem:$pathval

where I have used green text background to indicate stff that should have no sntax when eval'ed, and red to indicate syntax.

This works okay of the eval is immediate.   But if the results of eval are themselves evalled an unknown number of times, a simple bit is not sufficient.  Nesting...

---

khb commented:

So to be clear, you suggest that we have (for want of a better term) a bit to indicate that something should be treated as data? Was that the sort of thing the near legendary Intel 432 did? 

I'm not saying it's a bad idea, I'm just trying to map it into historical perspective ;>

Responding inline, because one of blogger's weaknesses is no formatting in comments. (Probably deliberate, to prevent complex conversations.)

• I do want metadata associated with data
• This might be a tag bit per memory location (but it doesn't have to be)
• actually, I can't remember if the iAPX 432 had such tag bits.  It may not have needed them, because it treated everything as object oriented
• one version of the i960, and the IBM AS/400, had such tag bits.  However, they used the tag bit for privilege - what they did would not have helped directly for SQL injection type attacks.
• But it doesn't need to be a tag bit.  Much discussion on

• http://semipublic.comp-arch.net/wiki/Memory_metadata
• http://semipublic.comp-arch.net/wiki/Tagged_memory
• http://semipublic.comp-arch.net/wiki/Potential_customers_for_tagged_memory
• http://semipublic.comp-arch.net/wiki/Poor_Man's_ECC
• http://semipublic.comp-arch.net/wiki/Compressed_memory_for_metadata
• It doesn't even need to be hardware.  Although I may sometimes discuss HW/ucode implementations, this particular issue, SQL injection and other non-machine code, eval, attacks, can be addressed completely in software.  
• E.g. have the language parser just operate on wide bytes, 16 bit or wider. Wider than you use for normal language input.
• Reserve the upper bits for these "tags".
• Have the compiler refuse to accept a semicolon-with-non-syntactic bit attached as language syntax.  Similarly for keywords.

Yes, it's tags.  But not necessarily HW tags like some old AI machines have.  SW tags are good enough.  And, in this case, only in strings that you are going to give to a language parser, compiler, evaluator. Basically, use wchar, wide char, 16 bit or wider characters, where some of the

Heck: I edit in gnu emacs, where every character in the buffer can have fairly arbitrary attributes - like 'keyword, 'variable-name, etc.  Why not make use of such attributes in the character set that the language parser accepts as input?  Especially if can reduce security bugs.

Note: we might not yet be ready to jump to semantically enforced tags - where keywords like IF THEN ELSE would be *required* to have the 'keyword tag on them.  We probably still need to be able to accept untagged ASCII characters as input.  But, we might be ready for semantic hints - tagging something 'definitely-not-a-keyword.  If the parser tries to take such definitely not tagged character as part of a keyword, that could be an error. (As opposed to making it part of an identifier.)

abort: filename contains ':', which is reserved on Windows: ...

The usual: record one of my stupidities, in the ope that it will save someone else some time in the future - perhaps amnesiac me:

Just spent longer than I should have wondering why I was getting 

abort: filename contains ':', which is reserved on Windows: ...

errors - when all I was trying to do was hg update -r default, back to a version that I had been using just yesterday.

This on cygwin, on windows.

Turns out that my current revision had a broken ~/.bashrc, so instead of using cygwin mercurial, /bin/hg, I was using Windows Mercurial, /cygdrive/c/Program Files/TortoiseHg/hg

And the Windows Mercurial enforces Windows naming conventions.

Fix: just say /bin/hg ...

Why Gmail doesn't want me to be efficient

I have long complained that Gmail seems to want me to be inefficient.  E.g. instead of giving me a scrollable list of, say, 500 emails in response to a search, they insist on showing me screenfulls at a time.   Plus, they don't easily adjust to my current screen size - they always give me circa 20 items at a time.   Even when I am on large screen that can support 120 such.

Now, there may settings, like showing 100 conversations per page.  But (1) those don't apply to everything, e.g. not the searches - and if there is a setting that does, I have not found it. (2) They have not provided easy to adjust controls, such as dragging to enlarge the numbers shown.

I used to imagine that this was due to web HTML / Javascript limitations.

But now I suspect another reason:  if they help me be more efficient, I look at fewer ads.

Positive confirmation of success

Test suites should not just say "all tests passed".  Or, even worse, return success silently.

They should at least optionally list what tests have been run.  Or at least a summary of how many tests have been run.

It has happened before that tests have been omitted by accident.  Test suites themselves can have bugs.

This is just an example of where positive conformation of an action can be important.

However, it is nice to be able to turn off that positive confirmation, e.g. in UNIX pipelines.

Myself, I like "folding"such confirmation.

Metadata for Mercurial

http://stackoverflow.com/questions/4443712/is-there-a-mercurial-extension-like-svn-propset/11679615#11679615

(I'm not saying anything that I haven't said before: metadata needs to be versioned (everything needs to be versioned), but metadata may be imagined as transcending versions in normal operation.  Looking at these Q&As helps me crystallize my thoughts.)

I'm going to add a really late quasi-answer to this question, since it is one that many, many, people using Mercurial, including me, ask - something we want to have, and expect to have, after using other tools like svn properties.

As far as I can tell, there is no such extension for Mercurial. yet.

Yes, the Mercurial way seems to be to track files and only files.  And it might be that they way to right such an extension is to put the necessary metadata into ...repo/.hg* files.

OK.  I've been playing around with that.  Doing stuff by hand, before trying to write tools.

The key weakness of the versioned .hg files approach is that if you check out a non-tip version, e.g. "hg update -r OLD-VERSION", you get an old version of the metadata.

I think the key thing, therefore, may be to put the metadata in ...repo/.hg* files.

But...  most operations should be performed with or upon the latest version of such files. I.e. I think such metadata files "transcend" versions - i.e. they want to be versioned, but in an ideal situation you might imagine them as overlaid upon the normally versioned files that you may have checked out an older version of.

Moreover, in many cases you want the branching of such metadata files to be handled separately.  E.g. imagine a file where you are trying to write a description of all branches, together. Perhaps comparing branches, like "branch1.1 is a more recent version of branch1".  You don't want that description to be on either branch; or, rather, you want it to be on both brancges, at the same time, and you want changes to be reflected on both branches.

Such a putative extension would either operate on "hg cat -r tip ...repo/.hg-my-new-metadata".  Or it would somehow overlay the versioned of the files with the normally version transcending metadata files.

I have made some progress on doing this with subrepos:

    superrepo
       files // normally-versioned-files <-- a subrepo
       metadata // version transcending metadata <-- a subrepo

this allows me to check out the latest metadata alongside an older version of the files

It's not quite there, because checking out a particular version of the superrepo may get old versions of the metadata subrepo. But at least the newer versions are in the subrepo.

Let me also note that, whether you put the metadata in a neighboring subrepo, or keep it in the same repo (but operate on the tip), there is a problem that you can do

    hg clone -r OLD-REVISION repo newrepo

This will strip out metadata later than OLD-REVISION.  Including the metadata that says that "OLD-REVISION passed all of the tests", i.e. it will strip out metadata from a later revision that might apply to OLD-REVISION.

This same problem happens with hg tags.

One might say "well, never do that" - never strip out history.  Unfortunately, that is often recommended as a way of "tidying up" the repository.

It seems hard to avoid this with Mercurial.

Wanted: hg branch -r REV

I want the following in Mercurial: -r REV argumn for hg branch.

Default "hg branch" is equivalnt to "hg branch -r ."

Branch of some other REV is "hg branch -r REV"

Actually, what I really want is this in revsets: "hg log -r "branch_of(.)"

Then I could do: hg log -r "max(ancestors(.) and not branch(branch_of(.)))"

- the most recent ancestor on a different branch

Ah: matching (just not in the version a work).  But it is not quite on...  it returns revs, not branch

co -r REV workspace = overlay co -r REV source + co -r LATEST metadata?

The Mercurial way seems to be to handle files and only files.

Metadata, like svn proprerties, indicating whether tests habve passed or not, in the Mercurial mindset lives only in files.  See, for example:

http://stackoverflow.com/questions/4443712/is-there-a-mercurial-extension-like-svn-propset

I actually am okay by this. Keep stuff in files.  However, I think there are issues wrt how a workspace should be assembled: more and more I think of a workspace as an overlay of metadata, in files, that transcends versions, and the actual versioned source code files.

I.e. if you have a file that describes all of the branches in a repo, and what they are used for, it rather defeats the purpose if, when you update the workspace to a branch, you lose other