UNIX tools and special characters in filenames

See, fior example:  bash - Is there a grep equivalent for find's -print0 and xargs's -0 switches? - Stack Overflow:



'via Blog this'





UNIX tools are great, with their composability - find | grep | xargs | etc.



But UNIX tools have problems handling entities or objects, such as filenames, that have special characters such as blank spaces or newlines within them.



UNIX tools typically operate on lines (grep, xargs'input), or on words separated by whitespace (e.g. backtick expansion, xargs' invocation of other tools).



Some UNIX tools provide the option of using null separated strings, such as find -print0 or xargs -0.



But as the stackoverflow page shows, people want such flexibility in other tools, like grep. Of course, GNU grep has provided it - --null - but there are probably other such tools.   ... cat?  but of course tr '\n' '\0' ...   still, the list continues.  Mercurial?  Git?



Moreover, null separated is by no means the last word.   What if nulls are allowed in the strings that your are manipulating?  Need either a quotation system, such as XML (and then we get into the issue of quotes upon quotes), or a strings-with-length system.



I have elsewhere talked about making all UNIX tools work with XML.  This is a generalization.



Strings-with-length is most general.  Possibly fragile.  Possibly XML clauses wrapped around simple "obvious" quoting.

I wish that EverNote / OneNote had 1990s era Infocentral's linking

Why InfoCentral?: 'via Blog this'



For the umpteenth time, I am trying to use EverNote to collect shopping research.  And it sucks because Evernote doesn't really have hierarchy.



Evernte has notebooks. And stacks. And tags.



OneNote is slightly, moderately, better than EverNote.  It has books, folders, groups of folders, and notes can have subnotes.  But that's it.  Oh, yes, it has tags.



Gmail has tags, aka labels.  Or are they folders?  Really, folders implemented by constraining the labels system.

 Better, but\the tree structured folder constraints make non tree structured labels harder to use.   Some labels want to bve tree structured, some do not.



I think the problem is that developers are trying to maintain a paper mindset, using "abstractions" that behave somewhat like real objects.  Real paper manila folders cannot be arbitrarily recursively nested, and hence EverNote // OneNote should not. Bzzt!!! Wrong!!! I want to take advantage of what a computer can do that paper cannot do.



  And, yes, tags in theory can be used to implement everything that a folder hierarchy has - but only in theory.  Because to really accomplish this you have to create a really ugly tag naming system.



I have elsewhere posted about how I even want my tags to be organized, possibly in a hierarchy.  Because just plain searching through the approved list of tags can be a pain, when you have a lot of tags.





--



Gnashing my teeth about this, I reminisced about InfoCentral.  The very first note organizng software that I used on a tablet PC - way back in 1996-7.



InfoCentral was by no means perfect, but it was better than tags, better than hierarchy.  Infocentral was all about links between objects. Links that were reversible, unlike in hierarchy.  But where you could use hierchical browsing up to the point where it faild, abd then "shale the tree"



So you could look at a family as



       Father - John

           Son - William

               Grandson - Simon

               Granddaughter - Evelyn

           Daughter - Sonia

                Grannddaughter - Mildred

Or shake the tree to look at it from somebody else's point of view

       William

           Father - John

           Son - Simon

           Daughter - Evelyn

           Sister - Sonia

               Niece - Mildred

and then continue browsing.

OK, so InfoCentral wasn't smat enough to know that son's son = grandson.

Or to group sons and daugters as childrewn.  Or sisters and brothers as siblings.

And Infocentral wasn't smart enough to do the classic pivoting:

          Sales/Year/Month

          Sales/Month/Year   for month comparisons between different years

But Infocentral allowed me to do a lot of what I wanted.

I wish something like nfoCentral were available on the web, in Evernote or OneNote.

I'd love to have the time to extend the approach.

Hidden Files in Perforce — Encodo Systems AG

Hidden Files in Perforce — Encodo Systems AG:

'via Blog this'

Security model:

• user can see everything - file names, file contents
• user can see file names, but not file contents
• with an error indication if trying to access forbidden file contents
• user can see neither file names nor file contents
• with an error indication "some information was forbidden for you to see"
• with no error indication

 Different error models when scanning / listing directory trees / enumerating
and probing for a single filename / identifier.


Any query that might potentially return multiple file objects - e.g. opening by "filename", on a system where there can be multiple file objects with the same name, to be disambiguated by extra metadata (keywords, version numbers) can have the above apply.


filenames are just one form of metadata that can apply to file objects.   Other metadata may apply: keywords, version numbers, cryptographic signatures.    Should be able to handle situation where some but not all metadata is accessible:

     e.g. filename is allowed, file contents access is allowed, but access to certain crypto signatures is not allowed - may not even be allowed to see who has signed things.

     Each such metadata instance should have any of the above properties: visible, forbidden with error notification, forbidden silently.


This extends past visibility to permissions such as writeable, appendable.


Similar treatment for "obliterate" - completely removing an object from repository.  E.g. removing proprietary code erroneously checked in to an open source project, or vice versa:

Such removal is just like a permissions failure, with no possibility of getting around it (except possibly for backups...).

Todo-lists versus task queues

New in my emacs org mode setup:

"TODO-LIST-EMPTY(-!)darkorange"     ;; empty list, waiting for stuff to be added.

;; really, there are 2 different concepts

;; (1) a todo-list - an object that may start empty and be completed

;; (2) a task queue - an object which may repeatedly transition from empty to non-empty to empty to ...

;; We use the same term for both.

E.g. there is a queue of personal items - my personal todolist

Or "to-do today" - really a queue.

Or, more like:

"todo today" is a list, which one hopes may be completed at the end of the day.  but usually is not.

items left over at the end of the day may be moved forward to tomorrow, or next workday.moved to a longer term todo list or tracker. Or abandoned.

Conceptually, todo-today is a view of my overall todo queue - not necessarily a snapshot at a single point in time.

Something like a query "all todo list items from (midnight,midnight], that are on my top priority list, as of the end of the daty (or the current time, if not at end of day)"

Smart alarms for when awake and working

The link is vaguely related, but this post was not prompted by it - I just wanted to have some link to the state of the art: Five free apps to help remind you to take a break - TechRepublic:



Like many folks, I sit too much working at my computer.



I have an activity monitor, my Basis Watch.  It tracks how long I sit still.   In the past few weeks, I have still for as long as 4.5 hours at a stretch - that's absolutely still, in my chair at my keyboard, typing.  Nearly every day I sit still for 2 or 3 stretches of 2 hours. Basis allows you to set a goal - "Don't Be A Sitter: from 9-5, get up every N hours."  I currently have the goal set at 2 hours - because whenever I move it lower, say 90 minutes, it gets depressing.



I have tried setting alarms to remind myself to get up and work.  Right now I have alarms set at 11:30, 2:00pm, and 3:30pm. Why so irregular? See below.



This post was prompted by my 11:30 alarm going off.  Unnecessarily, because I had just been active, walking over to a coworker's desk.



Having the alarm go off unnecessarily is irritating.  Having it good just as I am settling back at my desk to get back to work disrupts my concentration, breaks my flow.  Having the alarm go off when I am 30 or 45 minutes into a good working period, into flow, really pisses me off.  I have this theory that interruptions while you are in the middle of a critical b it of work, several things up in the air, in your head, is one of the primary causes of bugs.



What I want is an alarm, a reminder, to get up and move around, that is not at an absolute time.  What I want is an alarm that occurs, say, an hour from the last time I got up and moved around.  An alarm that is smart enough to reset itself



More: I want an alarm, a reminder, that is smart enough to detect (by some heuristic) if I am in flow or not.  (Hmm, I wonder if my Basis watch can reliably distinguish typing.  It's accelerometer is on my wrist, after all.  Since I am a hunt and peck typist, 60wpm, but 80% right handed, I might have to move my watch from left wrist to my right wrist.)



A reminder that is smart enough to try to look for a period to notify me, after I have been working for an hour, but before I have been working for 2.  Looking for a period where I am not working intensely.   Possibly looking fior a period where I am not typing intensely - or possibly looking at what I am actually doing, whether I am working, or in Blogger (like now).    Which is smart enough to look for a good period to interrupt me.  But which might interrupt me no matrter what I am doing after 2 hours of sitting.



You know those prtoducts that try to wake you up at a good point in your sleep cycle? e.g. http://www.sleeptracker.com/how-it-works/



What I want is a product that interrupts me at a good point in my work cycle.



(Ideally it might be smart enough not to interrupt me when I am in a meeting. )



--



Smart alarms when you are awake.



Smart alarms when you are workuing.



Smart alarms should not just be for sleep.

---



(Possibly it could have the sort of incremental alarm feature that a dawn simulator has.  A low priority background notification that ramps up gradually.)





---







OK, I should just go ahead and write it myself.   I installed Tasker for Android to write such scripts. I am not a big fan of Tasker - stupid graphical interface, but worse, my phone battery always drains.   I got the basic timer functionality working, but was not able to detect movement by accelerometer. (Blogging mwe this prompted to re-Google, and I found https://play.google.com/store/apps/details?id=com.kanetik.movement_detection_trial_premium)

Temporary files, security - and filesystem transactions

It is well known that temporary files can be security holes.

Hence

mkstemp(3)

which generates a unique temporary file name, creates and opens the file, and returns an open file descriptor for the file.

"Atomically".



But this may not be enough.

E.g. today I am trying to replace a file with a new file generated from it - by creating a temporary, and then renaming.



Problem:

* I can't rename the file  specified by a file descriptor

* if I rename the file

   * on Linux, the name may have been reused, since Linux allows files to be removed even though open

   * on cygwin, cannot rename if open. but if I close the handle, then the bad guy may be able to race and intercept



We can discuss kluges for this specific case:

* e.g. rename a file specified by descriptor



But the more general problem is

* atomicity

* and the fact that temporary filenames are globally visible.



If the temporary filename were not globally visible, then could securely



      create tmp

      write tmp

      close tmp

      rename tmp



with confidence that nobody else is squeezing between.



More generally, if we had filesystem transactions to guarantee atomicity



      BEGIN TRANSACTION

      create new file1, file2

      write new file1, file2

      close new file1, file2

       abort if error

     END TRANSACTION



Then we can create multiple such files, without having to mess with temporary filenames,

and without having to rename the temporary filenames to the official filenames.

We can use the official filenames from the get-go.



I.e. filesystem transactions automatically create secure hidden temporary files.

Without error prone programming.



---



The same may apply to shared memory transactions - but is most interesting when the shared memory has fine grain access control, e.g. capabilities, rather than the "shared-memory = security hole" we have nowadays.

The Linux Upgrade Treadmill - Goodbye, Microsoft®

The Linux Upgrade Treadmill - Goodbye, Microsoft®:



'via Blog this'



All software puts us on an upgrade treadmill.



Virtual machines might allow us to run old software that we are happy with.   But then we are just on a maintenance treadmill - upgrading the virtual machine that allows us to run the old software.  Perhaps not so bad if a single virtual machine had all personalities - but inevitably they don't / won't.



Moreover, the biggest reason to upgrade is to gety security patches.  Perhaps don't need to upgrade apps that are not exposed to attackers - but exactly what apps are those?  Open Office... nah.