My own blog entry http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html is a much larger example.
Related Blog PagesThe original page, http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html, is more of a shopping project - a page where I take notes, a sort of informal review. Posted in the hope that others may find useful.
It soon became obvious that I was very hopeful about Quadro, a particular iPhone remote control facility, with multiple buttons, whose main claim to fame seems to be using AppleScript/OSA commands. I was hopeful this might make it more reliable than the many, many, macro facilities I have used in the past. Perhaps; but Quadero still has reliability problems. Split into a separate page,
Quadro - I love it when it works, but it is often unreliable.
http://blog.andy.glew.ca/2016/01/scripting-raw-io-macros-versus.html is a somewhat generic discussion of the pros and cons of the various levels of abstraction found in such tools. One of my big frustrations with Quadro, second to reliability, is that I cannot drive all user interface actions because Quadro does not seem to provide all raw keyboard and mouse events.
http://blog.andy.glew.ca/2016/01/notes-and-thoughts-about-dynamic-keypad.html for still more generic thought.
For the umpteenth time, realized that I should have written this in wiki rather than blog. Googlesb blogspot/blogger blog really does not provide much support for pages that need to evolve by being split and otherwise refactored. Let alone transclusion.
Macros, scripting, etc.
It is good to be able to automate repetitive or hard to follow tasks. Workflows. This is often called scripting.
There are at least three different types of scripting:
- Raw I/O scripting - often called macros
- Sometimes limited to keyboard, keystroke, macros
- Sometimes able to generate all user input events, such as mouse clicks, mouse motion, etc.
- AutoHotkey on Windows
- Karabiner on Mac - although mainly a keyboard remapping utility, it can generate sequences of actions.
- EMACS keyboard macros (which can generate mouse clicks)
- Essentially, maps user input events to raw-ish user input events passed on.
- Often unidirectional - often only limited ability to see what the application(s) or system have done.
- Application Level Scripting
- EMACS elisp
- Microsoft Visual Basic for Applications - in many MS apps, such as Word/Excel/Office/...
- Applications often have scripting languages embedded within them - of varying strengths
- Sometimes simplistic and of limited ability
- But sometimes a "real programming language" (elisp, VBA, ExtendScript).
- Sometimes only a subset of app operations
- Sometimes able to do ANYTHING the app can do
- e.g eLisp, where pretty much everything in the app is written in eLisp, apart from low level C functionality
- System Level Scripting
- Comprehensive framework - assuming applications comply
- Extendable - applications can export a command vocabulary. Which is typically an interactive menu hierarchy
- Requires a messaging / IPC event framework
- UNIX shells (bash, etc.)
- More limited abilities than AppleScript
- Extensible - but only if apps actually provide functionality on command line.
- Mainly oriented to UNIX like "command = process spawn/execute/exit"
- But some apps go to the effort of creating special commands that can interact with a running app. (E.g. emacs-server)
Similarly, Cocoa keyboard defaults map a larger input set to Cocoa commands - which are neither full system commands nor application commands, but commands common to Cocoa input boxes.
- keystrokes, w/wo modifiers
- mouse clicks, w/wo modifiers
- mouse/pointer movement
- handwriting recognition
- speech utterances
- simplistic - event X gets mapped to event or command C, wherever
- context based - event X gets mapped to event Y or command C, but only in a certain window or application
- sequential - a sequence of events X1, X2, X3 can be recognized, and mapped to event Y or command C
- e.g. I used to regularly say something by voice to Dragon, then type a key
- and have the key action occur before the voice command, that I assumed had been performed first.
- it used to be common to see such ordering problems between mouse events and keyboard events, or between different keyboards (e.g. a regular keyboard and a separate numeric pad (I have a mouse that is also a numeric pad))
- similarly, if different keystrokes get routed to different event queues.
- such problems are less common nowadays, although I think it is mainly due to faster CPUs, and a reduced repertoire of devices
- I expect to see such problems crop up again if I use Quadro or keypad - i.e. if I use an I/O device that is across a network on a separate computer (smartphone), at the same time as I am actually using the PC keyboard and mouse.
- Send a command to application #1, whether by raw IO macro or by a command interface
- Then send another command, that assumes the the first command s completed.
Scripting often inserts arbitrary delays - "delay 1 second" to give the first command a chance to complete.
E.g. I have the Parallels virtual machine installed on my MacBook, and can script using Windows apps. If the VM is up, the commands run fairly quickly; but if the VM is not up, then a delay of minutes (yes! :-() may be required.
TBD: link to the AppleScript doc that says "Don't script interactive".
Problem: if one command in a sequence des not work, then they remaining commands may do something completely unexpected.
I have often suggested that filesystem level transactions be exposed to scripting languages like Perl.
AppleScript system level scripting has error handling - "try". But experience shows that many script programmers do not properly use it, and/or the applications that interface to them export commands, but not error handling.
Raw I/O macro scripting often has almost no error handling ability.
With the consequence that such "Accessibility" features are often used by speech recognition or handwriting recognition software.
Above, I started by talking about (1) Raw I/O, aka macros, versus (2) Application, versus (3) System.
An intersecting but orthogonal distinction is: what the output looks like:
- Raw User Input Events
- once again, keyboard events, mouse events, etc.
- Menu Commands
- Fully Capable Commands
This is what I mean by a Full Commands.
- e.g. automatically create "Global Menu Commands" from menu item titles
- warning when no (no longer) globally unique
- e.g. to permute full menu path navigation using menu item names
- again, warning if menu structure changes so that a path is no longer accurate
- e.g. to provide raw input event navigation to scripting
- and otherwise provide facilities to handle errors
- In addition to a Fully Capable scripting interface
- derived from the internal function names, if you are okay by that.