Disclaimer

The content of this blog is my personal opinion only. Although I am an employee - currently of Nvidia, in the past of other companies such as Iagination Technologies, MIPS, Intellectual Ventures, Intel, AMD, Motorola, and Gould - I reveal this only so that the reader may account for any possible bias I may have towards my employer's products. The statements I make here in no way represent my employer's position, nor am I authorized to speak on behalf of my employer. In fact, this posting may not even represent my personal opinion, since occasionally I play devil's advocate.

See http://docs.google.com/View?id=dcxddbtr_23cg5thdfj for photo credits.

Sunday, January 10, 2016

Quadro - I love it when it works, but it is often unreliable

Quadro is an example of a remote control app, and app that runs on an iPhone, and talks to a server process on a Mac or PC, allowing remote control of multiple apps on the target.  Quadro allows multiple  "palettes" of colorful buttons, with icons and text, to be created on the iPhone/iPad, with each being bound to a keyboard macro or what I am guessing is an AppleScript / OAS command.   I.e. I am guessing that Quadro's claim to fame is that it has a fairly generic OAS driving engine on MacBook side of the connection.

I.e. Quadro is an example of the sort of application that I have written about many times, and for which I started collecting "shopping notes" in http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html. This Quadro page started out in that more generic page, but I quickly became quite excited about Quadro, and started collecting copious notes.

Related Blog Pages

The original page, http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html, is more of a shopping project - a page where I take notes, a sort of informal review.  Posted in the hope that others may find useful.

It soon became obvious that I was very hopeful about Quadro, a particular iPhone remote control facility, with multiple buttons, whose main claim to fame seems to be using AppleScript/OSA commands. I was hopeful this might make it more reliable than the many, many, macro facilities I have used in the past.  Perhaps; but Quadero still has reliability problems. Split into a separate page,
Quadro - I love it when it works, but it is often unreliable.

(1-2 months later, I started trying to use Quadro again after an update, resulting in the post Quadro no longer worked after update ... but now it does (8h wild goose chase later).  Although I would rather not have endured 8h of thrashing trying to get Quadro working again, I am fairly happy with the improvements made, as I say in New Quadro Release - Nice Improvement. Although there are
Ways to Make Quadro Even Better, I emphasize:
I like Quadro enough that I want everyone to use it.  Therefore, I am very Very VERY happy with the new Quadro release!  Everybody should use it! I really do think that it is one of the best things to happen to user interfaces since swipes and GUIs.  When the Quadro folk explain that they have properly secured Quadro, I will be able to recommend it without reservation.
http://blog.andy.glew.ca/2016/01/scripting-raw-io-macros-versus.html is a somewhat generic discussion of the pros and cons of the various levels of abstraction found in such tools.  One of my big frustrations with Quadro, second to reliability, is that I cannot drive all user interface actions because Quadro does not seem to provide all raw keyboard and mouse events.

http://blog.andy.glew.ca/2016/01/notes-and-thoughts-about-dynamic-keypad.html for still more generic thought.

For the umpteenth time, realized that I should have written this in wiki rather than blog.   Googlesb blogspot/blogger blog really does not provide much support for pages that need to evolve by being split and otherwise refactored.  Let alone transclusion.

Quadro

http://quadro.me/

Googling kept turning up Quadro.

Eventually realized that I had written enough about Quadro to warrant aa page of its own: http://blog.andy.glew.ca/2016/01/quadro-i-love-it-when-it-works-but-it.html

Currently, Jam 10, 2016, after a week of using Quadro, 
  • I am still hopeful that Quadro will be a good tool
  • I am frustrated, driven to exasperation, by how unreliable and flakey Quadro is.
    • Almost par for the course with this sort of application, although I suspect that Quadro, because of its command rather than keyboard orientation, hangs more than most more "Raw I/O" based tools in this class.
    • I was about to give up using Quadro today, but I am figuring out that it is more reliable in some environments than others.
  • I can't say that I regret contributing 50$ to support Quadro development, the maximum freemium they provide.  But I sure wish that it was more reliable.
  • I remain concerned about security, lack of.
Apparently the developer's second go around: he tried to make a business of it with an earlier company, Actions.  In the Quadro blog he explains that the earlier company was not able to survive with a one time fee, so with Quadro he is going freemium+subscription: free base, 3$ to get the keyboard, 20$ annual to get advanced features, 50$ forever.
    This in itself makes me sympathetic.  It's hard to make a go, a living, as an independent developer.  I paid the 50$. 

The developer, Cristiano Troffei, writes his blogs and the software documentation and UI in a somewhat incoherent Italian inspired English.   Phraseology that is often hard to understand, and which can be somewhat irritating: "Ta-da!!" "Ehmm, I need to tell you something..."  "Automate any recurring iteration..." (which this native English speaker would replace by "Automate recurring tasks..."). Many more examples.
      I almost gave up on Quadro at this point. This sort of app is inherently a big security risk, and it seems that much malware is written in incoherent English. But I clicked around, and, coming from a non-English speaking country, I am willing to make allowances.  I am probably equally incoherent in French, and negligibly coherent in Cristiano's native language, Italian.  I hope that I do not regret this.
     The Quadro web pages could really benefit by a native English speaker cleaning them up.  If they almost turned me off, I am sure they are turning off other potential customers.

Security?

This sort of user interface application is a huge security vulnerability: you are giving the application the ability to type AS YOU, and to basically do anything you can do.

In this respect Quadro is no different than having a custom keyboard on your cell-phone to replace the clunky Appe or Google keyboard:  Swype, or LastPass. For me, in years past, the Graffiti handwriting recognition.
     Similarly, voice control software like Dragon NaturallySpeaking.
     Or any keyboard mapping software, like Karabiner on MacOS.
     AutoHotKey on Wndows.

Actually, Quadro may be a little bit more secure than a custom keyboard or AutoHotKey: Quadro is an adjunct to your keyboard, and is not necessarily the full keyboard.  I believe Apple had to arrange so that a custom keyboard on iPhone would not be used for the system password.  Quadro inherently does not or should not see passwords or credit card numbers you are typing in on your laptop keyboard. AutoHotKey and Karabiner, however, I believe see all keyboard activity.

But in addition to the security vulnerability of acting as an input device, Quadro adds to that the risk of communicating from one malware vulnerable device, yoyr iPhone, to another, your Mac or PC.   I sure do hope that the Quadro iPhone/PC communications channel is properly encrypted.  I do not know if it is or not - Quadro does not document that in any way that I have found.  I would have trouble recommending Quadro to others unless this can be proven.

Many of the things I most want on computers, like input mediation, are similar security vulnerabilities. This has inspired thoughts about what a security model should look like to avoid such problems.   E.g. not just "this is an approved input device", but also "this device is approved only for input to SW apps of low security importance".  Similarly for the queries that apps like Quadro have to use.

Quadro Problem Reports and Wishlist Items

Problem: Unreliable Swiping: on Quadro you are supposed to be able to swipe from left to write to get to a list of "palettes" (collections of custom pads for different apps).   On my iPhone - an iPhone 6+ - I cannot do this swipe reliably. More than half the time I end up hitting a button.  This can be really bad, if the button hit has a negative effect, like deleting an email.

Hmm...  possibly my swiping is unreliable because I have an Otterbox case. ... No, removed the case, still a problem.   Plus, I use many other apps, e.g. Flipboard, that have similar swipes, that work reliably.

I highly recommend that either the swipe be fixed, or that some other UI element be used or at least be available to get to the list of palettes and the settings and display - the "Quadro Top Level" inside the app.

(Several hours later: I just figured out that pressing on the star icon, at the bottom left hand corner, then Open Library, takes me to the library of palettes, with the settings gearwheel, etc.  So there is a way to get there apart from the unreliable let to right swipe.)

Problem: Need to be able to disable Quadro Palettes and sheets: Quadro comes with a big list of palettes already written for many apps.  This is good.   However, most of them I do NOT want to use, at least not yet.

Especially since my main usage model for Quadro is to have a pair of "email triage" palettes (actually, sheets) (for Apple MacOS Mail.app, for work and personal email).  I do not use the standard Quadro palette for mail. But I do not want to delete the standard.  Instead, I want to only to use my specified palette/sheet.

Problem: Switching out of App and Back changes currently active Sheet for App: E.g. as explained above, I have defined two similar but different "email triage" sheets for work and personal email.   But I still have the original palette /sheet for the Mail app.   When I switch away from Mail, e.g. to read a web page, and switch back, I am moved back to the original palette sheet.  And I then have to remember how to switch sheets. 

Problem: Switching Sheets is a Pain, Non-Obvious:  as explained above, I have three sheets for Mail: the original Quadro Mail palette, and two separate palettes for wrk and personal.  I often need to switch between them (more often than should be, because Quadro switches back to the top level Mail sheet too often).

The official way to switch sheets is the ellipsis, three dots "..." at the bottom of the screen. Not obvious.

I really wish that there was a way to create buttons that could switch to other app pallettes, and/or different sheets for the same app.  Buttons that I could name or provide a useful icon for.    In some ways, a "group" is much like this - a group button leads to what appears to be a sheet, but a sheet which does not appear on the top level of palettes.

Problem: What is Difference between a Sheet and a Group?  

I think that a sheet appears on the top level list of palettes, but a group does not. Also, you can select buttons, and move the to a different palette or sheet, but apparently not to a different group.

Problem: Undo:   Desperately need an Undo feature, for when I am doing edits.  (Like when I accidentally grouped some buttons I did not want.  It took me quite a while to move them back to where they came from, and then place them back into their original arrangement.

Undo is just standard for modern SW. Assumed.  

For my purposes, if Quadro had a text representation that I could version control, I could live without undo.

Problem: Need Text Representation, that can version control and diff 

Like Undo, this is just par for the course, standard, expected.

If I am maintaining a big set of palettes, ones that I will be constantly adding to as I add/remove projects (remember, my mostcommon button is "File to project folder"), then I need version control.

I would also like a text representation that I can easily generate.  Since I have to add folders outside of Quadro, I would like to be able to automatically keep Quadro in sync, rather than having to fall back to manual editing.

An XML representation should be easy.  Many if not most UI tools that allow buttons to be moved around use some such representation.

THIS IS IMPORTANT.  If I find a tool that has text config files for palettes, etc., I will switch to that and away from Quadro.  I will have trouble recommended Quadro until some such support is available.

Wish: Program Interaction on Windows 

Quadro is very Mac centric.  Perhaps not surprising, given developer.

Quadro only provides "automatic integration of menu items" on Mac.  Knowing a bit about Cocoa, I think I can guess at how this is done.

In my experience, Windows is similarly fairly easy to get automatic menus for: the Accessibility interfaces, used so that helper programs for disabled folk can read read the screen.  I highly recommend that such simple "menu scraper" be provided for Windows as well as Mac.

Problem: Odd Select Behavior: Tap, Press, Hold:   Quadro has icons at bottom of screen: star, ellipsis (...), question mark, pencil.

For a very long time I thought that Quadro only had strange "transient" menus: click and slide, menu dsappears if you lift finger.

Eventually I realized that Quadro has both:

Tap, menu stays open.

Press (i.e. tap more slowly), and menu disappears.

OK, I think that I can use this now.   But an overall comment, advice to Quadro developer:  probably one of the biggest markets for something like Quadro is people who, while perhaps not sufficiently disabled to need a screen reader and switches, may not have as much fine grain muscle control as a youngster like you.   Quadro's UI, in many ways, is an impediment to such users.

Several hours later, a similar problem:  All of a sudden the palette I was working on stopped working. Hitting the Archive button on this Mail palette did nothing. Repeated attempts.   Eventually, I restarted both my Mac and my iPhone.  Problem persisted.
      In frustration, I jabbed my finger several times on a button.  Suddenly it worked, deleting several emails I did not want to delete :-(.  Insufficient undo. :-( :-( :-(.

It looks like
a) a quick tap on the button makes Quadro do the action on that button
b) a long press on the button displays the note field as a banner across the top
c) but a moderate speed press and release - slower than a quick tap, but not so long to be interpreted as a long press - selects the button (the button changes color), but does nothing.

I wasted an hour figuring that out, including restarting two devices.   How many other users would just conclude that Quadro is broken, and give up?

By the way:  I wasted an hour on this because Quadro had crashed a few minutes earlier.   I think that it may have crashed because first I created a button to move a mail message to folder "Inbox-Processing/Defer Briefly", the in Gmail changed the folder name to "Defer/Defer Briefly".   I suspected that the change of folder name left an orphan Quadro button dangling.   Haven't debugged fully, but it did stop working - although that may just be this button speed issue - and it did crash Quadro on the iPhone (after briefly displaying overlapping buttons).


Plea:  Please, please, please, allow me to disable Quadro palettes and sheets I do not use!!!!   I really am sick and tired of staring at the Chrome palette while I type these notes into my blog.  The buttons are absolutely nothing that I want to use, and I keep hitting them by accident.  I will probably have to delete this palette before long and I would much rather just be able to disable it until I have time ti figure out how to use it.  But right now, I only have time to try to use my email triage palette/sheet - heck, I don't even want to use the standard Mail palette.

At this moment, there is only one palette/sheet I want to use: my personal email triage.  Soon, there will be two: personal ad work email triage.  I don't want to see anything else.

If I only have these two palettes, you could grep them to indicate not currently active.

Or give me a reduced list of currently enabled palettes.  But right now Quadro is giving me so much crap that I keep tripping over.

Worse, although it switches away from the palette/sheets I want to use, it keeps the bloody Chrome palette up way to often.  I would rather have my Mail palette up, so that I can switch back to it easily.

Suggestion: Work with Companies like Dragon, or SW Companies for the Disabled: As I explained above, SW like Quadro has a bigger hurdle or barrier to acceptance than much other SW.  Such input mediation software is inherently a security vulnerability.  A user has to trust that the Quadro users (a) are not themselves bad guys, and (b) have not committed any errors in app security.

It is interesting to try to think about security models that would make it easier to write SW like Quadro (and keyboard macros facilities, and ...), and "let a thousand flowers bloom".

But until such a security model exists, in a popular OS - until then, security concerns will be a problem for Quadro sales.

At the moment the only way to remove this trust barrier is

a) Open Source (apparently not the way that Quadro wants to go)

b) be part of an already trusted company

Something like Quadro will only come into wide use if a major company - Apple, Microsoft, maybe Google - integrates it, makes t a standard feature of the OS that they sell.  This has happened, slowly, with software for the disabled.

Before that, however, companies like Nuance Dragon are a step partway:

Dragon already has to be trusted, since it is an input device that a user can use for all interactions.  Plus, Dragon has many of the software issues that Quadro has to solve: automatically figuring out what menus are on the screen, etc.

Smaller scale: there are many smaller companies, down to consultants, that provide custom software to accommodate disabled computer users.

If I were the Quadro developers, I would try to work with, and possibly sell, Quadro to this range of company.  (Well, I'm actually more of an Open Source type of guy, but I understand the need to earn a living.)


Problem: Gmail WebApp support broken:  since my main usage model is email, one of the first things I tried was to create a palette for Gmail.   If memory serves, there was no existing palette in the library for Gmail.  But when I went to create one such a palette popped up.

I suspect the following: the Quadro developers are developing Gmail support.  But t is not ready yet.  So they removed it from the top level palette list.  However, they did not remove it from the overall library, so it was uncovered when I started trying to create my own.

Bad SW practice:  when a feature is not being shipped, it should be completely removed or disabled. Shipping something "invisible" but available is a big source of security bugs.  And such sloppy config management does not inspire confidence in using a product such as Quadro.

Problem: Back out of Making Changes:  e.g right now, I started trying to change an Icon.  Had second thoughts.  But in the "Looks" pane, there is no way to say "Don't do anything, throw away all of the changes I just made".

So now I have to go manually find the icon that you originally gave me.

Problem: Searching for Icons:  e.g right now, I accidentally changed to no icon.  OK, I am going back icon to try to get the old icon back.  Happily, there is a big list of icons I can scroll through.   Search for "fast" - I see a few  Search for "Soon"- None.   OK, perhaps I should scroll manually.   But now I can't do that.

To begin with, I am given a big list.   But if I search, it filters the list.  And there is no way to go back to "No filter". Apart from workaround below.

I can work around this bug by selecting No Icon, hitting the checkmark, and ten hitting "Icon/Change..." again - which now gives me the full list again.  But that is really annoying.

Problem: Overlapping Buttons:  I was going to complain about how hard it is to edit a palette in Quadro, versus how easy in the Keypad app.  Quadro does not allow buttons to overlap, which can be a pain when moving buttons around.  Keypad allows buttons to overlap, and has a sort order front to back to determine which is visible.
      I was going to complain - I still complain - but I just managed to crash Quadro in a way such that buttons overlapped. :-)

Wish: More Button and Other User Actions:  Quadro seems only to support tapping on a button to get an action (although long press pulls up some help text (a note), while a press longer than a quick tap but shorter than a long press does nothing, confusingly).

The Keypad app supports more: single tap, double tap, triple tap.  I have used other apps that have more, like long press.

Keypad also supports a "Gesture Pad": in addition to 1/2/3 taps, it supports gestures like swipe left/right/up/down with 1/2/3 fingers, pinching, and rotation.
     At first I was quite excited about Keypad's Gesture Pad, since the new generation of email apps that I am trying to improve on have made great use of swipes in their UI - and I have found that I am much faster at using such a swipe based interface than I am at moving my hand to a button. However, after some use of Quadro I have found that its buttons are almost as good as the swipe based interface. I suspect that the real win in terms of speed/ease of use is not having to move my hand so far.  A swipe interface like Zero or Triage apps on iPhone just allows me to move my thumb where it is, whereas a button in an email-reading app on iPhone woyuld require a reach (unless the bitton, by good design, is placed where my thumb rests).  But Quadro is not an email reading app - the email does not appear on theiPhone screen.  Instead, my hand remains in the same place, and I just get to tap the button. So there is not that much differene in motion between swipe and Quadro, whereas there is a big difference between them and an iPhone mail reading app.
     Bottom Line: both swipe and Quadro allow the full surface of the iPhone to be used for user interface, rather than being limited to the periphery and hence requiring motion.
     It may not be a question of amount of motion, versus precision.  Buttons on an iPhone mail reading app are necessarily small, whereas Quadro buttons are larger, and swipes can be larger still.

Anyway, I am feeling less need to try to multiplex multiple commands off the same button as I thought that I might.  Such multiplexing also goes against one of the theoretic justifications of a interface such as Quadro: that the label or icon displayed indicates what the action will be.
    But: I am running out of buttons, especially since Quadro's limitations mean that its buttons are bigger than the buttons I created with Keypad.
    I probably would like at least one simple multiplex: a tap to perform the primary action associated with a button, while a long press would bring up a group hidden beneath the button.  Typically, a group of related commands.
    E.g. I might have a "Defer" button on which a quick single tap just deferred until the next pass.   While a long press brought up a fancier scheduler, with times like "Defer to Luch", "Defer to this Afternoon", "Defer to this Evening", "Defer to Tomorrow Morning", etc.
      I have long been frustrated by the small set of standard defer times, and the annoyance of having to use an arbitrary date/time chooser if outside the set.
      I suppose that multiple select times could be selected easily by tapping.  Possibly as a wheel, with a display - tap once, it says "After This", again whatever the next period, again tomorrow, again... until it loops back.
      Also, I can imagine pressing a button and then gesturing - e.g. press and gesture N for next pass, e for this evening, T for tomoorow, w for this coing weekend.  But I think that I am dreaming in technicolor: this has graduated from Quadro suggestions, to what I might have in my dreams.

It might be nice to have swipes off the buttons supported.

I have log imagined a "swipe syntax" - e.g. buttons for verbs like "Move" and "Copy", and a set of active labels.  With implicit subject being the currently selected message, swipe verb to object (target label). 2 verbs, N targets => N+2 buttons, versus 2N buttons as Quadro would currently have it, requiring separate buttons for "Move to Label" and "Copy to Label".

(Of course, "Copy to" might be "Add a label, without destroying any existing labels", while "Move to" might be "Remove all preexisting labels and add a new label".  (Or, "remove all preexisting labels of a certain class").  With this label interpretation, a third verb might be imagined - "Create a new object".  With or without the labels attached to the original.)

Wish: Want ability to bind commands to swipes:  Quadro provides the ability to generate swipes via is keyboard, but only allows them to be used for their standard text actions. Some of Quadro's competitors, like Keypad, provide the ability to bind commands/actions/functions to different swipes.

Generic Problem: Race Conditions and Other Errors in Target Actions:  QI have noticed race condition type problems in Quadro: e.g. press the Quadro button for "Archive" too many times too quickly, and eventually the next message is not selected as it should be.
    This is a generic problem, not specifically Quadro.   It is a problem with any event driven UI, if the action is asynchronous: in particular, it is a problem with any macro facility, which "Sends and Forgets".
     I was hoping that Quadro, because it is more integrated with the app than a simple macro like keypad, might be smart enough to detect when the command specified was finished before releasing the next.  Apparently not.

Handling errors in general is the bane of macro facilities.   E.g. some of my old AutoHotKey macros select Outlook folders by sending arrow keys to navigate menus.  If the menu structure changes... Even worse is sending function keys, which may error or be ignored - so that the next set of keys does something completely undesired.
    Quadro may reduce some of these errors by asking the app what menu items it supports, and sending those.  Accessibility frameworks, in both Apple Cocoa and Microsoft Windows, typically support this.  Programs like Dragon take advantage.
     One can think of ways to make input programs even more robust against such errors.   E.g. provide "transactions" - either all of the commands in the sequence are performed, or none.

Debatable Problem: Quadro Buttons are too Large, Text too bulky: In Keypad, on my iPhone 6+, I can fit default circular buttons 6 across width, and 9 vertically.   Or 5 defaut rectangular buttons across, same 9 vertically.  But I can also redue height, so that I can have rectangular 18 rectangular buttons, vertically, with perfectly readable text
    In Quadro,  I only get 4 across and 6 vertically.
    Obviously, I can get much less on a Quadro screen than in Keypad.

Worse, keypad does not have a fixed grid of button sizes - or, rather, it has a grid, but buttons are about 10x larger.  So much more aesthetic effects can be attained. It's the difference between a fixed width font and a proportional font with kerning.

Worse still: I find that I can hardly ever create useful 1x1 buttons in Quadro.  The icon set is insufficiently self explanatory, and Quadro will not display both text and icon at 1x1.  So nearly all of my Quadro buttons are at least 1x2, to get a useful amount of text, or an icon plus a label. 2 acroos, 6 down => 12 total.

I recommend that Quadro allow the text label to be displayed below the icon in a 1x1 box.  Oftentimes just a single word, in small type, is enough to let somebody figure out what an icon is for - and eventually visual association kicks in.  It is faster to squint at small text than it is to long press to see the note.

I would also like to recommend that Quadro support smaller icons in addition to the current size.  But that is a second tier suggestion.

Now, I am surprised that I am thinking about this.  I really do advocate "Few Big Buttons", and surely 12 buttons is more than enough?
     Probably for "top level" keypads (aka palettes).
     But... I sometimes want to escape into larger sets.   E.g. in my usual email triage app, I usually only need Archive / Defer Briefly / Defer Longer plus maybe a few top level projects. Currently 10 buttons total, of which one is used only to work around a Quadro limitation.
      But even here, the 12 button de-fact limitation means that I cannot use a nice big 2x2 button for more than 1 or 2 buttons.
       I.e. allowing smaller buttons for infrequently used tasks allows larger buttons for frequent tasks.

Moreover, this is just in my top level keypad / palette.  From this I may escape into a scheduling palette, with a list of standard times, or a list of projects that is larger than my current WIP (Work In Progress) list.  Although I can usually only handle 2-4 WIP projects, I often have a dozen background projects going on.

+ Time:
++ Soon, as soon as I am finished reading email
++ Lunch
++ This afternoon
++ This evening
++ Tomorrow Morning
++ This weekend

+ Place:
++ At home
++ At work
++ On the beach
++ In the woods
++ Grocery Store

+ Person:
++ Wife
++ Daughter
++ Boss
++ Coworker 1

Problem: Error Handling: Quadro misbehaves if Apple Mail (Gmail) / Move to / folder has been renamed or moved:  Second time this has happened.  I am reasonably certain that there is no other problem, and that the misbehavior - ultimately a hang - has been reproduced, and is due solely to this problem.

So, here's the situation: I want to use Quadro mainly for "Email Triage" - going through my email as quickly as possible.  Mainly hitting "Archive" and moving on, sometimes refiling the email in folders such as "Defer", "Defer Briefly" (which means "handle this after I have scanned all of my email), "Delegate to PersonX", and possibly refiling it to a folder associated with one of a small number of projects. Usually, the small number of projects, 2 or 3, is a subset of a larger list.   Which projects are in my top-level triage list change regularly.  E.g. today my top level projects are "Giving" (receipts for tax purposes for charitable contributions made at the end of the year), and "College Search".

Like I said, these projects change.  They move around.  Sometimes they get deleted, or at the least renamed.  Sometimes they get reorganized, e.g. moved from Projects/Current/Foo to Projects/Completed/Foo.

This is what has just happened.  I changed the name of a project.  There was an existing Quadro button for the project.  When I opened the Mail app and the email triage palette/sheet, Quadro worked - but the button that moved the selected message to the renamed folder did nothing.  It dod nt appear to hang, it just did nothing.

However, when I further tried to edit the button, when I opened "Application", Quadro just plain hung - before displaying the application list. Rotating wheel.  ... Eventually - 10, 15 minutes later? - it timed out.    Since there appears to be no way of cancelling a button (pad) edit in Quadro, I had to tap the check mark. Wt this point I got asked to place the button - then death.  The button eded up placed overlapping other buttons.  Quadro nonresponsive.  Next time I hit checkmark, it just disappears.

When I reopen the Quadro app on the iPhone, it "works", in the sense that I can switch palettes.   It thinks that it is connected to the Mac QuadroSync.   I get Mac notifications.  But none of the buttons do anything.

When I got to this point last time, I restarted both iPhone and Mac.  Restarted Quadro. Deleted the button(pad) that I thought might be causing the problem.   Was frustrated that that did not solve the problem.   Ended up deleting almost all of the Quadro buttons I had created, and then realized that in addition to the "missing move to target", I was not tapping fast enough.  And when I recovered from that, the problem seemed to go away. Now, let's see...

Restarting the Mac but not the iPhone.  Quadro restarts.  Works at first - iPhine Quadro switches between Chrome and Mail when I move between Chrome and Mail on Mac.
      But after first time I tap on the possibly bad button(pad), Moving to the renamed folder, same sort of misbehavior.  Hangs.   Currently I am getting a "Drag the Pad to position", for a pad that I was not editing.
      I.e. gives every indication of a bug such as a buffer overflow.

Quadro hung, so I double-click iHone home button, ad swipe it away. Restart. Still hung. At this point I should probably look at the QuadroSync process on Mac.  But I have other stuff to do.

No, wait...  the QuadroSync process has disappeared.  Restarting.

This time, I am very careful to delete the button with the filter that got renamed.  NOW I can recreate it, without Quadro hanging (yet).  The new button works,

But... I kept a duplicate of the old button in a group.  Click on that, same problem.  Hangs for a while. Eventually, the Quadrosync app on Mac disappears from the status bar.  I assume it has timed out.   However, the iPhone Quadro app still; hangs - either it does not have a timeout, or I have not waited long enough without typing at it.  If I type at it, eventually it dies.  I can hasten its death by double-clicking home and swiping it out.

CONCLUSION: it looks pretty darn sure that if Quadro tries to do Apple Mail.app, "Move to", a folder (mailbox) that no longer exists, that Quadro misbehaves.

This is not a total surprise.  This sort of problem often occurs in scripting that is "blind", which does not check the return status of an application.   Moreover, it is quite likely that, if Quadro is using AppleScript or te like, that there is no way to check that the Mail app is having a problem.   UNIX shell scripts can check, when every command is a separately forked/exec'ed process, by looking at the process exit code. But there are fewer standards for checking the error of a command sent to an app, as if human interaction.   UNIX expect scripts can look for error indications, but there must be some sort of error log or console for expect to run on.  Again, no standard that I know of for such error handling on Mac.

If there is such a standard, the Quadro is not using it.   That would be regrettable.

If there is no such standard, then that is especially regrettable.

Even though I can understand Quadro having this sort of problem, nevertheless the sort of behavior Quadro exhibits is bad.  It gies every idication of a buffer overflow.   I will sketch out below what proper error handling should liuke like, eve if the underlying Mac tools do not support error handling as well as might be wished.

I was really, Really, REALLY hoping that Quadro, because it is more tightly integrated than a VNC sender like Keypad, might handle such problems.  Seems not.

Glew motto: Error handling is the true test of software quality.

TBD: how to deal with errors such as this:

Separate iPhone Quadro and Mac QuadroSync processes, obviously.  On the Mac, need separate comm process and process to interact with applications. Possibly more than one of the latter, so that actions such as "Kill Hung Process" can be scripted via QuadroSync.  Keepalives and timeouts. On UNIX, siagalrm signal handlers.  Report an error when timing out, instead of randomly performing what appears to be a buffer overflow.

Problem: Quadro hangs and otherwise misbehaves on an error such as a "Move to" folder target no longer existing:  The first bug is that Quadro hangs wen its button configuration does not match the app configuratuon.  Whiole pne might argue that is a user error, it is still bad to hand.  (And it is stupid to say that it is a user error.)

Problem: Quadro hangs and otherwise misbehaves on an error such as a "Move to" folder target no longer existing:  The second bug is that Quadro should be able to detect when the folder targeted by Mail > Message > Move to ... no longer exists - because Quadro obviously can get the folder list to set up the button.  So evn though Quadro should not hang it would be even better if Quadro could detect the error before hanging, and proide an error message that the user can use to figureout which button(s) are bfoken, and fix them.

Problem: Need to be able to see "Menu Path" to a Function/Action :  actions in Quadro's functions for buttons (pads) are usually obtained via a path, that often corresponds to a menu hierarchy, probably because Quadro is automatically inspecting the app menus: e.g. Function > + add actuion > Applications >  Mail > Message > Move to > (select folder from a browse list, e.g. "tax" and then "giving")

However, when inspecting a button, all that I can see is the last element "giving".

Problem #1: not necessarily unique.  The same name can appear multiple places in the tree.

Problem #2: if a bug such as described earlier, where a target path got moved without updating the Quadro button....  well, it makes it darned hard to update the Quadro button if you cannot inspect to see where the problem is.

It would we really, really, nice to e able to see what the full path is from the Quadro UI.

Having a text readable data format for palettes would go a long way towards this, and more: the I could grep for invalid folders, and/or automatically keep in synch.

Problem: It is annoying to have to reseelect App each time I add a button:  My application involves creating many buttons in the same palette for the same app.   This will not be just at start: I will have to do this constantly, as projects get added and deleted.

Each time I hit edit (check) and + to add a pad, then applications, I have to scroll a LONG LONG WAY down thelist to find Mail.  And then into the mail.

At the very last, Quadro should remeber that the last app accessed was Mail, and take me there.

(Quadro does put the current app at the top.  But since log each change in a website, this is always Chrome, never the app that I am adding a Quadro action for, Mail.)

Lesser Problem: Folder (Mailbox) list in Apple Mail.app "Move to.." loses structure:  I have three email accounts connected to the Apple Mail.app. Quadro loses some of their structure when it makes them available for the user to choose from.

The snippet below shows the structure, which I will summarize as



















  • Mailboxes

    • Inboxes
      • Exchange
      • iCloud
      • Google














  • Various Mail.app mailboxes, like VIPs and Flagged
  • Exchange













  • exchange folders















  • iCloud

    • iCloud folders














  • Google

    • Gmail folders
    However, while the folders and subfolders are represented with the classic unfolding tree, the top level is indicated by having "Mailboxes", "Exchange", "iCloud" and "Google" in a different font.

    Unfortunately, Quadro, when it makes the folder (mailbox) list available, loses that structure. So, for example, "Google" appears undistinguished from its first folder (mailbox), "Starred".

    (Half of the moving of folders around above, that caused Quadro to hang and then crash, was to try to minimize the ugliness of this flattening.)


    Problem: Quadro does not understand terminating at an interior node of a menu tree:  I will provide first the example that bit me just now, related to email filing. However, this is a generic issue with Quadro which I was already planning on posting about.

    Many Quadro actions correspond to traversing a menu tree.

    The following example corresponds the menus for Apple's Mail.app, as understood by Quadro.  The bold stuff are standard; the italicized stuff are folder names. (Inbox is bold an italic - its a folder name, can even be changed by user, but is standard):

    • Application
      • Mail (Apple's Mail.app on MacOS)
        • Commands
          • Mail
          • File
          • ...
          • Message
            • ...
            • Move To
              • Inbox
                • Exchange
                  • Defer 
                    • Defer Briefly
                    • Someday
                  • Projects
                • iCloud
                • Google
              • ...generic per server Drafts, etc. ...
              • ...  etc
    Note: on my Exchange account, I have subfolders of Inbox. In Apple parlance, I have mailboxes inside the Inbox mailbox, and sub-sub-mailboxes, etc..  E.g. "Inbox/Defer/Defer Briefly".  They work in Exchange.  They work with Mail.app.   (Gmail, on the other hand, does not allow you to create subfolders of Inbox, although it does allow you to create subfolders of most othe folders (labels, in Gmail parlance)).

    Here's the problem: Quadro thinks that you have to traverse all of the way to a leaf of the menu tree.  
         So Quadro does not allow me to Move To my Exchange mailbox/folder Inbox/Defer.   Nor does it allow me to Move To Exchange>Inbox.  It requires that I go all the way to Inbox/Defer/Defer_Briefly.
         This sucks.  I have defined a mailbox hioerarchy, and Quadro does not allow me to use it.  To make my hierarchy work with Quadro, I would have top create special "other" subfolders all over the place.

    This is NOT a problem with Apple's Mail.app.   Apple allows me to Move To Exchange>Inbox and Exchange>Inbox/Defer.  It is solely a problem with how Quadro interprets, traverse, the menu tree.

    Before I was aware that this was a problem with moving to email folders, I realized that it was also a problem with any menu tree.   Many of my own mashups of commands have done things like open a menu top level or interior node, but not traverse all the way to the end. Then, when the macro stops, the menu is open on the screen, and you can finish traversing it via keyboard or mouse commands, which you can have bound to still other macro keys.
         This provides a Quick and Dirty way of parameterization.  Instead of having the toll snarf down the whole m,enu tree, the tool just traverses the hard part to the obvious part.  Often the problem is getting past the first few levels of the menu tree.

    Macros like this are easy to do
    • with Karabiner on Mac
    • with AutoHotKey on Windows
    because they are sending raw I/O events, like keystrokes and mouse movement and clicks, to the apps.
         In fact, that is pretty much the definition of a "macro".

    Command processing systems like AppleScript or Cocoa key bindings may or may not allow stuff like this.   

    Quadro allows  "shortcuts" - but these appear to be only keystroke shortcuts.  Searching frantically, I cannot find any Quado actions that emulate mouse clicks.
           Keypad, conversely (still my main competition for Quadro), supports mouse clicks and keystrokes.  Keypad does not directly support traversing the app menus, as exposed through accessibility and other interfaces,.  But it is straightforward, although annoying, to bind to commands in Keypad.

    I would not care so much about not having mouse clicks on Windows - Windows is pretty good about having a keyboard equivalent of every mouse commands.
         Apple MacOS, however, has long resisted having fully functional set keyboard menu navigation. For example, as far as I can tell there is no standard way to open the context menu from the keyboard.  ... Well, there is Accessibility NumPad+Ctl+5 to open the context menu (Fn+Ctk+I/5 on MacBook). Unfortunately, I can find no way to send a numpaf+ctl+5, as opposed to control+5, from Quadro.
           Similarly, Ctrl+F2 accesses the menu bar, and you can type in menu names and arrows to navigate the menus.   But I can find no way to enter Ctrl+F2 in Quadro.

    Quadro's "shortcut" appears to be limited to opt-ctl-cmd-shift-... plus a keystroke  that can be sent via the iPhone keyboard.  The  default iPhone keyboard has letters a-zA-Z, nuymbers 0-9, and the usual collection of symbols.  (Plus emoji).  But as far as I can see, it has no way of generating function keys like F1.
          Perhaps if I install an alternate keyboard?  Damn, yet another thing to research.

    (OK, I can see that, since Quadro has a keyboard, and also what amounts to a touchpad with swipes.  So obviously Quadro can generate arrows keystrokes.
        Now, I want to be ab le to send them from my own buttons.  I don't want to have to switch to a keyboard to type them.  Besides, in my usage of Quadro, I might as well switch to a real keyboard.
        E.g. in my Quadro palettes for Apple's Mail.app, I can move between messages by quadro keys bound to command+P/N keystrokes, which Quadro's shortcut feature allows.    But Microsoft's Outlook.app uses arrow keys, and hads to P/N bindings that I can find. So I have to switch to the keyboard.)

    I am considering creating Mac OSX shortcuts like cmd+opt+shift+ctl+P/N for arrow motion, etc.   Bit this is suboptimal, a pain to maintain, and part of the reason I tried Quadro in the first place.
         

    OK, the following seems to be  a fairly generic way of getting to the Mac top level menu, that uses only keys Quadro can currently send (no function keys or mouse clicks):

    • cmd+shift+?: get to Help menu in menubar
      • in a text box typing a search term.
      • now, how do I get out of that text box
    • right arrow: wrap around the menu bar to the apple menu
      • more importantly, get out of the
    Unfortunately, macro-ifying to navigate the menu bar is subject to all sorts of problems.  What if menu items change?  
           Oh, wait: there is no obvious way for Quadro to send an arrow,

    Anyway: this started out as "I want to be able to move to an interior node of a folder hierarchy in Mail".

    Generalized to "Often I want to just plain open a Menu, but not traverse to a terminal command".

    And morphed into "Quadro may be okay at exercising application exported commans (menu names), but Quadro does not seem to have ways of transmittig full input macros.  

    Problem: Quadro  macro facilities: no obvious way to tranmit mouse clicks, function kerys, arrow keys, etc:  Quadro's shortcuts seem to be limited to modifier key + a-zA-Z0-9 plus a few symbols.  But no function keys. No arrow keys.  No mouse clicks.
           It looks like Quadro is concentrating on the menu commands exported, typically to Accessibility input managers, also to Apple Script and Apple's standard Keyboard Shortcuts.  Command driven is good, avoids many of the problems of macros - like, what if the menu sequence has changed?
          But, often the command set is incomplete.  At which point you have to fall back to sending raw user input events.

    It looks like Quadro has concentrated on AppleScript-like scripting, and not so much on raw user input.  It only has a limited set of "shortcut keys", and appears incapable of generating function keys and mouse clickls.  I believe that I came across a Quadro blog saying"Do that with AppleScript" or the like.
         I have started writing custom AppleScripts emitting system events for the stuff that plain old AppleScript and Quadro cannot do.  Nooby problems, plus the usual problems with raw I/O macro scripting.  Interestingly, the more of these I have, the  less I need Quadro - I can probably call them from other systems, like Keypad, or even VNC.
         I sure would like it if Quadro providing full "Raw Mouse and Keystroke Event Scripting", in addition to the AppleScript level scripting it sees to be supplying now.

    Inpsired to meta-genericize:
    http://blog.andy.glew.ca/2016/01/scripting-raw-io-macros-versus.html


    Problem:  two versions of app "Microsoft Outlook", Quadro gets confused.
         I have two versions of apps called "Microsoft Outlook":
    1) the Mac native Microsoft Outlook (formerly Accompli)
    2) I have the Parallels virtual machine installed, and Windows Outlook in the guest.
          Quadro starts the Parallels version.   But it sees to send commands to the native version.  Te Windows version exports no commands to OSA / AppleScript / Quadro, although the native Mac version does.

    While I cannot criticize Quadro too much for this - first world problem, aka power user problem (if I am a power user, and not a wants-too-mich-power-for-his-own-good user) - it is still a problem.

    TBD: try to fix by editing resources to give unique names.

    TBD:  I wonder if Quadro's problems with Gmail are caused by having multiple Chrome windows into Gmail?  Hmm....

    While I can imagine it might be hard for Quadro to handle multiple instances, it can probably detect and warn user.

    Problem: "NOT CONNECTED / Tap here to open connection panel" does not work - the tap here does not work - when editing a page (creating a new pad):
          Pretty self explanatory.  More detail: I am trying ti create a new pad.  I hit "Applications".  I get the NOT CONNECTED warning.   But "tap here to open connection panel" does not open connection panel.  I need to exit out of adding a pad - and, in fact, add a broken pad and then delete it (more reason to want a "Cancel Pad Edit), and then hit Star, Open Library, the display oicon to get to the connection panel.
          Typical problem: and error message that may or may not work in one context, but not another.  (Interestingly, I do not even get that error message when I am not editing.  Nothing happens at all.

    Problem (Enhancement): I want 2 distinct pads for same app - Email Triage, personal and work.  Many buttons same, but some (folders) different.  Quadro's sharing imposes annoyances:
          At the moment I have two different sheets for Work/Personal.  5 buttons(pads) that are the same (mark unread/undo/cloise/open/archive).  But 4 buttons that are different - particular refile folders.   And 1, or 2, that are parallel but different - "Defer Briefly" is in both, but different "Defer Briefly" folders fir the different accounts. Plus "More..." button that takes me to

    Now, I know how to have a single sheet for all the common stuff, and different sheets for the per-Account stuff.   But that means that I cannot have a single sheet that has the most common buttons for Work/Personal, some of which are common, some of which are not.
         Pus, switching sheets is not obvious.  I want a button that can switch sheets.

    Similarly I know about groups. I could have separate groups for the common stuff, Work Stuff, and Personal stuff.   But that requires switching between groups.  Does not allow the most common stuff visible all at same time.

    SUGGESTION: superimpose sheets.
       E.g.    Work Triage = Common Stuff with Work Specific Triage stuff superimposed
       E.g.    Personal Triage = Common Stuff with Personal Specific Triage stuff superimposed

    (Plus, of coursem ability to jump to the suoerimposed compoments separately.)

    Plus ability to have buttons link to sheets.  (Which would almost do away with need for groups).

    Problem (Enhancement): Groups are Modeful:
          From my main triage palette/sheets, I go into a group to use the actions that do not fit at top level.  I hit a button(pad) for the action.   But I then have to exist the group to return to the main sheet.

    Reflects two different mindsets:  sometimes a sheet/group is a persistent, modeful, context.  Commands used over and over together.

    But sometimes a sheet/group is an escape:  Too many pads to fit at top level sheet, so escape to another sheet/group, select a command.   But now want to return to top level.

    Once again, having actions that change Sheets/Groups would accomplish both.

    Bug: Quadro just crashed. Jan 7, 2016, 2pm.
        I have experienced crashes before, but before I could guess - e.g. accessing a folder that was renamed in Gmail but not in Quadro.
         This time, no idea.  It just became unresponsive, and then *pop*, the iPhone app disappeared.   Apparently the Mac app, QuadroSyc, also disappeared.

    Bug: repeated Quadro hangs ... 3pm
        I would love Quadro - if it worked reliably!!!
        What I have works lovely - when it works.  But it has hung a dozen or so times this afternoon.   It works for a while, then stops working.   And I have not yet been able to figure out a pattern - although I am beginning to suspect it may just be due to having too many buttons/pads.

    I have rebooted/restarted both my Mac and my iPhone several times.  But I am beginning to see that it may not always be necessary to do a full reboot/restart.

    Request: please provide a "Restart QuadroSync" option
        While debugging the problems mentioned aboe (or, rather, trying to work through them), I have several ties restarted QuadroSync  on ny MacBook.   Usually by going to the Quadrosyc icon in the menu bar, hitting its only pulldown Exit, and then reinstalling.  There must be a better way.
         I highly recommend adding a restart entry to the pulldown menu.   If it worked, it wouyld make this much less frustrating.

    :-) Almost two hours without a crash or hang: :-) Yippee!!!!
        What did I change?  Not sure ....
        Except that I did put Mail.app into "Offline" mode.
        After some of the early hangs I saw warnings like "Mail abort: 2 messages could not be sent to the mailbox "Archive-Exchange".
         I conjecture that these networking problems were and are related to the hangs.

    Of course, I may just have a lot of such problems that will be reflected as "synchronization errors" when next I connect.

    I don't expect Quadro to eliminate such problems.  I do expect Quadro to be well behaved in the presence of such errors.  Ideally, to provide enough information to help figure out what they are - which things are haing problems.

    I was hoping that Quadro, because more tightly integrated (via AppleScript) than a raw keyboard/mouse macro program, might be more robust. While Quadro may be more robust, it is not clear if it is doing all of the error handling that it should be.

    Bug: Timing Sensitive Behaviour Observed
        I just observed a slight Quadro stall (not a full fledged hang), during which I tapped on the keypad a few times, thonking I may have mistyped.   And the events were qyueued up, and applied to the wrong emails
        Thank goodness for the Undo command in Mail.app.
        All SW should have an Undo command.  (Yes, Quadro, I mean you)

    I was hopeful that Quadro would be able to observe when actions are complete.  But it seems, more and more, that Quadro is largely AppleScript/OSA based, and ApplesScripts often need to add delays.

    Timing sensitive programs: bad idea.  I wonder how many of Quadro's built-in palettes have delays built in?

    Problem: Quadro needs ability to rotate dis
        Quadro on my iPhone only runs in Portrait mode.   I badly need it to be able to run rotated, in Landscape mode, if only because at my desk at work I can only prop up my phone in Landscape mode.  And because the iphone cable gets in the way in portrait mode, coming from the bottom.  (I need to plug in charger while using Quadro all day.)


    Problem: Quadro annoying error messages llike 'QuadroSync connection error / The daemon in the cellar just shout: 60
        I am sure that the developer thinks these error messages are amusing.
        Me, I just find them annoying.  And off-putting. Non-professional.
        Professionalism matters, because a product like Quadro, as I noted above, is a big security vulnerability.   This sort of childish error message is the sort of thing that you find in malware and Warez sites.  Every time I see an error like this I think to myself "I wonder if the Quaadro guys have stolen my identity and are emptying my bank account now?"
        Ditto the poor English grammar: "just shout: 60"
        60 what? I assume that it is an error code.  Error code returned from what?  AppleScript.

    Inspires more thinking about security model: Quadro is better than a third party keyboard or mouse, because not all of my I/O goes through it.
         Of course, I don't klnow that: being non-open, Quadro might have hooked itself into the OS in a place so that it sees I/O from  other devices.
        But assuming not that, then the problem is that Quadro can generate everything.  Again, if the Quadro events and commands were tagged with a capability, I could make sure that the really security sensitive apps do not accept Quadro input.
        Q: tag with a capability given to Quadro?  Or tagged with Quadro's ID?  Both...

    Amusing Quadro side effect: causes smartwartch to count spurioius steps
         I wear both a Pebble smartwatch and a Jawbone UP2 fitness band.  Long story as to why I suboptinally wear two.
         Yestertday, I was wearing watch on right and Jawbone on left.  Both counting st5eps.
          I noticed that watch reported 16,000 steps, while Jawbone reported 6,000.  Usually they are within 10% of each other.
         I conjecture that Quadro was causing me to move my right arm more.
         I think that is good: big muscle movements rather than small finger and wrist movements.  And none of the stretching for modifer keys.  I am fairly certain that changing from mouse to trackball had a similar effect, and helped my "computeritis" a lot.


    Jan 8, 2016, 11am: I *LIKE* Quadro
         I like Quadro so far very much, even with the problems above.
         Using Quadro, I have been able to acheive Inboz Zero in my personal email for the first time in months, and am almost caught up with work email.
         I continue to like the Zero iPhone  app - but I suspect that Quadro will make me much less reluctant to use my laptop.
         I like Quadro enough that I am seriously considering buying a cheap tablet to run Quadro on. Larger surface area, more buttons than fit on my iPhone 6+.

    Problem: Quadro really needs to be on Android, other cheap tablets
         I like Quadro very miuch. Enough that I am considering getting a tablet just to provide extra surface area for more buttons.
         Problem: Quadro only seems to be available on Apple iOS, iPhone and iPad.
         This is a major obstacle.  Cheap Android tablets are widely available. Apple iPads are much more expensive.  Even a used iPad first generation costs >90$, at least on Amazon.  New Android tablets are available for <40 br="">    (Plus, I have two cheap Android tablets that I never used - they were given away as conference swag.  Never used, because too slow.  But may be fast enough for Quadro usage.)

    By the way, it would be nice to have a statement about minimum system requirements.  Will Quadro run on a 16GB original iPad?

    Long-term - if the Quadro developers really wish to push it - of course, don't even need Android or iOS.  Just need a cheap touchscreen and a radio.  Probably Linux OS.

    The biggest reason to be on Apple is the ecosystem - there is greater certainty that the developer will get paid.  Android almost as good, although easier to pirate the software.  Linux ...   might as well give up hope of getting paid.
          But there is probably a market for selling a dedicated Linux I/O device like Quadro.  But that gets you into the hardware business, which is not much fun.



    Minor Problem: QuadroInstaller.HelperTool left running, even though QuadroSync Exit invoked

    I have started looking at the running processes, since I have to restart QuadroSync and Quadro so often   I noticed the following minor problem.  (The fact that I need to restart so often, that Quadro hangs so often, is not a m inor problem.)

    "Exit" should mean "everything".   There should be no Quadro stuff left running

    Initially QuadroSync is running
        $ bash $>  ps alxwww > ps2
        $ bash $>  grep -i quadro ps2
    0  6125     1   0  31  0  2469944   2688 -      Ss     ??    0:00.01 /Library/PrivilegedHelperTools/com.quadro.QuadroInstaller.HelperTool
          501  6127     1   0  46  0  4123340  90048 -      U      ??    0:01.74 /Applications/QuadroSync.app/Contents/MacOS/JavaAppLauncher
          501  6132     1   0  46  0  2576088  12320 -      S      ??    0:00.04 /Library/Application Support/Quadro/QuadroNGui.app/Contents/MacOS/QuadroNGui
    I go to the QuadroSync menubar icon, right click, click on exit
        $ bash $>  ps alxwww > ps3
        $ bash $>  grep -i quadro ps3
    0  6125     1   0  31  0  2469944   2688 -      Ss     ??    0:00.01 /Library/PrivilegedHelperTools/com.quadro.QuadroInstaller.HelperTool
    I.e. 2 of the Quadrosync processes were killed, but not the third, the com.quadro.QuadroInstaller.HelperTool
    This may be left from installation.  Be that as it may, it should no longer be running.
        $ bash $>  uname -a 
       Darwin SYSTEM-NAME 14.5.0 Darwin Kernel Version 14.5.0: Tue Sep  1 21:23:09 PDT 2015; root:xnu-2782.50.1~1/RELEASE_X86_64 x86_64
        $ bash $>

    Problem: Immense Frustration when Quadro hangs repeatedly; great happiness when Quadro works

    Yesterday, reading email from home, I was frustrated to point of despair by repeated hangs.  Often hangs seem to be die to error handling - Mail.app dialogs about what appear to be Exchange server errors.

    Was about to give in.  Leading to shopping for other Android apps, XKey physical keypads.

    Today, reading email from work on a Sunday, nobody else in office, Quadro worked well enough.

    It would be sad if Quadro Mail.app not reliably usable from home.   But useful if could consistently use from work.  Unfortunately, on other days Quadro has died even at work, so such simple "it works here" do not quite apply.

    But even if I can only use something like Quadro once a week, only on a Sunday when network is unloaded, would be useful enough.  I just need to learn when it can be reliably used, and when it should be avoided.

    Problem: Partial ls of Quadro crash report logs:
    /Users/glew/Library/Logs/CrashReporter/MobileDevice/AG6i/Quadro_2016-01-05-131041_AG6i.crash
    /Users/glew/Library/Logs/CrashReporter/MobileDevice/AG6i/Quadro_2016-01-05-132416_AG6i.crash
    But these logs do not seem to include all of the hangs, when I have been able to "cleanly" exit/restart.

    Problem: Must constantly reenter (changing) IP address 

    As I move from home to work, from net to net, I must constantly determine and reenter IP addres.

    (Usual problem with pairing - yet another reason why BlueTooth, with an existing pairing solution, might be a nice way to go.)

    Good: works across tether / Personal Hotspot 

    If my iPhone and Mac had two wifi connections, one for such local Wifi based device connections, and the other for talking to the outside world, this might allow me to not have reenter TCP/IP address so often.

    Works when Mac is wired - still can use Personal Hotspot wifi.

    But not when Mac needs wifi to connect to net, when I don't want Mac to send everything across phone.

    Problem: Often "bounce" when hitting a group, also hitting button at same position in group entered 

    I use a Quadro "group" as an escape to a collection of pads that do not fit on the main triage screen for my personal/work email triage palette.

    Unfortunately, often when I hit the group button/pad, I end up accidentally hitting the button that happens to be at the same position on the screen in the new group,   Which means that the incorrect action is often performed.

    Thank goodness for the "Undo" button!!!  (Which I quickly learned to place on every frequently used Quadro palette/screen.

    If this were a keyboard, I would say that the problem was "key bounce".

    Key bounce should not apply to a touchpad.  But it is almost as if the Quadro is asking to read the touch position on entry - or if somehow that reprogrammed keys are being reread, which a new "press down" event.

    If I am very careful and quick, I can quickly tap and release and not get the extra key.  But if I press at a moderate speed, this "bounce" occurs.

    I do not have this sort of problem with other apps.

    Good: Quadro worked well today, no problems: Jan 10 
    I am going to start recording when Quadro works both with and without problems.

    Today, at the office, no major problems.  The key bouncing issue on the "More..." button.  But no crashes, no hangs.   If it always works like this, I love Quadro.

    Problem: Quadro timeout when client and targdet on different nets - no way to sto, had to wait until timeout 

    This morning connecting at home.  My iPhone, the Quadro client, and my MacBook, the Quadro target, got onto different wifi networks.  Since using non-routable IPs, it is only to be expected that they could not connect:  my complaint is that I had to wait until the connection attempt timed out.  None of the usual ways to cancel a pending connection - hitting the back arrow, etc. - worked.  I was about to doubleclick and kill when it timed out, and I could reconnect properly.

    This raises a bigger issue: normally I have my phone and laptop on different wifi nets, with no peer-to-peer.  I can change this at my home - I have done so - but not everyone will be able to do this.

    For example, at work my company laptop will be on a wifi SSN XXX-Staff, while my phone will be on wifi net XXX-Staff-BYOD, or XXX-Guest.   There will be no peer to peer.  I think this means that Quadro will be useless in this environment, unless I set up a tunnel, traversing thew net out and back in.

    This has not bitten me at work yet, because my laptop has been using a wired ethernet, and has also been able to use personal hotspot tethering.  But it will hurt me when I travel to other offices in the next week.

    I do have the option of using personal hotspot / wifi tethering, and sending all of my laptop traffic through my phone's personal dataplan.  I am not enthusiastic about that. Plus, will require VPN even when I don't otherwise need it.

    Many similar scenarios can be imagined.

    In particular, if Quadro-like software is ever on an Internet-of-Things device, not a full iPhone - well, nobody I know who cares about security is going to want to allow IoT devices onto the same wifi segments as the machines that actually matter. (Of course, an extended input device with Quadro arguably really does matter, and possibly should be on the secure network.)

    This is one of the advantages of using BlueTooth - even if the Quadro "extended input device" tunnels TCP/IP across BT, the fact remains that computer systems are set up for device-to-device Bluetooth, while having wifi and other networking going elsewhere.

    Problem: want more 'Context' than simply app name

    Quadro seems to use the app name as the context.

    I would like option of using the full window title.   E.g,. I use Quadro Aplle Mail.app for both personal and work.  So I want to use a work palette/sheet for Quadro when window title is 'Inbox - COMPANY 76 messages', and personal when window title is 'Inbox - Google 453 messages'.  Regexp?

    I know this can be done. RescueTime does it.  On Windows, AutoHotKey does.

    Unexpected Benefit: RescueTime less confuses
    An unintended side effect of Quadro is that, since Quadro's client is running on my iPhone, my RescueTime time tracking is more accurate.  Whereas, when I do the equivalent of Quadro using a tool running on my PC, such as AutoHotKey, RescueTime cannot always attribute the time correctly.


    Thursday, January 07, 2016

    Exchange 2010 SP1 Limits the Number of ActiveSync Devices You Can Synchronize | MobilityDojo.net

    Exchange 2010 SP1 Limits the Number of ActiveSync Devices You Can Synchronize | MobilityDojo.net:



    'via Blog this'



    It ain't just "numbr of devices".  I don;t have 10 phones, and I have not tried and discarded phones multiple times.



    I "only" have the following devices:



    a) my phone

    b) one or two iPads



    plus one or two laptops and PCs.



    No, for me the real problem is not the number of real devices, but the number of different mailreading SW apps I use.



    On each of the phones and tablets, I nearly always use two mail reading apps - Microsoft's Outlook.app, and the Zero email app.



    Plus I often have several more emails.



    Some of these apps share access via Apple's and/or Googles shared accounts.



    But some of these apps - often the most innovative - have their own iMAP, rtc., access.







    Mutiple apps per device, not sharing authentication => overflow of permitted apps.

    Scripting: Raw I/O Macros, versus Application Level, versus System Level

    The techwars visual basic versus applescript link ids not very good, but is an example.



    My own blog entry http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html is a much larger example.

    Related Blog Pages

    The original page, http://blog.andy.glew.ca/2016/01/dynamic-keypad-user-interface-elements.html, is more of a shopping project - a page where I take notes, a sort of informal review.  Posted in the hope that others may find useful.

    It soon became obvious that I was very hopeful about Quadro, a particular iPhone remote control facility, with multiple buttons, whose main claim to fame seems to be using AppleScript/OSA commands. I was hopeful this might make it more reliable than the many, many, macro facilities I have used in the past.  Perhaps; but Quadero still has reliability problems. Split into a separate page,
    Quadro - I love it when it works, but it is often unreliable.

    http://blog.andy.glew.ca/2016/01/scripting-raw-io-macros-versus.html is a somewhat generic discussion of the pros and cons of the various levels of abstraction found in such tools.  One of my big frustrations with Quadro, second to reliability, is that I cannot drive all user interface actions because Quadro does not seem to provide all raw keyboard and mouse events.

    http://blog.andy.glew.ca/2016/01/notes-and-thoughts-about-dynamic-keypad.html for still more generic thought.

    For the umpteenth time, realized that I should have written this in wiki rather than blog.   Googlesb blogspot/blogger blog really does not provide much support for pages that need to evolve by being split and otherwise refactored.  Let alone transclusion.


    Macros, scripting, etc.



    It is good to be able to automate repetitive or hard to follow tasks.  Workflows. This is often called scripting.

    There are at least three different types of scripting:
    • Raw I/O scripting - often called macros
      • Sometimes limited to keyboard, keystroke, macros
      • Sometimes able to generate all user input events, such as mouse clicks, mouse motion, etc.
      • Examples
        • AutoHotkey on Windows
        • Karabiner on Mac - although mainly a keyboard remapping utility, it can generate sequences of actions.
        • EMACS keyboard macros (which can generate mouse clicks)
      • Essentially, maps user input events to raw-ish user input events passed on.
      • Often unidirectional - often only limited ability to see what the application(s) or system have done.
    • Application Level Scripting
      • Examples
        • EMACS elisp
        • Microsoft Visual Basic for Applications - in many MS apps, such as Word/Excel/Office/...
          • Adobe ExtendScript - a Javascript language - in many Adobe apps, like FrameMaker, Illustrator
      • Applications often have scripting languages embedded within them - of varying strengths
        • Sometimes simplistic and of limited ability
          • But sometimes a "real programming language" (elisp, VBA, ExtendScript).
        • Sometimes only a subset of app operations
          • Sometimes able to do ANYTHING the app can do
            • e.g eLisp, where pretty much everything in the app is written in eLisp, apart from low level C functionality
    • System Level Scripting
      • Example:
        • AppleScript
          • Comprehensive framework - assuming applications comply
          • Extendable - applications can export a command vocabulary.  Which is typically an interactive menu hierarchy
          • Requires a messaging / IPC event framework
        • UNIX shells (bash, etc.)
          • More limited abilities than AppleScript
          • Extensible - but only if apps actually provide functionality on command line.
          • Mainly oriented to UNIX like "command = process spawn/execute/exit"
            • But some apps go to the effort of creating special commands that can interact with a running app.  (E.g. emacs-server)
    Which is better?   The usual religious wars.  Why do programmers like such useless religious wars, instead of fair and balanced inquiry and evaluation?
         IMHO no single "better" - they each have different abilities, pros and cons.  I wish that scripting like this could work better.

    Application scripting and system scripting tends more to generate "command" actions.   They can map user created scripts to commands already provided - encapsulation.  And they often can connect user input events - keystrokes, mouse clicks, etc. - to these user created scripts.

    Raw I/O scripting more often maps user entered input evens, like mouse clicks and keystrokes, to the same type of user input events that are actually presented to the application.

    MacOS keyboard shortcuts are in-between and limited: they map user input events (a very constrained set of keystrokes) to commands.  Commands apparently as provided by AppleScript and OSA.
         Similarly, Cocoa keyboard defaults map a larger input set to Cocoa commands - which are neither full system commands nor application commands, but commands common to Cocoa input boxes.




    User input events can be
    • keystrokes, w/wo modifiers
    • mouse clicks, w/wo modifiers
    • mouse/pointer movement
      • gestures
      • handwriting recognition
    • speech utterances


    Mapping of actual user input events can be
    • simplistic - event X gets mapped to event or command C, wherever
    • context based - event X gets mapped to event Y or command C, but only in a certain window or application
    • sequential - a sequence of events X1, X2, X3 can be recognized, and mapped to event Y or command C
    Ordering is a classic problem with mappings that involve sequences of user input events:
    • e.g. I used to regularly say something by voice to Dragon, then type a key
      • and have the key action occur before the voice command, that I assumed had been performed first.
    • it used to be common to see such ordering problems between mouse events and keyboard events, or between different keyboards (e.g. a regular keyboard and a separate numeric pad (I have a mouse that is also a numeric pad))
    • similarly, if different keystrokes get routed to different event queues.
    Timestamping all user inputs would be a global solution to such ordering problems - assuming there is a way of providing consistent timestamps to multiple devices.

    • such problems are less common nowadays, although I think it is mainly due to faster CPUs, and a reduced repertoire of devices
    • I expect to see such problems crop up again if I use Quadro or keypad - i.e. if I use an I/O device that is across a network on a separate computer (smartphone), at the same time as I am actually using the PC keyboard and mouse.
    Similarly ordering issues happen with commands:
    • Send a command to application #1, whether by raw IO macro or by a command interface
    • Then send another command, that assumes the the first command s completed.
    Oftentimes "send a command" is asynchronous - the application command is sent, but there is no standard way to wait for completion.
         Scripting often inserts arbitrary delays - "delay 1 second" to give the first command a chance to complete.
         Obvious problems: (1) can produce really slow scripting solutions because of unnecessary delays, and (2) can break, if the delays are insufficient.
         E.g. I have the Parallels virtual machine installed on my MacBook, and can script using Windows apps.  If the VM is up, the commands run fairly quickly; but if the VM is not up, then a delay of minutes (yes! :-() may be required.

    Needed: scripting systems with the ability to see if a command has been completed.

    Expect was a great step forward in automating such scripts for programs that use a text/terminal interface.   But AFAIK there is no common standard for GUI programs.  

    AppleScript's approach is to assume that scripting is not GUO based.  Often true, but loses much of the advantage of scripting.  (E.g. I want to use Quadro to create an alternative UI, more friendly to physical (dis)abilities.)
        TBD: link to the AppleScript doc that says "Don't script interactive".

    Opportunity: Standardization of "I'm done now".  Rather like a SYNC or MFENCE instruction, except for scripting - whether for Raw I/O, or command driven. Problem: requires a standard channel back from app to whom commands are being sent, to controlling apps.

    Scripting is often blind, unidirectional: send a command, whether raw I/O or command.  Or a sequence of commands.  Assume that they all work.
            Problem: if one command in a sequence des not work, then they remaining commands may do something completely unexpected.

    Opportunity: Transactional Command Sequences: give scripting user the ability to say "either do all of these commands, or none of them".  Or, at least, to stop at the first error, and inform the user.

    I have said many times to the Transactional Memory people, like Ravi Rajwar and Jim Larus, that the true value of transactions may not so much be in improving performance by parallelism, but in making better error handling possible.
        I have often suggested that filesystem level transactions be exposed to scripting languages like Perl.

    Application level scripting often has powerful error handling.
         AppleScript system level scripting has error handling - "try".   But experience shows that many script programmers do not properly use it, and/or the applications that interface to them export commands, but not error handling.
          Raw I/O macro scripting often has almost no error handling ability.

    Exceptions (events) are the best error handling system.  Not return codes / I/O that must be greped for.

    Command level scripting systems are often associating with ADA (American Disabilities' Act) mandated support for disabled users - blind, etc.
         With the consequence that such "Accessibility" features are often used by speech recognition or handwriting recognition software.


    Security:   the security model for scripting is poor. Opportunity for improvement.
          Demand for third party keyboards on cellphones drive some innovations in security:   e.g. OS enforcement that some third party keyboards can be used for most things, but not for passwords.
         Similar, MacOS allows accessibility controls on an app by app basis.
         Any scripting system that involves interaction is really a huge security vulnerability.  I want such improved input systems, but need a security model.

    But really need finer grain control.

    At least on an app by app basis - e.g if I am using a Quadro style keypad, I want to be able to restrict the commands and I/O events it generates to only certain apps (in my case, email), not others.
         And within such apps, to only certain commands (Archive, Refile) but not others (NOT scan all email looking for contacts or password reset email).

    I may have a one track mind with security, but I think this amounts to fine grain capability based security.  And, as I have said elsewhere, I think that capabilities need to almost be at the level of individual methods and objects in the target app. By default, so that programmer doesn't need to work too hard to create.  But then need ways of bundling such capabilities together.

    But the requirements of scripting I think have a further corollary: such capabilities need to be attachable to the commands and raw I/O input events generated by the scripting facility. Rather like taints/poison/DIFT.

    TBD: more thoughts.


    Above, I started by talking about (1) Raw I/O, aka macros, versus (2) Application, versus (3) System.

    An intersecting but orthogonal distinction is: what the output looks like:

    • Raw User Input Events
      • once again, keyboard events, mouse events, etc.
    • Menu Commands
    • Fully Capable Commands
    Emacs elisp is my category example of fully capable commands: its a full language.

    AppleScript / OSA is capable of being a full command language.  It has most of the features of a real programming language, albeit with strange syntax.

    But, de-facto, many applications that provide an OSA interface to AppleScript and friends really are just providing a traversal of the menus that they provide on the user interface.   This is facilitated because some programming systems support doing so automatically. I.e. it is easy to code.

    Moreover, you don't have to think that much about security and consistency, because you know that, since these commands are already made available to the user, they will maintain whatever datastructure consistency constraints you wish to maintain.  Whereas exporting low level commands, as is done with Emacs Elisp, and to a lesser extent with VBA, might allow a script (or an alternate user interface) to violate such constraints.

    The problem with macro scripting emitting Raw User Input events, keyboard shortcuts and mouse clicks, especially those that navigate a WIMPY interfacer such as a menu tree is that MENUS CHANGE.   So, if you have to emit three down arrow keypresses to get to the menu item that you want to select, you will be out of luck if the next version of the software changes the menu.   Or if the current version of the software allows menu items to be greyed out and skipped over.

    Menu interfaces that allow menu items to be selected by typing the first few letters of their name are better, slightly more robust - but, again, menus change.

    Menu interfaces that allow menu items to be selected by typing the full menu item name are more robust - but, again, menus change.  A common change is a "split", two menu items corresponding to an original, or a submenu where there was originally only only menu item.

    The problem with scripting by navigating any menu hierarchy, whether by raw events of by full menu item names, is that MENUS CHANGE.  The hierarchy can change.   Command may be moved completely elsewhere.  A menu of a small number of settings may be changed into a dialog box, and vice versa.

    It seems that some scripting systems have "global menu item names" or "menu commands".  For example, Mac OS-X keyboard shortcuts associate a "Menu Title" (Enter the exact name of the menu command you want to add) with a keyboard shortcut (keystroke plus modifier events).   But, again, the problem with this is that MENUS CHANGE.   It is otherwise reasonable to have two menu items, in different parts of the menu tree, that display the same text in the menu, but which map to different commands - but this Mac OS-X keyboard shortcut facility prevents that.

    Well, if making the menu title that is displayed to the user unique is a bad way of identifying the macro-scriptable command, and if making the path through the menu hierarchy is similarly a bad idea (because menu paths change), what then?
         This is what I mean by a Full Commands.   
         Like in Emacs, where there is a distinction between command executed, and menu item displayed that invokes the command.  Ditto keyboard and mouse bindings. Speech, etc.
         If desired, one can imagine flyovers or queries on a menu system that would display what the current binding is, providing discoverability.

    Oftentimes, there is a further interface layer:   the actual commands bound to menu items, keystrokes, mouse events, speech and gesture commands, etc., may have input medium specific interface code, that does stuff like extracting a keyboard prefix argument, or which mouse key.  Or, prompting for further input.  And which then invokes an actual, scriptable, command.

    Finally, we must of course distinguish between the Full Commands provided to a scripting interface, and internal commands and functions.  For the purposes of maintaining consistency in internal datastructures.

    But, as usual: although it is important to be able to create such distinctions, it is also important to be able to provide nice defaults. 
    • e.g. automatically create "Global Menu Commands" from menu item titles
      • warning when no (no longer) globally unique
    • e.g. to permute full menu path navigation using menu item names
      • again, warning if menu structure changes so that a path is no longer accurate
    • e.g. to provide raw input event navigation to scripting
      • warning...
      • and otherwise provide facilities to handle errors
    • In addition to a Fully Capable scripting interface
      • derived from the internal function names, if you are okay by that.
    IDEA:  just like Emacs has an (interactive) declaration in defun functions, perhaps there should also be a (scriptable) declaration,.

    THINK ABOUTL: Elisp (interactive) really is just a shorthand, an easy way to create an interface function.
















    Tuesday, January 05, 2016

    Dynamic Keypad / User Interface Elements (Quadro, Keypad)

    BRIEF

    When I started I expected this to be a review of several user interface packages, beginning with Keypad.   But I fell in love with the second package I tried, Quadro, so it has turned into a set of notes, mainly about problems, with Quadro.

    Let me be clear:

    Quadro (and Keypad, and several other apps) are "dynamic UI touchpads".  Software apps running on your touchpad cellphone or tablet, where you can define macros to interact with apps, typically on another device like your Apple Mac or Windows PC.

    Quadro runs on an iPhone or iPad, and controls a Mac or PC.  Closed source, questionable security.

    But... I love Quadro.   I want Quadro, or tools like it, to be popular, available everywhere, and successful - because I believe that tools like Quadro will help make me more successful in my work.

    I like Quadro enough that I very quickly paid 50$ to support its development. Quadro is freemium, free to most users - but I believe in supporting stuff that I like.

    I like Quadro enough that I am considering buying a used iPad so that I can get more surface area for buttons.

    Quadro has problems, some noted here.   Some of these problems are severe enough that at the time I am writing (Jan 8, 2016), I would not recommend Quadro to most users.  But if these problems are fixed, if Quadro works reliably, I would recommend it to almost anyone who uses the computer for more than a few hours a day.



    DETAIL

    ISO of better User Interfaces

    Not just "apps written by the original author that have better user intefaces" - although that is good.  Here I am specifically talking about tools to give myself a better user interface for an existing app written by someone else.

    I have long been an advocate of "single command key big button email reading" (se notes at bottom). I have mashed up macros and menus and buttons to do this in many systems: AutoHotKey, various web tools.  Custom keyboard shortcuts. Etc.  Emacs, of course.

    Recently, over Xmas, sharpening this particular saw: started off by starting to use Karabiner on MacOS (which allowed me to create keyboard shortcuts bound to spacebar, the only "big button" that is commonly available).  Frustrated because I need more than one big button, I started googling for physical buttons that I might be able to bind actions to, which led me to https://flic.io/.  Looks neat; if I had seen this during the funding phase, I would have signed up; but the current flick button looks too thick for my application: I would like to have a few such buttons stuck to my laptop, next to the trackpad.  Thin enough so that the laptop would close. Besides, I really wish that it had electric paper display surface so that I could display its current binding.

    This line of thought led me to thin about an app or apps that would run on my iPhone (or one of my old Android devices), touchscreen key shapes, that would send commands, shortcuts, to my laptop, or to other apps on the iPhone.   User definable, so that I could always have a custom pad of the 4 or so commands that I am using most often.

    Searching for things like "Android remote keyboard for laptop" and "iPad touch control of PC" led to first the Keypad app for iPhone http://keypad.iosproapps.com/, and then to Quadro  http://quadro.me/

    Actually, to many, many, remote keyboard, mouse, keypad, joystick and other apps.  Keypad and Quadro are just the first two that I have started to look at.

    Briefly: (more detail in separate sections)

    Keypad easy to use.  VNC based.   Does not work with encrypted VNC - 8 character passwords. Gah!!!!  But easy to use.   Got something working in a few hours.  And then ran into problems like lack of version control, GUI only, no text file representation.

    Looking for something better: of the many alternatives, Quadro seemed to have the biggest aspirations.   Security unknown.  Handshakes, interacts, with apps, not just macros sent blindly across. Version control?  Text files?  Harder to set up than Keypad.  Nevertheless, has aspirations, expressed in incoherent Italian inspired English: putting my money where my mouth is, I am about to pay 50$ to support further development.

    This blog entry to record notes about Keypad, Quadro, and the overall field of such apps.   Notes about Quadro to collect and hopefully suggest to developer; notes overall because I want such better user interfaces.   I'm a CPU designer, but it's all about the user interface, baby!!!


    (Gosh, I wish that blogger had a TOC.  Need to stop using this.)

    Keypad App

    http://keypad.iosproapps.com

    Very easy to set up a custom keypad.  Nice collection of buttons, widets.  Swipe pad. For that matter, allows full touch control.

    The usual problem of macro shortcut bindings: lack of interaction.  What happens if a binding is pressed, but not appropriate for the current app? Etc.

    VNC based.  Recommends non-encrypted VNC, only 8 characters of password.  Gah!

    Security offputting.  My last straw was realizing that there was no easy way to diff configurations.  No text file.   They do allow you to save and email config fules to yourself, but the format is binary.

    So I kept looking.

    Quadro

    http://quadro.me/

    Googling kept turning up Quadro.

    Eventually realized that I had written enough about Quadro to warrant aa page of its own: http://blog.andy.glew.ca/2016/01/quadro-i-love-it-when-it-works-but-it.html

    Currently, Jam 10, 2016, after a week of using Quadro, 
    • I am still hopeful that Quadro will be a good tool
    • I am frustrated, driven to exasperation, by how unreliable and flakey Quadro is.
      • Almost par for the course with this sort of application, although I suspect that Quadro, because of its command rather than keyboard orientation, hangs more than most more "Raw I/O" based tools in this class.
      • I was about to give up using Quadro today, but I am figuring out that it is more reliable in some environments than others.
    • I can't say that I regret contributing 50$ to support Quadro development, the maximum freemium they provide.  But I sure wish that it was more reliable.
    • I remain concerned about security, lack of.

    After trying Quadro

    As mentioned above and elsewhere, I became quite exited about Quadro, hopeful that the Quadro developer had improved the state of the art in such applications, which often have problems wrt error handling.   However, Quadro drive me almost to tears, being *almost" good enough - good enough for much of my needs - but failing, hanging, crashing a slot.  So I have kept looking.

    Android "Remote Macro Keys"

    I have not yet installed to try.  Notes from the Google Play store and googling.

    "Very light"

    Java server on PC.

    Sounds as if mainly macro based.

    No mention of Mac.

    No mention of non-I/O macro, OSA/AppleScript or VBAS or ...

    Limited ads.

    Sounds "sincere", but the usual worries.

    Security: UNKNOWN.

    Open Source: UNKNOWN.  Have not verified whether the Java server is bytecodes or source.

    Config file: ??
    Diffable: ??
    Version controllable: ??
    Text config file: ??

    I am not sure of whether "Remote Macro Keys" has a versionable-diffable-text config file.  



    I do know that just before I found Remote Macro Keys, I found a similar Android project, macro keys or the like, that describved itself as Open Source and having an XML config file - but I am not sure if it was local macros, or remote.

    It is rather typical for Android products to be more open source, more likely to use text config files, in "the UNIX way".



    Android "Mouse and Keyboard Remote"

    More Android SW.  PC Server.  Speech?


    DIY physical keypad 

    http://www.instructables.com/id/Making-a-powerful-programmable-keypad-for-less-tha/

    These guys reminded me about xkeys.



    X Keys 

    http://xkeys.com

    This was the sort of physical device that I was originally looking for, when I decided to go search for programmable

    Reputation: good: I have used XKeys products before, mainly the Y-mouse. Never bought the keypads before, but vaguely remembered them.

    Keypad flavors

    • XK-24 keys, 4 horizontal x 6 vertical
    • XK-80, 10 horizontal by 8 verrtical
    • XK-128, 16 horizontal by 8 vertical
    • sticks: XK-4, 8, 16 - that many keys in a 1 high row, nice to put at top edge of keyboard.
    All keys same size (I remember seeing similar products with keys of different sizes, possible addons)

    Modern devices all USB.  Macros are programmable using PC/Mac software; once programmed, retains state, so that can use on other devices that accept USB input.

    Keytops are NOT dynamically changeable, the way something like Quadro or "Remote Macro Keys" is.



    LCD Keytops - E3-Keys

    Finally Google coughed up a device that has physical keys with LCD, dynaimic, keytops:

    http://www.e3-keys.com/english/index.htm

    I knew it was out therre.   ICAWTK, googling "LCD keytops" found it (too many false hits for "LCD keypad", etc.)

    Nice: they have history going back to 1978 in an IBM Technical Disclosure Bulletin. http://www.e3-keys.com/english/history.htm.  Apparent a German company LcBloc Computer Technologie GmbH  1984.

    (Mouse Parker between these two.)

    Doesn't look consumer ready.   Doesn't look cheap.   But may be able to find somebody else.

    Of course, why would I want LCD keys, if I can get a touchpad like Quadro working?   Only if one does USB and works everywhere, and one does not.  Making an IOS or Android device into a HID doesn't seem to have been done.

    Optimus Keyboards

    Same query as found e3-keys also found

    Optimus family - Art. Lebedev Studio

    www.artlebedev.com/portfolio/optimus/

    and

    https://en.wikipedia.org/wiki/Optimus_Maximus_keyboard

    I have seen this before, and lusted after it.   But it appeared at the time to be vaporware, indeed receiving a Wired Magazine Vaporware award, 2007.  However, if wikipedia is to be believed, it is shipping now - not just full keyboard, but also smaller keypads.  The ArtLebedev website shows some as "sold out", the Popularis as shipping now, and several concepts.

    Reviews are mixed.

    http://www.artlebedev.com/ shows several more, such as the Tactus, no physical keys, just touch everywhere, or the Upravlator.  But "Expected to be available for sale in the second half of 2008" several with no updates in 7 years do not inspire confidence.

    Overall, http://www.artlebedev.com/ seems to be a design house, with many neat ideas.  But probably not so much a consumer computer products company.

    The keyboards are not for sale on their web.store. Well, maybe the Optimus Popularis is, at
    http://store.artlebedev.com/electronics/optimus-popularis/  - the website has twisty pasages that do not directly connect.  1500$US.  Ouch!

    Sonder Keyboard

    https://www.sonderdesign.com/

    e-Paper keytops.

    Bluetooth.

    Website with a pre-order button in Jan 2016, 200$, Pilot 2016Q2.


    Virtual Laser Keyboard

    There are several "virtual laser keyboards that project a pattern onto a flat surface, and sense where the user types.

    http://www.ctxtechnologies.com/products/vk-200-keyfob-virtual-keyboard/

    All that I know of provide a standard keyboard or numeric keypad.

    I wonder if any of them could be programmed dynamically?


    Some Personal History

    Single Command Key Big Button Processing

    of Email, and other Repetitive Stuff


    I have long been an advocate of "single command key big button email reading".

    But I oversimplify:

    + Not just single command: I often use multiple commands, e.g. the command I use most often in email, Archive, as well as a few others: Defer to handle later, file into one of 2-4 top projects, etc. Just not the full set of commands that the application provides.

    + Not just single key: although I prefer to use a single big key, like spacebar or caps lock spacebar for archiving,

    ++ sometimes I multiple key chords or sequences, and more than one of those: e.g. control+spacebar, and sometimes multiple keys - command+1 fir project 1, command+2 for project 2

    ++ sometimes I will use menus - especially menus that I can navigate by keystrokes as well as mouse

    ++ sometimes I bind the commands to the many buttons on my trackball (4 buttons, with various click combinations and modifiers)

    ++ sometimes I mash up controller pages, like webpages, with hot touch areas

    ++ Not just big keys:  as above, although if I am at a keyboard in an editor where keystrokes are hardwired, I may use small keys

    + Not just email: although email is my current hot button task, I have done things like this for repetitive tasks such as evaluating patents.  Whereas many folks were only able to read 100 patents a day, because of the slowness of clicking around to various information sources, I mashed up an interface (in Excel, last time), that traversed a list, automatically opened the information sources I needed, and which allowed me to give a score plus certain keywords - and was thereby able to process tens of thousands of patents.  So many that I cannot remember details of most of them (so don't accuse me).

    More Personal History - Devices

    I have a collection of I/O devices that I have used for this sort of task: gamer joysticks, keypads, gamer gauntlets that I could bind commands to ever finger motion of.  Foot pedals, etc.

    These separate from my collection of I/O devices for normal use.  I never expected to use these gamer button pads for day-to-day computer use.  But when I have 2-3 months of a repetitive task,  like reviewing patents or papers, I felt tat a bit of investment in custom hardware was warranted.

    I have long wanted a keyboard that had LCD or e-Paper tops, where the current bindings could be indicated.  e-Paper for long lasting.  LCD or other quickly changeable, e.g. for Emacs multi key bindings.  Type in ESC, and see symbols on the keytops for next actions.   I vividly remember a McGill meeting, circa 1984, with some vendor showing us a then state-of-the-art workstation, and Der Mouse Mike Parker flummoxing him by asking for LCD keytops.  I re,ember there was a Russian company promising such a keyboard, more than two decades later, but nothing came of it.

    Friday, January 01, 2016

    Metronome for Walking Pace/Cadence while listening to Podcast or Audiobooks

    Brief

    Looking for a metronome app to provide cadence while walking and listening to podcasts and audiobooks. iPhone can't, but Android can.

    FAIL: I felt all good about kuewlsoft's Metronome app - but on the first longish walk I learned that it turns off when the phone is locked.  Trying other Android apps. So I created the table at Metronome Comparison Spreadsheet.

    I thought that I was narrowing down to Rhythmic Metronome, Zenom Metronome, and B'Metronome - but after a week I switched back to kuewlsoft's Metronome app.  kuewlsoft's good metronome features - a tolerable click that does not offend my ears, the ability to specify emphasis clicks, volume controllable separately from the overall volume of the podcast - make up fr the problem of turning ff when I lock the phone.

    I cope with kuewlsoft's Metronome turning off if I lock the phone by not locking the phone - after all, it is not my real phone, just an old Android phone (because iPhone can't mix two sound sources). At first when I put it in my pocket I had problems with accidentally hitting screen buttons - but my low-tech solution to that is to have a stuff piece of cardboard that covers the screen.

    I can live with kuewlsoft Metronome, but I would still like to find a better Metronome app.  Must-have wants: simultaneous, works when locked, sounds nice, emphasis, 3:4 time, rates 100-160 bpm.  Wanna haves: controllable volume and rate from my headset.  In my dreams: a "smart" metronome that adjusts to encourage me to be faster, even if I slow down climbing a hill.

    Haven't looked at pay apps. Some of the free but $ upgrade app$ look interesting, but I don't have time to search all.  Looking for software sucks.

    Walking

    Walking is my main form of exercise. (One hour+ per day, 3+ miles per day; because I have a dog; because my knees are shot from too much hill running and soccer stomping in my youth; because I live next to many miles of beautiful trails in Portland's Forest Park.  I also swim and climb, but I do not live next to a swimming pool or a climbing wall.)

    I love walking in quiet wilderness - but in a city like Portland, the woods are often not so quiet.   Plus, 2 hours a day really impacts other things, like continuing education. So I often walk while listening to podcasts (NPR news, computer security), or audiobooks.

    Plus: when I do exercise in a gym (elliptical, treadmill, rowing), I was bored out of my skull until I stated listening to podcasts.

    Problem: listening to human conversational voices, in news, podcasts, audiobooks or lectures, leads to a moderate, sauntering, walking/jogging/running pace.  Too slow a rate of exercise.

    Now, folks who listen to music while exercising often choose songs with a target beats per minute rate.   But, I prefer learning to listening to music.

    I have tried podcast players that compress, speeding up.   Unsatisfactory.


    ISO Metronome for Exercise Cadence

    I decided to try having a metronome tick or ... at my target cadence and gait, while listening to my podcasts.  How hard can that be?

    I tried running such a periodic metronome on my Pebble watch, buzzing 100-120 times per second.   Drains the battery much too fast.

    So, instead let's put the metronome on my phone. iPhone - I wish.

    There are many, many, metronome apps for iPhone or Android.  Most for music, some specialized for running or hiking.

    Annoyingly, apparently iPhone iOS is incapable of having both a podcast app and a metronome app running at the same time, and combining (mixing) their audio output.   Start the metronome playing, then start the podcast, and the metronome app stops, and vice versa.

    Annoying but not necessarily surprising, given what I know about iOS.  Android is more of a real, multitasking OS, so is much more likely to be able to play sound from two apps simultaneously.

    Gives me a reason to revive one of my old Android phones, to use as a dedicated podcast player and metronome.   Especially since I have not found any iPhone podcast player that is as useful as BeyondPod on Android.  Lossage:  my iPhone won't be able to act as a phone on my walks - but most of my walks in Portland's hills have no or little cell phone contact.   Actually, the biggest lossage is not so much the cell phone functionality, as having another device to have to keep charged, and having to pair my bluetooth headsets between Android and iPhone.

    Note: I do not want to do my own audio mixing.  I might if I were playing music, but I tend to listen to podcasts or audiobooks once and then never again.  Plus I want to be able to dynamically adjust.

    Note: real audio mixing, combining output from two apps, might be most desirable.   Require separate volume control.  Might be nice to be able to overlay arbitrary music on my podcasts and audiobooks.  But, metronomes are a special case, and may not require full mixing.


    What I have found and am using

    Anyway: 6-12 hors of research, and evaluation of Android apps.  Here is what I have ended up using:

    FAIL: can't find a way to play both a metronome and Podcast/Audiobook on iPhone

    FAIL: metronome app on Pebble watch causes battery to drain too quickly. (If battery life were not a problem, I think that I would very much like the Pebble watch as a pacemaker.)

    WIN: old Android phone, with the "Metronome" app by keuwlsoft, with BeyondPod and Audible audiobooks.

    Metronome by Keuwlsoft:
    • no ads
    • no permissions
    • adjustable beats: major, minor, muting, subbeats
    • adjustable rate beats per minute
    • 3 sound effects
    For my purpose, the visual effects are useless - the phone is in my pocket, the sound plays in my earphones.

    This is what I homed in on for my current use.

    FAIL: I felt all good about keuwlsoft's Metronome app - but on the first longish walk I learned that it turns off when the phone is locked.  Certain other Android apps do not. :-) See Metronome Comparison Spreadsheet. But after a month, I am still using keuwlsoft Metronome, with a kluge (a stiff piece of cardboard to cover the screen)  to cope with keuwlsoft turning off if I lock the screen.

    Criteria for Exercise Metronome
    • Simultaneous metronome and other audio source
      • MUST HAVE
    • Ability to adjust metronome rate.  
      • NICE: dynamically.   
      • NICER: dynamically, hands free. 
        • I don't know any app that does this.
    • Separate volume control
      • not quite must have, but pretty darn close
    • Separate ticks in each ear, for left/right foot
      • NICE?  helps keeping pace
    • Metronome sound
      • Many metronome apps have a single, hardwired, beep/click/chirp.
        • This may work for practicing piano
        • But it is not good if this chirp makes it impossible to hear the podcast or audiobook
      • VERY NICE, almost ESSENTIAL: ability to select a more pleasant sound
        • Possibly nice if you have a wide selection, or can provide your own
        • However, the metronome app I chose, keuwlsoft, had only 3 hardwired choices - 2 of which were acceptable.
    • Beat patterns
      • at first I thought that only a fixed beat pattern would be sufficient.
        • Q: tick every foot, or only on left foot?
      • but I quickly learned that a pattern of major/minor beats is helpful
        • NOT 1/2 time - not major every left, minor every right - seems to cause asymmetric gait
        • BETTER 3/4 waltz time: TICK-tock-tock-TICK-tock-tock - so that major beat alternates feet.
          • 3/4 time is a bit of a misnomer - the 1/4 is irrelevant.   It is really just a cycle of 3 beats, with different ephasis.
          • After a few days I figured that TICK-TICK-tock works better that TICK-tock-tock.  Go figure.

    Ads and permissions: 
    • many metronome apps (and apps in general) require excessive permissions and serve obnoxious ads
    • it may be reasonable to allow a metronome app permission to
      • write files (to save carefully crafted rhythms, e.g. if you are a drummer)
      • read files - in particular, sound clips that you want to be use for the ticking
      • automatically mute when the phone rings
    • but I chose the keuwlsoft Metronome app that required no such permissions
    Advanced Features for an "Exercise Cadence Metronome"
    • Accelerando
      • if you are trying to speed up, as I am, it might be nice to start at a first rate, say 100bpm, and smoothly accelerate to 120bpm over the space of a half hour or so
      • some musician oriented metronomes, such as the keuwlsoft Metronome app I chose have such an accelerando feature, but the timescales are not so appropriate to exercise
      • I am doing "long term gradual accelerando", trying to add 1 step per minute to my pace - every so often.
    • Rate programming
      • ramp up/down
      • intervals
    • Dynamic Rate Feedback
      • Some running oriented metronomes can detect your running speed, and compare it to your desired goal.  
      • I can imagine real dynamic feedback
        • trying to gradually accelerate to desired goal
        • with warmup and cooldown
        • possibly with adjustment according to incline
      • Feedback with heart rate and/or landspeed
      • After use, my main hope for feedback is
        • Detect the rate I am stepping at, and time clicks to try to encourage me to be just a bit faster
        • It may not be appropriate to just click at 128 spm when I am walking at 127 spm
        • I think that it might be better to dynamicaly adjust by phase: e.g. to advance a tick by a small amount - 1ms? - perjaps non-uniformly through my step cycle.  Not increasing rate before it looks likely that I can match it.
      • I would live to have dynamic cadence adjustment that
        • adjusts automatically if I slow down to walk up a hill, or other
    I have noticed that my cadence varies during my walks, according to conditions: hill, hard vs soft surface, fatigue. I can walk faster in the morning than just before bed.
         It is a bit of a pain to adjust this, since it is a pain to have to take my phone out of my pocket.  I wish that I could adjust it from the headset that I wear to listen to the podcast and metronome, or possibly from my smartwatch.  


    Thoughts about Wearable Devices

    UI devices: phone (rectangular slab of glass with touchscreen); watch (wrist mounted display, with vibes, buttons and/or touch); headset (sound output, sound input, buttons and/or touch in several places.  Voice control with audible feedback, but that's hard to get right.)

    More and more I wish to avoid having to click on my phone.   It may be okay for the phone in my pocket to be my "personal server", connected to devices like headset and watch by wireless like BlueTooth.  The phone is big, can hold a big battery, and probably diffuse more heat.  The big screen is okay for interaction when I want to browse the web.  But I really wish that I could do more, control more, without the phone.

    I can control overall volume using my headset.  But I cannot control the volume of the metronome separately from the podcast - I have to pull out my phone to do that. :-(

    I can skip forward or back in podcast from my headset.   I can stop/start the podcast from my headset.   But I cannot styart/stop the metronome.

    I wish I could adjust the metronome rate from my headset.

    My headset is cheap.  It has essentially 5 controls: on/off, with long presses for extra control.  Volume up/down.  Select forward/back. All of these contyrols are on one ear.
         I would gladly be willing to pay a bit extra to get 5 more controls on the other ear.

    Or perhaps I should try to get these controls from my SmartWatch.   I.e. use watch and headset in conjunction.   The watch display gives a more dunamic selection of controls.   More irritating that just touching a button on my headset, but more variety, and less irritating than having to pull my phone out of my pocket, unlock it, and then interact.

    I have done limited amounts of Siri voice control handsfree from my headset, but Siri requires too much screen interaction.




    How many Steps per Minute?

       Note: writers differ as to whether you would walk at 60 steps per minute or 120.  The factor of two difference is due to counting steps by both feet or not.

      http://www.ontherunevents.com/ns0060.htm

      Heart rate more important than speed or steps per minute. Mix it up. Interval training.


      LEVEL 1: VERY INACTIVE: 80-100 steps per minute = 2 mph (30 minute mile)

      Some widely referenced studies talk about 100spm as moderate exercise.


      LEVEL 2: LIGHTLY ACTIVE: 120 steps per minute = 3 mph (20 minute mile)

      Weight loss: slow, 3 mph, 60-70% MHR (Maximum Heart Rate). Not aerobic. 1-2 hours a day. 120 steps per minute. 


      LEVEL 3: MODERATELY ACTIVE: 130 steps per minute = 3.5 mph (17-18 minute mile)


      LEVEL 4: ACTIVE: 140 steps per minute = 4 mph (15 minute mile)

      Aerobic: 75% MHR. 20-30 3x/week. 140 steps per minute. 


      LEVEL 5: VERY ACTIVE: 150 steps per minute = 4.3 mph (14 minute mile)

      Jogging, or race walking.


      LEVEL 6: EXCEPTIONALLY ACTIVE: 160 steps per minute = 4.6 mph (13 minute mile)

      Endurance/Fitness/Speed: 80-90% MHR. 160 steps per minute,


      LEVEL 7: ATHLETE: 170 steps per minute = 5 mph (12 minute mile) 

      Most walkers need at least a year to develop paces of 170 spm (12 minute mile) or faster. 


      LEVEL 8: ATHLETE : 180 steps per minute = 5.5 mph (11 minute mile)


      LEVEL 9: ATHLETE: 190 steps per minute = 6.0 mph (9-10 minute mile)


      Using CUA ^C/^X/^V as well as command-C/X/V on Mac

      I have been using my MacBook as my primary computer for a year now - and I still have trouble typing command-C/X/V for copy/cut/paste, rather than the Windows like control-C/X/V.

      More precisely, perhaps, the CUA, Common User Access, key bindings, that date back to Xerox PARC (and earlier, I think). http://en.wikipedia.org/wiki/IBM_Common_User_Access

      Oddly enough, I had managed to make the transition in my first few weeks or so, perhaps a month, on the Mac - up until I got a proper emacs working on my Mac. Now, I do NOT use ^C/^X/^V key bindings on Emacs.  But for some reason using Emacs pushed any new key bindings out of my cache, and I found that I started experiencing control/command confusion on Mac apps more often. Perhaps this happened because emacs uses SO MANY control key bindings - and I decided not to use any of the several ways of having emacs treat Mac command key as equivalent of control in standard emacs key bindings. (TBD: notes on this: briefly, doesn't always work).   So control is in the front of my mind when using emacs, even on a Mac.   Or perhaps this confusion became more prominent because when I am using emacs I work faster - so the errors waste a greater fraction of my time.

      Sometime over Xmas this became really frustrating, so I set out to fix it.

      Now, I do not want to globally swap command and control.  I want to use standard Mac key bindings as much as possible.  Also, globally swapping command and control messes up other apps, such as Windows RDP.   No, instead all I want to do is to take the keys that are problematic - typing ^C instead of command+C, etc., in my web browser - and remap only those keys.  While leaving the standard command+C/X/V bindings unaffected.

      Ideally, do this in only the apps where I am constantly making these errors - mainly, cutting and pasting between my web browser and emacs.   Or do it globally, but ideally without affecting the existing emacs keybindings - i.e. have ^C behave like command+C in my web browser, and in most other places, but not in emacs or any place where the standard Mac bindings do not apply.

      Here's a post along these lines:

      http://apple.stackexchange.com/questions/66970/remap-ctrlc-ctrlv-and-ctrl-x-only

      Here's a great webpage about Apple Cocoa key bindings:

      http://www.hcs.harvard.edu/~jrus/site/cocoa-text.html

      Surprisingly, Apple's Cocoa key binding mechanism is almost as powerful as EMACS'.  It permits nearly any key to be rebound, it allows emacs-style multiple key sequences to be bound, and it allows multiple bindings to the same command.

      Surprising or not, most Apple key binding tools do not provide anywhere near this control.  Why not?  I conjecture that this is a carry-over from NeXT, from the old days when Men were Men, and Power Users wanted real keyboard control.  But that it has fallen into disuse with the rise of mass market non-power-user computing, and "usability" gurus who deprecate functionality.

      Also, while the remapping facility is powerful, it only applies to Cocoa apps.  It is not ubiquitous.  Apple has a long history of "software model tunnel vision", reflected in security bugs such as the "We assume that everyone is using Objective C,and cannot forge message names".

      Anyway, here's what I ended up doing: placing the following in ~/Library/KeyBindings/DefaultKeyBinding.dict

      {
          "^a"        = "selectAll:";                     /* C-a  Select all */
          "^x"        = "cut:";                           /* C-x   Cut */
          "^c"        = "copy:";                          /* C-c          Copy */
          "^v"        = "paste:";                         /* C-v          Paste */
          "^z"        = "undo:";                          /* C-z          Undo */
          "^y"        = "redo:";                          /* C-y          Redo */ 
      }
      Now, this provides the mappings in all Cocoa apps. No ability to say "do this only in Chrome".  But it seems good enough for now.