Cupertino 1.19.0
Version 1.19.0 ·
Added
apple_desktop_hover, because a click is not a pointer. The desktop surface could press, click, type and read a tree, and still could not make a tooltip appear.clickposts a press and a release and nothing in between, so a tracking area sees a button go down inside it having never seen the pointer arrive — which means every control that only exists under the cursor was unreachable: a tooltip, a hover readout, SwiftUI'sonContinuousHover. Driving a chart's hover band is what found it, and the workaround was a throwawayCGEventscript outside the repo.Addressed by
handlein preference to a coordinate, and for a sharper reason than the other verbs have. A point fromui_treewas true when the walk ran; a window that has moved since — and one being driven moves often — leaves it aimed at bare desktop, where a hover misses in silence. There is no control to fail to press and nothing to report, just a readout that never appears. So the handle form re-reads the element's frame at call time. It also brings the target application forward first and refuses to post if it did not come, because a hover is delivered only to the frontmost application; the same reasoning the simulator surface already applies to a tap.It is the first verb here that is refused on account of the person at the keyboard. A hover takes the physical pointer for as long as it sweeps, which is worse than a click rather than better, so it declines while somebody is using the Mac. The check could not be
secondsSinceUserInput()alone: that reads.combinedSessionState, which counts Cupertino's own events, so a plain idle test would have refused every hover after the first — biting hardest when the agent is working alone, the case it exists to allow. The driving indicator already knows whether the last input was ours, and the idle floor sits under its linger so a sequence composes. The reading is returned with the answer, for the before/after comparisonapple_desktop_user_activitydescribes.Annotated non-destructive and idempotent, unlike
click: moving the pointer twice to the same place leaves the same state and presses nothing.The simulator surface introduces itself on connect. A client loads the
initializeresult'sinstructionswithout being asked to; the guide resource is pull-only and goes unread unless something names its uri, which is the wrong shape for the handful of facts that change a caller's very first move.InProcessRPC.dispatchtakes an optional instructions string and omits the key entirely when it is nil, so a surface that declares none answers byte for byte as it did before. Simulator declares a short one — only what changes that first move — and points at the guide for the rest.
Changed
The role-filter guidance was drawn from Catalyst apps alone. The advice about filtering
apple_desktop_find_elementsby role came from a sample where controls report asAXGenericElement,AXStaticTextandAXImagefar more often thanAXButton. That is true of Catalyst and was overstated as the norm. Re-measured across 21 regular apps: Catalyst still misses about 80% of them (Maps, Messages, Calendar, System Settings), while AppKit and plain SwiftUI apps miss only about a fifth, and widening the filter to the whole button family clears most of that. The rule does not change — the stragglers are exactly the clickable heading or row a caller cannot predict, and asking for pressable costs nothing — but the tool description anddocs/desktop.mdno longer claim the miss rate is typical.
Fixed
apple_desktop_activatewas refusing every target, and it was never a permission. Cupertino is anLSUIElementbroker nobody ever clicks, which since macOS 14 is on its own enough forNSRunningApplication.activate()to refuse — whatever the target, and with no grant that changes it. Measured on macOS 26.6.2: refused forcom.apple.mailandcom.apple.finderalike when called from Cupertino, while the sameactivate()on Mail from a freshly launched command-line process returned true and moved the foreground..activateIgnoringOtherAppsis not the way out either, having been deprecated and a no-op since macOS 14.When LaunchServices says no and the app holds Accessibility, the driver now sets
AXFrontmoston the target's application element instead — measured raising Mail from behind another app witherr=0. Either path is then checked against the window server's ownfrontmostBundleIdrather than the return value of the call that did it, because activation is asynchronous and a caller that posts a keystroke before it lands types into whatever was already in front. That check is whatapple_desktop_hoverleans on when it declines to post into an application that did not come forward.