Show HN: Let your AI agents paint big arrows, boxes and text on your screen

來源原標題

Show HN: Let your AI agents paint big arrows, boxes and text on your screen

作者

franze

原始標籤

hackernews

來源訊號

HN points 349　HN 留言 149

原始連結

https://github.com/franzenzenhofer/big-arrow-on-the-screen

摘要原文

Let your AI agents paint big arrows, boxes and text on your screen big-arrow-on-the-screen ( bigarrow ) is a macOS command-line tool, plus a skill for Claude Code and Codex, that draws an arrow and a sign on top of every window. Clicks go through, your keyboard focus stays put, and the arrow removes itself. MIT licensed. Real apps · What is it for? · Install · Commands · Looks · Creative arrows · Starred by · FAQ · How we know it works · For agents · Plan · Prior art · License Your AI agent can refactor a monorepo, write a migration and explain monads, but when it needs you to click one button it prints "please click Allow in the dialog" into a terminal you are not looking at. bigarrow gives it a finger. bigarrow point --element " Allow " --app " System Settings " --text " Franz, click Allow: Ghostty may control your Mac " One transparent window above everything, on every display and every Space. Drawing needs no macOS permission at all . One Swift binary: no daemon, no menu-bar icon, no account, no telemetry, and, we checked twice, no AI inside. It is an arrow. Real apps, real use cases Real apps on a real Mac (macOS 27), the real bigarrow , staged by scripts/real-scenes.sh . macOS Desktop & Dock: stop "click wallpaper to show desktop" "The most annoying change in Mac update history" , fixed in six arrows ( MP4 ). System Settings: grant a permission open " x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility " bigarrow start --element Terminal_Toggle --app " System Settings " \ --text " Franz, switch this on: Terminal may control your Mac " --from right --color green --close-button bigarrow start --element Add --role button --app " System Settings " \ --text " Not in the list? Plus. Then find it. " --from bottom-right --style ring --color orange --shape zigzag --size S Keynote: a three-step how-to bigarrow start --element Animate --app Keynote --role radiobutton --text " 1. Click Animate " --from right \ --style ring --color purple bigarrow start --element " Add an Effect " --app Keynote --text " 2. Add an Effect " --from right \ --style box --corners sharp --color " #FF9F0A " --shape straight bigarrow start --element Play --app Keynote --role button --text " 3. Press play. Bask in the applause. " \ --from top --color teal --shape zigzag --size S Print dialog: helping Mom save a PDF bigarrow start --element PDF --role button --app TextEdit \ --text " Mom, click PDF, then Save as PDF " --from bottom --color pink --border white-black bigarrow start --element Cancel --role button --app TextEdit \ --text " Not this one, Mom " --from bottom-right --color black --size S Chrome: the right tab out of 14 bigarrow start --app " Google Chrome:Sourdough " --element " Sourdough - Wikipedia " --role radiobutton \ --text " It's this tab, not the other 13 " --from top --shape zigzag --color " #5856D6 " --size L --app "App:tab title" raises the window and selects the tab first. Finder: "no, the other grid icon" bigarrow start --element " icon view " --role radiobutton --app Finder \ --text " Not this one " --from top-left --style ring --color black --size S bigarrow start --element Group --role menubutton --app Finder \ --text " No, the other grid icon. This one. " --from top --style box --color green A real guide: allow screen recording The whole guide as a 6-page PDF, every screenshot an arrow an agent drew. Download the PDF or read it here . What is this actually for? Fair question. Arrows have existed since roughly the Paleolithic. What changed: agents now do real work on your Mac, and sooner or later hit a step only a human may do , or one the human wants to learn. "Click Allow." Permission prompts, OAuth consent, "Open with...?". The agent finds the button but must not, or cannot, press it. It can point. "Your turn." 2FA, CAPTCHAs, passkeys, payments, signatures. It points, you decide, it continues. "I need you, and you're making coffee." --say reads the sign aloud. Your Mac will literally call you back to your desk. "Show me how." Ask how to do something in Blender, and the agent points at each control in turn. A product tour, minus the product. Helping someone else. On a parent's Mac or over a screen share, pointing beats "the button at the top, no, the other top". Demos and docs. Highlight while recording, or render straight to a PNG with --png . Debugging coordinates. Point at your Accessibility or Peekaboo coordinates and look. --dry-run --json says where it would point. Not a screen annotator, not a click bot, not a screenshot tool. It never clicks, types or captures. It only points. Deliberately. Install brew install franzenzenhofer/tap/bigarrow bigarrow install-skill # teaches Claude Code (~/.claude/skills) and Codex (~/.agents/skills) From source: swift build -c release (Xcode 16+, macOS 14+), binary at .build/release/bigarrow . The binary is pure Swift; scripts/ only records screenshots and runs tests. The three commands an agent needs bigarrow point --element " Allow " --app " System Settings " --text " Franz, click Allow " # by label bigarrow point --at 760,500 --text " Franz, click HERE " # by coordinate bigarrow start --window " Safari:Inbox " --text " This window " && bigarrow stop # until stopped Every arrow ends by itself. Nobody has to clean up after an agent that forgot: Time limit --duration 10 ( point 8 s, start 300 s, 0 = no limit) Start and stop start returns at once; stop (or stop --all ) removes it The agent goes away the arrow ends with the agent process that drew it ( CLAUDE_PID or BIGARROW_OWNER_PID ) The human answers bigarrow stop --hook as a Claude Code UserPromptSubmit hook clears that session's arrows The human closes it --close-button (opt-in) Targets: --at X,Y , --rect X,Y,W,H , --mouse , --window App[:title] , --element Label --app App , --peekaboo ID --snapshot see.json . Coordinates are global top-left logical points, as Accessibility and Peekaboo report them; --display N makes them relative to one display. --app App[:window or tab title] (or --window ) first brings that app, window or Chrome/Safari tab to the front, because pointing at a window hidden behind your terminal is a special kind of unhelpful. If another app covers the target later, the arrow hides until it is visible again. --no-raise opts out. bigarrow elements --app X lists what --element can match; bigarrow doctor shows permissions and displays. Every command takes --json . Exit codes: 0 ok, 2 bad input, 3 target not found, 4 permission missing. Agents love exit codes. Humans tolerate them. Looks It is an arrow, so we spent an unreasonable amount of time on how it looks. Six looks over white, macOS grey, dark, black, red and a busy web page: --shape bend|straight|zigzag|spiral : zigzag for when it is really urgent; spiral loops once around the sign, for when it must be impossible to miss --style arrow|ring|box (rings and boxes are outlines, the target stays visible), --size S|M|L , --corners round|sharp --color red|orange|yellow|green|teal|blue|purple|pink|black|white|#RRGGBB --border shadow|white-black|black : white border with drop shadow (default), a thin black edge instead of the shadow, or just a thin black outline --border-color , --text-color , --edge-color , --close-color , --close-x-color ; left out, each picks a readable colour --close-button puts an X inside the sign's right end; --follow moves with a window or element; --until-click ends on a click on the target; --say speaks the sign Several arrows at once keep their signs out of each other's way scripts/gallery.py renders every combination and zooms into every sign-to-shaft joint ( junctions ), because a seam at the joint was, apparently, unacceptable. Creative arrows Fake dialogs, real bigarrow , clean CI runner ( BACKDROP_ARGS=--cover scripts/funny-scenes.sh ). The dialogs are fake. The feelings are real. (349 points, 149 comments on Hacker News)