What Is Screen Context for AI? Visual Awareness for AI Commands
Screen Context is a feature that captures a screenshot and sends it alongside your text to an AI model, giving the AI visual awareness of what's on your screen. This enables context-aware responses that consider both your selected text and the surrounding visual environment.
Explanation
Without Screen Context, AI commands only see the text you selected. With Screen Context enabled, the AI also sees your entire screen - the application you're using, surrounding content, UI elements, and visual layout.
This is powerful for tasks like: asking AI to help with code while it can see your IDE, getting feedback on a design while AI sees the mockup, writing a response to a message while AI sees the conversation thread, or describing what you see for documentation purposes.
Screen Context uses macOS ScreenCaptureKit to capture your display efficiently. The screenshot is sent as an attachment to your AI provider along with the text. It's toggled per-command, so you control exactly when AI sees your screen.
When using voice instruction without any selected text, Screen Context switches to screen-only analysis mode where AI focuses entirely on what's visible.
How Echoo Helps
Echoo's Screen Context toggle works with any command - built-in, custom, or voice instruction. Enable it per-command in Settings. Uses macOS ScreenCaptureKit for efficient capture. When no text is selected with voice instruction, Echoo switches to screen-only analysis mode for visual questions about your display.
Related Terms
AI Text Transformation
AI text transformation is the process of using artificial intelligence models to modify, improve, or convert text. This includes grammar correction, translation, summarization, tone adjustment, and rewriting.
AI Voice Dictation
AI voice dictation is the use of artificial intelligence to convert spoken words into written text. Modern AI dictation engines run locally on your device, support multiple languages, and can post-process transcriptions with AI.
macOS Accessibility API
The macOS Accessibility API is a system framework that allows applications to interact with UI elements in other apps. It enables tools like Echoo to read selected text and replace it with transformed content in any application.
Related Use Cases
Use Screen Context with AI Commands on Mac
Give AI commands visual context by attaching a screenshot of your screen. Toggle Screen Context per command so AI sees what you see on Mac.
AI Code Review with Keyboard Shortcuts on Mac
Review and improve code on macOS with AI-powered keyboard shortcuts. Get instant refactoring suggestions in any code editor.
Related AI Providers
OpenAI
Connect Echoo to OpenAI GPT models for powerful AI text transformation on macOS. Use GPT-4, GPT-5, and more with keyboard shortcuts.
Anthropic
Connect Echoo to Anthropic Claude models for thoughtful, nuanced AI text transformation on macOS. Claude Opus, Sonnet, and Haiku.
Google Gemini
Connect Echoo to Google Gemini AI for free, fast text transformation on macOS. Gemini Flash Lite offers a generous free tier.
Related Commands
Frequently Asked Questions
The screenshot is sent to your chosen AI provider (like OpenAI or Anthropic) alongside your text. If you use local LLMs via Ollama, the screenshot stays entirely on your device.
Yes. Screen Context is toggled per-command. Enable it only on commands where visual context is helpful. It's off by default and requires Screen Recording permission.
Explore More
Set up OpenAI
Connect Echoo to OpenAI GPT models for powerful AI text transformation on macOS. Use GPT-4, GPT-5, and more with keyboard shortcuts.
Set up Anthropic
Connect Echoo to Anthropic Claude models for thoughtful, nuanced AI text transformation on macOS. Claude Opus, Sonnet, and Haiku.
Set up Google Gemini
Connect Echoo to Google Gemini AI for free, fast text transformation on macOS. Gemini Flash Lite offers a generous free tier.
Echoo vs Raycast AI
Compare Echoo and Raycast AI for text transformation on macOS. Free vs paid, privacy, features, and workflow differences.
Echoo vs Text Blaze
Compare Echoo and Text Blaze for text transformation. AI-powered shortcuts vs template-based text expansion on macOS.
Echoo vs Espanso
Compare Echoo and Espanso for macOS text productivity. AI-powered transformation vs rule-based text expansion.
Ready to Try It?
Download Echoo for free and start transforming text with AI shortcuts.