Echo is a macOS application that demonstrates how to interact with macOS applications using Accessibility and OpenAI APIs. It allows users to send prompts to the frontmost application and receive responses, showcasing a functional chat-like interface.
![Screenshot 2024-11-18 at 2 58 45 PM](https://private-user-images.githubusercontent.com/1411778/387368176-b7d2a457-d4c6-430b-8fae-c3364ea4c7af.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODY4MTQsIm5iZiI6MTczOTE4NjUxNCwicGF0aCI6Ii8xNDExNzc4LzM4NzM2ODE3Ni1iN2QyYTQ1Ny1kNGM2LTQzMGItOGZhZS1jMzM2NGVhNGM3YWYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTBUMTEyMTU0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OWQ1ODFjOGVmNGZhYzgxNzVmMzhhZDIzMmFmYzg0YTRhOGZlNzdmZjM5ZTViYWNiOTU2ZWMwMzFjYzRjYTMwZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.PzgeHNS5HKtFerbhXTL0uB7RTdhddgC1ntIpAuWlcKo)
- Real-Time Frontmost Application Detection: Automatically detects the currently active macOS application.
- User Prompts: Send prompts to interact with text area of the detected application.
- Chat View Interface: Displays interactions in a chat-like interface
Warning
Project Echo is an experimental prototype that demonstrates an approach, use in on your own risk
- macOS 14.0 or later
- Xcode 15.0 or later
-
Clone the repository:
git clone https://github.com/username/Echo.git cd Echo
-
Open the project in Xcode:
open Echo.xcodeproj
-
Build and run the project:
- Select your target device (Mac).
- Press Cmd + R or click the Run button in Xcode.
- Launch Echo on macOS.
- Activate/launch 3rd party application
- Type your prompt into the text field
- Click the Send button to interact with the frontmost application.
- View the response in the chat interface.
Note: You should add OpenAI API key to environment variables.
-
ViewController
:- Manages the main app logic, including UI setup, handling user prompts, and interacting with applications.
-
MessagesStore
:- Stores and manages messages displayed in the chat interface.
-
Agent
:- Processes user prompts and generates responses based on the content of the frontmost application.
-
FrontmostApplication
:- Observes and detects the currently active macOS application.
-
AccessbilableApplication
:- Provides Accessibility API integration to fetch and update content in the frontmost application.
- Chat Interface:
- Built using
SwiftUI
and embedded intoNSViewController
viaNSHostingController
.
- Built using
This app requires and asks automatically macOS Accessibility permissions to function correctly. Follow these steps to grant permissions:
- Open System Preferences > Privacy & Security > Accessibility.
- Click the lock to make changes and authenticate.
- Add Echo to the list of allowed applications.
This project is licensed under the MIT License. See the LICENSE file for details.
Created by MacPaw Research