This Android app operates the screen with commands from vision LLMs.
It can now execute commands directly in Termux via API. After each update, click "Restore System Message" to access the new tools!
The hotfix that works when the app is new installed is currently in the review queue at Google. Until then, get the hotfix from here.
Due to a GitHub restriction, you must be logged in to view the nightly download links:
nightly builds from Github actions (You probably must reinstall the app because of different signatures) or install the
Screen Operator v4.1 (hotfix).apk (without log in)
Updates in Github are much faster than on the Play Store.
This app can be developed entirely by AI and therefore its development can be continued by anyone:
To vibe coding for free I use Claude.ai. You will have to fork the project first so that you can edit it. It has a free quota that will be every 5 hours refreshed and you can also easy switch the account to refresh the free quota. It's best to use different browsers for this. Use a GitHub access token and Claude can automatically push the changes. Sonnet 5 sometimes refuses to work with the token. In this case, use Sonnet 4.6. Add the token and perhaps the link to the repo/branch (code) to your preferences in Claude. Then you don't have to enter the same information every time. Select the branch with the latest changes. Always enter tasks, if necessary numbered, and written very precisely and in detail. You can also use the free arena.ai's agent mode with your github token and link to your fork.
You can build the apk with Github actions: Stay in your fork, on your user account (you won't be able to start it otherwise), on mobile, click the gear icon and then Actions, and on desktop, click Actions directly. Click Workflows, select Android Build APK (app) or Android Build APK (humanoperator), and start your chosen branch. The selected module is always built, after about 5 minutes your app will be ready!
Here you can share your own Skill Set
First attempt ever is recorded (YouTube)
If you in your Google account identified as under 18, you need an adult account because Google is (unreasonably) denying you the API key.