Entropic AI model can now control your computer + video

Entropic’s latest AI model, Claude 3.5 Sonnet, has introduced a new feature in beta mode that can control a computer by looking at the screen, moving the mouse, clicking buttons, and typing text. This feature allows Claude to work with the computer as a human would control the computer.

In its new blog post, Entropic emphasizes that the Computer Use feature is still in the experimental stage and its use can be difficult and prone to errors. The company says it’s rolling out the feature to developers first and expects it to get better over time.

Computer control by entropic artificial intelligence model

In the video below, Sam Ringer, a researcher at Entropy, shows how Claude can fill out a form on an external website by looking at information in a spreadsheet, searching for company information after analyzing its CRM system, and then understanding that information.

“Starting today, developers can ask Claude to work with these devices in the same way people use computers by accessing (this capability) through the API,” Entropy’s official statement reads. Claude 3.5 Sonnet is the first pioneering artificial intelligence model that is capable of using computers.”

This version of Claude has been told to stay off social media. This model, with the help of some mechanisms, realizes when Claude is told to participate in activities related to elections, for example, or to create content in social networks.

The Computer Use feature is currently only available through the API; So normal users cannot use it easily. Entropic has taught Claude to interpret what he sees on the screen. Developers can tell this model to handle various tasks such as browsing websites or even using apps.