SIMA: AI agent for virtual environments

DeepMind, Google's AI powerhouse, has unveiled a groundbreaking new AI agent named SIMA (Scalable Instructable Multiworld Agent).
Introducing SIMA, a Scalable Instructable Multiworld Agent
Introducing SIMA, a Scalable Instructable Multiworld Agent

This remarkable model demonstrates a never-before-seen level of versatility by understanding and interacting with diverse virtual environments using simple, everyday language.

How Does SIMA Work?

  • Intuitive Control: No complex coding or game-specific knowledge needed! SIMA understands instructions given in natural language, just like you would talk to a friend.
  • Future Vision: SIMA leverages powerful image-language mapping and a predictive video model to perceive and anticipate its surroundings.
  • Human-like Input: Unlike other AI agents, SIMA controls game characters using familiar keyboard and mouse inputs, allowing it to interact with potentially any virtual world.
SIMA comprises pre-trained vision models, and a main model that includes a memory and outputs keyboard and mouse actions.

SIMA's Training Grounds

To develop its impressive skills, SIMA was extensively trained on nine complex 3D games. This training has allowed it to significantly outperform AI agents designed for individual games. Its gaming adventures have included titles like No Man's Sky and Goat Simulator 3, honing skills in complex tasks like resource management and navigation across vast landscapes.


SIMA was evaluated across 600 basic skills, spanning navigation, object interaction, and menu use.

The Future Potential

DeepMind believes AI models like SIMA hold the key to revolutionizing the gaming experience. Imagine AI allies capable of carrying out complex tasks with expert precision, ultimately offering new levels of strategy and support. SIMA's adaptability could open doors to entirely new forms of interactive entertainment.

About the author


AI Evangelist. Digital twin at

AI Pill

Take AI 💊 Deep Dive Into The Coming Wave.

AI Pill

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to AI Pill.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.