Microsoft has introduced Muse, its first-ever generative AI model intended for assisting with game ideation. The model executes the generation of game visuals, controls, or both.
It was initially known as the World and Human Action Model (WHAM) before changing its name to Muse. Muse boasts a collaborative development effort between the Microsoft Research Game Intelligence team, the Teachable AI Experiences (Tai X) team, and Ninja Theory at Xbox Games Studios.
According to Microsoft, Muse imagined using the breakthroughs in AI to create awesome tools for the process of creativity. They discovered that gaming was a key area for the application of Muse.
Right now, generative AI is being employed to produce numerous digital inputs, including text, images, audio, music, video, and sequences of gameplay. However, it often underdelivers the vision of the creatives depending on it. This puts the technologies in a difficult position for adoption into the creative workflow.
What is more difficult are the lower-level requirements of actual 3D game production, such as generating immersive 3D environments, designing gameplay challenges, and facilitating interactions with NPCs.
As it has been described by Microsoft in the journal Nature, the World and Human Action Model (WHAM) model works on the time dynamics of a contemporary video game.
WHAM is trained on human gameplay data to predict game visuals called ‘frames’ and player controller actions. The model captures the three-dimensional structure of the game world, the influence of controller inputs, and the temporal dynamics of gameplay.
The model is capable enough to generate complete game scenarios in response to diverse user prompts which demonstrates coherence while giving the users the opportunity to make adjustments.
Galvin Costello, the technical director at Ninja Theory said, “It’s been amazing to see the variety of ways Microsoft Research has used the Bleeding Edge environment and data to explore novel techniques in a rapidly moving AI industry.”
