How to feed a game idea to an AI agent and get a working prototype with assets and physics

Usually, attempts to get a neural network to write a game end up producing a couple hundred lines of code. You run this code in the browser, see a gray square jumping over a green rectangle, and the magic dissipates there. To make a proper prototype, you need to manually configure the engine, generate sprites or 3D models, hook up physics, fix shaders, and make sure the scene renders without artifacts.

The godogen repository takes a different approach. It's not a game and not even a ready-made binary generator. The author built a pipeline that deploys a project environment for autonomous agents like Claude Code or Codex. You describe the mechanics in text, and the agent itself sets up the engine, calls external APIs for texture and 3D mesh generation, runs the build, and views the result through video recording or a live instance.

How this pipeline works

The whole idea rests on a chain: godogen generates the repository structure for the chosen stack, and then control passes to the agent. The output is not a mess of scripts, but a clean scaffold with instructions for the LLM.

The godogen source repository itself is lightweight. Inside it contains:

• Runtime manifest in prompts/runtime.md
• Asset generation specifications in the asset-gen/ folder
• Compact guides for each engine (engines/godot.md, engines/bevy.md, engines/babylon.md)
• Script publish.sh that assembles everything into a working project folder

When you run the publish script, it forms a minimal set of files (for example, CLAUDE.md or AGENTS.md with skills). The agent reads a one-page instruction and independently recreates the boilerplate, folder structure, and screen capture scripts.

Supported engines and asset generation

The tool is tailored for three specific game stacks:

1. Godot 4 on C#/.NET. The agent generates scenes at build time, writes logic scripts, and hooks up Jolt physics.
2. Bevy on Rust. Fully code-based ECS scenes and headless builds with offscreen rendering.
3. Babylon.js on TypeScript. Vite-based projects that are immediately served by a local web server.

With assets, the process is pragmatic. The agent doesn't try to blindly draw pixels, but instead calls specialized networks:

• Gemini generates concepts, references, and character sprites
• xAI Grok handles environment textures, simple objects, and looped animations with background removal
• Tripo3D converts images into 3D models and applies basic rigging for humanoid characters

The main feature: visual proof over compiler claims

Regular code generators consider the job done if the compiler doesn't throw errors. In games, this doesn't work: a project may build successfully, but the character might fall through the floor texture or get stuck in a wall due to a crooked collider.

Godogen implements a proof-over-claims feedback loop. The agent runs the game through a virtual display xvfb or browser, captures 15–20 seconds of real gameplay, and evaluates the frames. If invisible collisions are flying around on the video or a model is rotating the wrong way, the agent goes back to the code and fixes the scripts until the picture matches the description.

You can work with this in two modes:

• Interactive. You run the project locally or open the browser with Babylon.js, watch the agent's actions, and guide it at decision points.
• Background. You specify the requirements, go drink tea, or leave the task running overnight on a remote server. At the end, the agent will deliver a ready repository and a video report confirming functionality.

Quick start

For deployment, you'll need the installed engines and system packages like ffmpeg, xvfb, imagemagick, and vulkan-tools. If you plan to build for Godot, you need .NET SDK and Godot 4. For Bevy, you'll need Rust with Cargo installed, and for Babylon.js — a recent version of Node.js (22.12 or later) and a browser with hardware WebGL2.

Before launching, export the generator API keys:

After that, call the script publish.sh, specifying the engine, agent, and target directory:

Then go to the created folder, start an agent session there (for example, claude), and pass a text query with the mechanics description.

Running on a server

Since the iterative cycle with 3D asset generation and video runs can take several hours, running all this on a work laptop isn't always convenient. The project fits well on cloud GPU instances running Ubuntu or Debian.

The easiest way to keep a session running on the server is through tmux, and you can manage the process remotely via the session sharing interfaces available in Claude Code and Codex. In this scenario, the agent handles the build in the background without heating up your local CPU.

Who is this for

Godogen is a great fit for those who regularly participate in game jams or quickly test game design hypotheses. Instead of spending two days hunting for free models on itch.io and setting up basic scene scaffolding, you can get a working skeleton with unique assets and configured physics in a couple of hours.

You shouldn't look here for production code for large commercial releases: the agent will inevitably make mistakes with complex architecture. But as an experimental pipeline for rapid mechanic prototyping, godogen looks very practical. It's definitely worth trying if you already have a subscription to one of the CLI agents and have an interest in Godot or Bevy.