Programming game AI is challenging.

Whether you’re trying to program a game master, path finding, enemies to shoot at or allies to play with, there is a sweet spot you need to hit. When you hit that spot the AI blends so seamlessly into the game that the player doesn’t even register that it’s there. Miss that sweet spot and it will stick out: “I ordered that unit to go there, why did it go around that building?”, “Why is that enemy firing at me? He can’t see me, he doesn’t know I’m here!”.

For our next game I have the task of creating the enemy AI. I started off doing this by programming heuristic algorithms; I was overwhelmed pretty quickly. Heavily physics based movement, varying environments, plenty of obstacles and a wide range of situations made a good heuristic algorithm difficult to come up with and write. There were many situations where any human could see that the AI was lacking and behaving in a very inefficient way. It was only going to get worse.

At this point I had to decide if it was worth continuing down this path or find another way. Stefan and I had previously discussed using machine learning to create our AIs but my experience in the subject was pretty thin. Stefan suggested I experiment with it for a week or two and decide what to do at that point; GREAT IDEA!

Over the past week or so I’ve been playing around with Unity’s ML-Agents Toolkit. With this toolkit, Unity has made it much simpler to use machine learning to create game AIs. Within a day or two of tinkering I had figured out the basics and started to train an AI for the enemies.

While not impossible, training a single brain to control every aspect of an enemy was out of reach for me; I had to break it down.

I had previously struggled with creating an AI which could follow a set of waypoints. There were a lot of little maneuvers a human would perform which was hard to code in a way that was easily extensible and maintainable.

ML-Agents to the rescue. Creating a training environment and designing a reward system for an AI was much easier for me to wrap my head around. Watching the AI teach itself and come up with it’s own set of maneuvers was pretty magical.

After a few days of optimisation I now have an AI which works pretty well in the training environment.

I wonder if the AI will be any fun to play with. I have a little more work to do before I can test that; more on this later!

New Logo

Out with the old! Thanks to Andrew McDonald for the amazing job with our new logo. He asked all the right questions and has crafted a logo which is more in line with the image we want to project. Highly professional, superb quality of deliverables.

Contact him at