The brains are trained! It was time to use them in our game.
I’m not sure if it was because my Google-fu was weak but there didn’t seem to be a lot of information out there about applying trained brains in production.
One issue I noticed was there was no way of disabling the configuration of the environment during the initialisation phase of the academy. We can’t have the academy screwing with the resolution or time scale in game.
I wanted to avoid making changes to the toolkits Academy class but there didn’t seem like any other way (if there is, please let me know).
I had to modify the ConfigureEnvironment() function so that it could be overridden. In our ActorAcademy class I added a switch which allows me to turn off training mode so that ConfigureEnvironment() is not run.
Simplifying the Agent
When creating an agent you need to implement the following:
- The observations that the agent collects
- The way the agent controls its body (or whatever it’s controlling)
- How the agent is rewarded
- When the agent is done and needs to be reset
- How to reset the agent
This can lead to a fairly chunky Agent class.
In game mode, only the observations and how the agent controls its body is needed. I created another Agent class and stripped out all the unnecessary, this greatly reduced the amount of code and removed quite a few dependencies.
I’m still piecing all of the bits of the AI together to create a fully functional game AI; more on how I’m doing this later.