Username:

Password:

Fargot Password? / Help

Tiered Behaviour Architecture

In recent years, virtual character artificial intelligence has developed more slowly than other aspects of video games, such as graphics, mainly due to the cost of scripting complex and believable virtual characters. Our research led by Richard Zhao and Duane Szafron develops a Tiered Behaviour Architecture model for controlling the behaviours of non-player virtual characters. The architecture contains the circumstances, schedules, objectives and roles of the virtual characters, and selects the appropriate role-level controller to generate behaviours. The role-level controllers use specialized self-contained methods, such as behaviour capture or behaviour learning, to generate fine-grained behaviours for particular roles. We believe the architecture provides more expressive power and flexibility while allowing game designers to retain control over the behaviours of characters. The architecture should help designers to produce believable behaviours in games.

Behaviour Overview in Skyrim

Tiered Behaviour Architecture

We propose a new tiered architecture model and its implementation for game designers that allows them to create behaviours for virtual characters in story-based games, without having to learn programming. The model allows for multiple circumstances, schedules, and objectives, as well as stochasticity in the schedules and dynamically chosen roles to satisfy the objectives. We devised a set of user studies and experiments to validate the expressiveness, performance, and quality of the proposed Tiered Behaviour Architecture.

Full publication can be found here.

Cyclic Scheduler

For the automatic generation of daily schedules for virtual characters in large open-world games, a cyclic scheduler is used to determine the general objectives of the virtual characters and the roles that will satisfy these objectives. With the cyclic scheduling tool designers are able to specify partial constraints (such as sleeping for 8 hours a day) and use the scheduler to fill in the details. This allows for rapid creation of daily schedules for characters. A user study showed the cyclic scheduling tool increases completeness, correctness and efficiency over traditional scripting methods.

Full publication can be found here.

Behaviour Capture

Once a character assumes a specific role, we propose a new method of creating fine-tuned behaviours called Behaviour Capture, based on the concept of motion capture. With motion capture, sensors are attached to the bodies of actors, and as the actors move their bodies, the spatial locations of their body parts are recorded. The data is used to animate virtual characters to move in the same way. Our system of Behaviour Capture is based on a similar idea of using captured traces to guide virtual character behaviours, but Behaviour Capture is not a generalization of motion capture. Our behaviour traces represent high-level intentions as opposed to motion trajectories in space.

Behaviour capture enables a game designer to take control of a particular character during training and perform exemplar behaviours. It captures traces of the exemplar behaviours and generalizes them. Generalization is necessary to generate natural behaviours with short training times. First, interactions are generalized. If a particular character should talk to a particular set of characters (such as rich people only) in a tavern, then the trainer can train this character by talking to only one rich person. If another character should have the same behaviour, the trainer does not have to train the second character separately. Second, during game play, the character should not repeatedly follow the exact training trace. We present several trace generalization mechanisms, including a technique that learns a Hidden-Markov Model (HMM). Our capture and generalization do not require scripting knowledge on the part of the game designer.

Full publication can be found here.

Behaviour Capture

Behaviour Learning

Certain scenarios require virtual characters to quickly adapt their behaviours to react to the rapidly changing virtual environment. One example is combat, where virtual characters have to react quickly to the tactics of their enemies to stay alive. Another example is companion characters (followers) who learns to help (or not help) the player character in disarming traps. In these scenarios, we employ reinforcement learning (RL) techniques to allow the virtual characters to choose the appropriate behaviours.

We extended Sarsa(λ), an online single agent RL algorithm (by Sutton and Barto) with adaptive learning rates, adaptive exploration rates, double-reward system, softmax-greedy action selection policy, and agent modelling. The resulting algorithm, ALeRT-AM, learns and adapts to virtual character behaviours online. On each time step, virtual characters perform an action that they think is best and observe the consequences. Characters use the consequences to update their own understanding of the environment so that they can choose better actions the next time. This process is done offline to give the characters an initial set of appreciate behaviours, and continues as the game plays so that the characters can adapt to the changing environment, including the behaviours of the player.

Full publications can be found here and here.