The Last of Us Part II Delves Into the Story Behind the Incredible Realistic Facial Animation

The Last of Us Part II truly pushed the envelop regarding its story, art director, sound design, and gameplay. Its animations, specifically facial animations, are no exception. Keith Paciello, the studio animator who created the ambitious Emotional Systematic Facial Animation system, extensively details the process surrounding the creation of these in-game animations.

Warning: This article contains massive spoilers about story details for The Last of Us Part II, including the ending.

Look at the scene recorded below, for instance, of Ellie gazing at artwork in a museum:

As you can see, she’s actually reacting to that picture. Paciello explains how the magic unfolds: “In that instance, you as a player are aiming with the controller for Ellie to look at the painting, which is triggering a ‘look at target’ placed by a designer. On top of that, I animated small eye darts (saccades) within the character’s facial idles to try and indicate an overall thought process. So animated eye saccades sitting on top of the eye-aim, work together to create what looks like focus and thought process.”

The Emotional Systematic Facial Animation system picks a facial expression from a range of nearly 20 different emotional states for any of the 25 key characters that are on-screen. That covers leads, co-op partners, enemies, and even Infected to an extent. This facial animation works in unison with eye movements, body language, breathing, which are all interlinked and triggered by script beats, dialogue, encounters or ambient moments, like Ellie seemingly being absorbed by a painting. The illusion of emotion is constructed with mathematical precision. “It brings a depth to the characters that we’ve never seen before,” says Paciello.

The creation of said engine, believe it or not, came from a simple blade of grass. According to Paciello: “Everyone was stepping up their game for TLOUII. We were looking at and talking about how to make a blade of grass even better [in game]. In doing so, we panned up, and there was this blank face on the character. I was like, ‘Oh.’ It was then I wondered how we could simply, across the entire game, add these emotional beats to the characters, so at any point, you can tell what that character is feeling.”

Using the Ellie facial model as a base, he started by sculpting expressions based on seven universally recognizable emotions: joy, sadness, fear, anger, surprise, disgust and contempt. Liaising with the dialogue team, they tagged emotions to be triggered at specific lines, building a solid foundation and allowing characters to smoothly transition from one to another. “We could emotionally pace out our characters from the absolute beginning of the game to the absolute end of the game and blend in and out of cinematics seamlessly.”

Paciello was then asked to support melee, fashioning realistic reactions for combatants. He also aided in animating the breathing system (“six animations, from small breathing all the way up to exhausted”) which would go to such lengths as to see characters switch from open to closed mouth breathing depending on their proximity to an enemy.

Outside of script-specific triggers, characters would also have a number of ‘neutral’ idle states with emotional overlays.

Paciello also hand-sculpting poses for each character to match their emotional states, which racked up to some 40 poses for each of the 15-20 emotion sets. By the end of production, he created a staggering 15,000 individual, hand-sculpted poses. Ellie’s reaction to seeing the Tyrannosaurus Rex statue (shown below) during a flashback “was specially made for that moment.”

This process naturally transferred over to Abby by allowing the devs to redo all of her facial animations in order to give that climatic beach fight the emotional power it needed.

“We all learnt a lot from that,” says Paciello. “When I first got to it, Abby, being so emaciated, still looked savage. I talked to Christian and said ‘I’m going to duplicate all her emotional sets for melee and I’m going to make them all exhausted.’ That way you choose between fierce and exhausted, so maybe she musters up enough strength but after she throws [a punch], goes into her exhausted state.”

When asked about what Paciello’s own favorite moment from the game was, he came right back to the beach fight. He sees it as the culmination of everything the teams had worked on and proof that the system had achieved its goal. “It made me really made me feel like, ‘Okay, we’ve really pushed it.’ It’s what I wanted, it’s what I dreamt of when I pitched the idea of the system.”

For even more insight into the developmental process behind The Last of Us Part II sound designer Beau Anthony Jimenez took to Twitter to detail the process behind voicing the various Infected enemy types, as well as crediting the talented voice actors who gave performances for them.

Some recently released numbers demonstrate the sheer amount of manpower needed to bring such an ambitious game to fruition. Naughty Dog’s final tally was 2332 people total to develop the game, which includes 2169 developers credited and 163 extra “thanks” to help with The Last of Us Part II. 14 outsourced studios, two of them devoted to sound design and mixing with the other 12 dealing with art direction.

Of course Naughty Dog’s development process hasn’t been controversy free, as the studio has been in the hotseat for quite some time concerning its own issues with the unfortunately common industry “crunch.” According to Neil Druckmann in an interview concerning the practice: “We don’t try to babysit people. We draw people who want to tell these stories and who want to leave a mark on the industry. And they’re gonna work very hard to do it. We need to put some guardrails [in] so they don’t injure themselves, but I don’t think we could prevent them from working hard and still make the kind of games we make.”

For any newcomers to the franchise, or those who haven’t played Part II yet, Features Editor put together a guide of everything you need to know about The Last of Us. And you can check out our review from Features Editor Ryan Meitzler, who absolutely adored the game.

The post The Last of Us Part II Delves Into the Story Behind the Incredible Realistic Facial Animation by Allisa James appeared first on DualShockers.

from DualShockers https://ift.tt/3gFEyM0