The success of sports games relies on engendering a sense of reality - not just in the ebb and flow of the sport itself, but in selling the player on the fantasy that they really are in the sneakers of their favourite players.To capture that sense of authenticity in its famous basketball series, 2K relies on the fidelity of Faceware...
Video game facial animation has come a long way, from the first twitch of Mario's moustache to the near-Hollywood production levels of titles like Grand Theft Auto and Halo. Today's characters can emote with a visual fidelity that's long since leapt over the Uncanny Valley and entered the realms of true-to-life realism.
But dramatic impact isn't just the domain of narrative-led action titles. A sense of believability is also key to the next generation of sports games - games like NBA 2K16. Not only must these games accurately reflect the strategy and physicality of the sport that they emulate, but also the intensity, the passion, and the all-out drama experienced by the players.
That's where Faceware Technologies comes in.
I've been working on the NBA series for six years now, and our end goal hasn't changed in that time - we just want to get better and better at selling the emotion of basketball, begins Anthony Tominia, performance capture stage manager at 2K. We're working on 2K16 now, and we want everything to be as realistic as it possibly can be. That means everything from mo-capping the basketball itself to realistically representing the real-life players in digital form.
This year we're trying to get more expressive emotions; better-looking lip sync in our story mode; distinct reactions that mirror what happens on the actual court - Faceware Technologies is an important contributing factor to all of that.
Capturing every wrinkle
2K has used Faceware's hardware and software solutions for it facial performance capture for many years, and utilised them once again to bring NBA 2K16 to life. For Tominia, the technology was nothing less than integral in capturing the raw kinetic energy of the sport.
Basketball is a story, and emotion helps you to tell that story, he says on his team's approach to NBA 2K16. Those physical facial expressions add to the drama of the game. The player might get irritated if they miss a shot, or reach for the skies when they make that last-minute alley-oop, and seeing that makes your actions all the more engaging. We really wanted to delve deeper into those little narrative movements this time and sell what's happening on an emotional level.
So, in 2K16 we've really exploited our rigs to the extreme so we can match every nuance of the person, continues Tominia. We're making sure that not only does each digital character look like their real-life counterpart, but they also show the signature emotion of that person. When you look at Shaquille O'Neal this year it's not just a representation of him - it has every tic, every subtle nuance that makes his face uniquely O'Neal'. It's not just a celebratory face, it's Shaq's celebratory face, based off his real-life performance. And that's thanks to Faceware.
Custom software
Faceware's technology was used to capture the facial performance of over 30 different NBA athletes, including O'Neal. Actions both on the court and off were recorded, with players undergoing full performance capture for their in-game animations and also recording dialogue for the My Career story mode. Once this acquisition was complete, it was time to transplant the performances into the game.
To make the next steps of the process more streamlined, the 2K team dived into the Analyzer and Retargeter scriptability.
The great thing about Faceware's script interface is that it's really simple, so we can easily develop scripts that make our job easier, explains Tominia. In this case, we wrote a bunch of Python scripts that allowed us to automatically solve everything that little bit faster. There were over 232 lines spoken by each of the 30 players, and we found that we could easily batch, through Analyzer, 232 lines per player without having to even touch the data.
We did the same thing with Retargeter, he continues. In our pipeline the Python script does the retargeting, creates play blasts and saves those files immediately for review. The process doesn't have to be manned, so we could capture all day, then leave the system to analyze and retarget overnight. We would then just come in the next morning and quickly identify which files needed the most touch up and pass those on to the team. The speed could not be better.
Speed and power to rival LeBron
The ease of use of the software, along with the custom development done to streamline the pipeline, meant that after analyzing and retargeting the data, clean-up time on NBA 2K16's animation was down to three-four minutes per second of data.
At the end of the day, this kind of work will always rely on the touch of a human animator to flesh the capture out - that's just a matter of fact, begins Tominia. However, in our line of work we want to get as far into the pipeline as we can before a human has to touch that data, and Faceware delivers excellent results in that regard. Four minutes per second wasnt really that much time, because there was nobody touching the file until that point, and that's a huge advantage in a production such as this.
Combined with the raw power of Faceware's solutions, this speed makes for a particularly lethal combination. Last year for NBA 2K15, we recorded 120 hours of facial animation - that's massive amounts of data, says Tominia. In comparison, most feature films are two hours long and have a maximum of three CG characters on screen at any time, so that's just six hour
