Video game facial animation has come a long way, from the first twitch of Mario's moustache to the near-Hollywood production levels of titles like Grand Theft Auto and Halo . Today's characters can emote with a visual fidelity that has entered the realm of realism. But dramatic impact isn't just the domain of narrative-led action titles. A sense of believability is also key to the next generation of sports games - games like NBA 2K. Not only must these games accurately reflect the strategy and physicality of the sport they emulate, but also the intensity, passion, and the all-out drama experienced by the players.
I've been working on the NBA series for six years now, and our end goal hasn't changed in that time - we just want to get better and better at selling the emotion of basketball, says Anthony Tominia, performance-capture stage manager at 2K. We're working on 2K16 now, and we want everything to be as realistic as it possibly can be. That means everything from mocap'ing the basketball itself to realistically representing the real-life players in digital form.
This year, we're trying to get more expressive emotions; better-looking lip sync in our story mode; distinct reactions that mirror what happens on the actual court - Faceware Technologies is an important contributing factor to all of that.
Capturing Every Wrinkle
2K has used Faceware's hardware and software solutions for it facial performance capture for many years, and utilized them once again to bring NBA 2K16 to life. For Tominia, the technology was nothing less than integral in capturing the raw kinetic energy of the sport.
Basketball is a story, and emotion helps you to tell that story, he says on his team's approach to NBA 2K16. Those physical facial expressions add to the drama of the game. The player might get irritated if they miss a shot, or reach for the skies when they make that last-minute alley-oop, and seeing that makes your actions all the more engaging. We really wanted to delve deeper into those little narrative movements this time and sell what's happening on an emotional level.
So, in 2K16 we've really exploited our rigs to the extreme so we can match every nuance of the person, continues Tominia. We're making sure that not only does each digital character look like their real-life counterpart, but they also show the signature emotion of that person. When you look at Shaquille O'Neal this year, it's not just a representation of him - it has every tic, every subtle nuance that makes his face uniquely O'Neal.' It's not just a celebratory face, it's Shaq's celebratory face, based off his real-life performance.
Custom Software
Faceware's technology was used to capture the facial performance of over 30 different NBA athletes, including O'Neal. Actions both on the court and off were recorded, with players undergoing full-performance capture for their in-game animations and also recording dialog for the My Career story mode. Once this acquisition was complete, it was time to transplant the performances into the game.
To make the next steps of the process more streamlined, the 2K team dived into the Analyzer and Retargeter scriptability.
The great thing about Faceware's script interface is that it's really simple, so we can easily develop scripts that make our job easier, explains Tominia. In this case, we wrote a bunch of Python scripts that allowed us to automatically solve everything that little bit faster. There were over 232 lines spoken by each of the 30 players, and we found that we could easily batch, through Analyzer, 232 lines per player without having to even touch the data.
We did the same thing with Retargeter, he continues. In our pipeline, the Python script does the retargeting, creates play blasts, and saves those files immediately for review. The process doesn't have to be manned, so we could capture all day, then leave the system to analyze and retarget overnight. We would then just come in the next morning and quickly identify which files needed the most touch-up, and pass those on to the team. The speed could not be better.
Speed and Power to Rival LeBron
The ease of use of the software, along with the custom development done to streamline the pipeline, meant that after analyzing and retargeting the data, cleanup time on NBA 2K's animation was down to three to four minutes per second of data.
At the end of the day, this kind of work will always rely on the touch of a human animator to flesh the capture out - that's just a matter of fact, begins Tominia. However, in our line of work, we want to get as far into the pipeline as we can before a human has to touch that data, and Faceware delivers excellent results in that regard. Four minutes per second wasnt really that much time, because there was nobody touching the file until that point, and that's a huge advantage in a production such as this.
Combined with the raw power of Faceware's solutions, this speed makes for a particularly lethal combination. Last year for NBA 2K, we recorded 120 hours of facial animation - that's massive amounts of data, says Tominia. In comparison, most feature films are two hours long and have a maximum of three CG characters on screen at any time, so that's just six hours. But we did 20 times that in the space of a year.
When you put that into context, it's insane, he marvels. A keyframe animator might be able to give you five minutes of animation a day, and that's pushing it. That would take one animator nearly four years of nonstop work to achieve, and if you're paying them $30 an hour, it's going to total up to an incredible amount. Really, there's no number of animators on the planet that could be assembled to put out 120
