Final Fantasy XVwas in development for 10 years. Hundreds, maybe thousands of people across six different game development studios helped created it. Yet its camera -- a fundamental part of the way players interact with the game’s world -- is severely flawed.

The weight on the game’s metaphorical shoulders was enormous -- not just for the future of Final Fantasyas a game series, but possibly for the entire future of game consoles in Japan, according togame director Hajime Tabata. And it delivered on its enormous promise in many ways. How is it possible that a game into which so much was invested can have such a seemingly simplistic flaw?

The camera is the interactive window through which we experience video games; the term describes not just our perspective and view of a digital space, but the freedom of or restrictions on how we as players control that viewpoint.

You can see and feel camera problems in games whenever the view is obstructed by a tree during a crucial moment, or when the perspective flips around the wrong way to show you the opposite of what you’re trying to look at, or when the camera just won’t do what you want no matter how you poke and prod at your controller.

These are the problems that have plagued third-person action games -- games in which the camera hovers outside the character’s body rather than peering through the character’s own perspective -- since Super Mario 64popularized 3D games 20 years ago. How is it these issues haven’t been ironed out over the last two decades? What’s really going on in games that have camera problems? Why is it apparently so difficult to make video game cameras just work?

We reached out to the developers behind third-person games with their own unique camera challenges -- including Final Fantasy XV, Gravity Rush 2, Abzu, Hitman, and A Hat in Time-- to find out.

The common answer: it’s harder than you’d think to make a great video game camera.

The underwater problem

Many developers run into problems making their cameras work in games that involve simply running around on the ground -- a 2D plane -- within a 3D space. Developer Giant Squid’s underwater exploration game Abzuadds another axis to that equation and a totally different set of control challenges -- problems that few developers of games with underwater controls have ever been able to solve.

“It’s an interesting problem doing a camera in an underwater game because you have this unique environment where the character can swim in all three dimensions of the space,” said Matt Nava, Giant Squid’s founder and Abzu’s director.

The developers looked at every other underwater game they could get their hands on, and Nava said the most common solution in the past has been for developers to restrict player movement by not letting the character’s vertical pitch extend too far up or down -- preventing them from doing a vertical flip, basically.

“One of the first things we did in Abzuwas decide, you know what, we’re going to try and solve this problem. We’re going to let the player do full loops, because that’s super fun and graceful,” Nava said. “We had to solve: what happens when you’re upside-down?”

What indeed? If the camera followed the character like it normally would, the camera would turn upside-down with the character, which would be nauseatingly disorienting. Instead, the game tracks multiple factors to detect when the player is attempting to do a loop or turn-around and automatically zooms the camera out to show the full arc. It tracks the character’s movement speed and the speed at which the character’s pitch is changing to determine how big the loop will be, and how far out the camera should move

Meanwhile, if the player steers the character in a half loop, leaving the character facing the opposite direction and upside-down, the camera automatically sets itself back behind the character while the character flips over on its own. During this animation the game also briefly wrests camera control away from the player -- often a cardinal sin in action games, but a necessary one here.

“We got it just right, so people didn’t really notice that there are moments in the game where steering doesn’t do anything,” Nava said.

Balancing for two types of players

Taking camera control away momentarily isn’t a perfect solution, though. More experienced players tend to move the camera manually more often than inexperienced ones who are less accustomed to steering characters through three-dimensional space. The veteran player wants full camera control, while the noob needs the camera to move automatically without them constantly adjusting it. Balancing between the two can be a massive headache.

Many games’ solution is to let players adjust the camera but impose a timer that automatically resets it behind the character again if the game doesn’t detect any input from the player for a set amount of time.

Issues can arise when that timer is too short or too long, especially in games where you move at high speeds, like driving games. If the camera takes too long to reset, inexperienced players might struggle with the controls. But if it resets too quickly, experienced players get annoyed feeling like they don’t have control over it. That’s one problem that plagued the late 2016 release The Last Guardian, a long-in-development game with lots of strengths but undeniably outdated controls and camera mechanics.

“I think that probably the most common thing that [developers] do wrong with the camera is they try to auto-correct it at the wrong time -- they do it when you really want it to let you control it yourself,” Nava said.

Square Enix’s 2016 Hitmangame does the exact opposite, to its great credit. “We are fairly atypical in the way that we actually allow the player to walk toward the camera,” said IO Interactive’s Jesper Hylling, Hitman’s lead game designer. “It’s much easier to control the character if you just look at his back all the time, and typically the direction you want to move is the direction you want to look. Except in Hitman, that’s not always the case.”

Often in Hitmanyou need to observe a target while assassin Agent 47 faces the other direction to seem nonchalant, or flee from pursuers while keeping an eye on them. “So having a camera that tries to reset itself behind the shoulders of the main character wouldn’t really work for our game,” Hylling said. “Also you get to do these cool moments where you walk away from explosions.”

That also makes it the level designers’ responsibility to direct players’ attention towards timed, scripted events in subtle ways without having to yank the camera controls away from them. For example, in the game’s Paris mission the main target is first spotted descending a staircase in the mansion’s lavish entrance hall just as players walk in. He’s hard to miss, and that’s deliberate, Hylling said.

The drawback, he conceded, is that some players -- especially less experienced gamers -- struggle with the controls because they’re used to cameras that correct themselves. But “it’s sometimes really, really hard to guess what the player actually wants to look at,” he said. “So we try to do that as little as possible.”

Camera controls as gameplay

The developers of Gravity Rush 2faced a similar obstacle, compounded by the game’s unique premise. The early 2017 PS4 game’s gravity-shifting gameplay lets you totally alter the main character’s perspective at any time and hurtle through the air at great speeds in any direction. No rule of physics or game design is sacred -- even the horizon can flip this way and that, proving disorienting for many players.

With the game’s totally unfettered range of movement, it would have been impossible for the developers to design a system that could predict where players wanted to look to move the camera there automatically. So they didn’t try. Instead, they left the camera controls entirely to the player.

“There were a lot of other challenges during game development, but designing the camera controls for this game is perhaps the challenge I remember most of all,” the game’s lead designer, Sony Computer Entertainment’s Junya Okura, said through a translator.

“For this game, we made the decision to enable free-flowing travel and complete autonomy over camera control -- these two principles superseded all else,” the game’s lead player action programmer, Toshitake Tsuchikura, explained. “If we programed the camera to move automatically, the unique gameplay experience we crafted for players would suffer significantly.” Although newer players might find it hard to get used to, he said, they thought this totally free system would “satisfy our biggest fans” and that players would feel a sense of progression from mastering the camera controls over time.

Even so, Okura said their main objective in designing the camera was “to prevent a feeling among action game players who usually don’t have to think about controlling the camera from a feeling of being forced to do something that they’re not used to.” Tsuchikura added that they wanted the camera to “follow players’ intentions faithfully.”

In other words, they wanted control over the camera to feel natural, even for people who aren’t used to doing so in a game that leaves it entirely up to the player. To help accomplish that they made Kat, the game’s lovable protagonist, the camera’s focal point at all times -- she never leaves the center of the screen.

Motion controls are another factor. By default, players can make fine camera adjustments by rotating the PS4’s DualShock controller, which Tsuchikura said “provides players with a feeling of true motion and a floating sensation.”

“Free swirling with an unfixed horizontal line sometimes gives you strong motion sickness,” he said. “On the other hand, a sense of naturally swinging movements as if you’re floating in the air in this game is achieved through the motion sensor detecting the hands’ movement. It’s essential for the floating experience that Gravity Rush 2is pursuing.”

Designing the camera’s behavior during Gravity Rush 2’s chaotic combat was its own challenge. Because enemies can appear all around Kat at any point, which the developers call “omnidirectional battles,” it remained important to let players retain full camera control during combat. But that also makes it difficult to track enemy locations, and more than one player has wonderedwhy the game doesn’t provide a way to lock onto enemies.

Okura said they considered adding a lock-on mechanic during the early phases of development on the original Gravity Rush,which launched on the portable PS Vita system in 2012. There were two sides to that internal debate, he said: on one hand, adding lock-on could make battles easier to manage, which would appeal to more players; but on the other hand, it would “destroy” the camera controls, the freedom of which is a core part of the series’ design. “The idea of trusting our players with autonomy matched with our design philosophy,” he explained.

“This idea may be old fashioned, but I believe that it’s one of the important elements of action games to experience the joy of overcoming some difficulty or stress and to be proud of the improvement of your skill,” Tsuchikura added. “Please don’t be prejudiced against the right stick. It’s one of the powerful weapons for action gamers!”

Breaking the line of sight

Even a comparatively simpler game like the in-development A Hat in Timefaces these problems. As a 3D platformer in the tradition of games like Super Mario 64and Banjo-Kazooie, AHiTwrestles with the same questions those pioneering games dealt with in the ‘90s.

“Even though the game has an idea of the things the player thinks are interesting, it can be wrong,” said Jonas Kaerlev, CEO of small Danish studio Gears for Breakfast. “That’s why, if at any time in A Hat in Timethe player moves the thumbstick, the game just says, ‘OK, we’ll chill out for a moment to let you do your thing, and then once we’ve figured out if you’re done with your thing, we’ll take over again.’”

The game reacts differently to varying amounts of input on the camera controls, including trying to detect accidental inputs in order to ignore them. But that’s an extremely resource-intensive process and can be taxing on the game’s overall systems, which Kaerlev said is why many modern platformers, like Super Mario 3D World, choose a fixed perspective rather than more traditional 3D camera controls.

“Sometimes you just want to focus on the basics and always keep the camera on what’s interesting,” he said. Plus, that lets you avoid breaking what he said is one of the “golden rules” of game cameras: “never break line of sight.” In other words, the player always needs to be able to see the character they’re controlling, which can be difficult to accomplish when players have full movement and camera control.

Traditional solutions for this vary. In Super Mario 64 the camera moves automatically to attempt to keep Mario in view, one of the many factors that made the 1997 classic’s pioneering 3D controls infamously wonky. The new The Legend of Zelda: The Breath of the Wild, on the other hand, handles obstacles like tree foliage by making them transparent if they would otherwise obscure your view of the hero, Link.

But Kaerlev said that can suck a game’s processing resources dry, especially as game graphics have become exponentially more complex over the years -- which is is why newer games (Breath of the Wildbeing an exception) tend to find other solutions.

A Hat in Timeuses a novel two-pronged approach. “Whenever we know the player is definitely, unquestionably going to have the camera be blocked by this thing, we put in the extra power [to turn the object transparent],” Kaerlev said. Other times, though, they simply draw the player character, Hat Kid, on top of the object as a silhouette, which accomplishes the same goal -- letting the player see the character at all times -- while using far less computing power.

Gears for Breakfast is a small studio, and Kaerlev is relatively inexperienced. But throughout development he’s been struck by the fact that developers of all sizes and backgrounds have wrestled with these issues throughout the history of video games -- but they’ve also all managed to come up with elegant solutions.

“When I started development on A Hat in TimeI had no idea how to make the camera,” Kaerlev said. “I just gave the player full control and said OK, whatever, do your thing. But I quickly found out that players don’t want full control. So I’ve sort of had to learn, as I’ve developed the game, that these kind of rules make a lot of sense. It’s kind of beautiful, in a way, that a lot of developers with very different backgrounds come to the same conclusions.”

Putting the camera back in players’ hands

The problems with Final Fantasy XV’s camera weren’t due to inattention. Hidemi Mizoguchi, the very engineer responsible for the game’s camera system, made obvious the staggering consideration that went into its development at every step.

Yet FFXV’s camera is plagued by all the issues that many game developers strive to avoid -- from objects in the game obscuring your view of the camera to control being yanked away by the game.

Mizoguchi said (through a translator) that some of their challenges stemmed from the game’s wildly varying scenarios. You’re not just running around as main character Noctis. The game has to keep track of four characters plus multiple enemies of different sizes during combat. Or you could be driving the Regalia down the highway, or riding chocobos through the forest. One of the developers’ goals was “to create a camera system that would guarantee quality in any situation,” which even in a far simpler game is a tall order.

Part of that involved tweaking the environment design to better accommodate the camera. “To get rid of any element that might have caused difficulty in exploration and battles, we adjusted background modeling and camera collision countless times while staff worked on the environment,” he said. “Even if props and background elements looked good as a part of a landscape, we adjusted and changed placements of any object without hesitation if it created stress during battle.” That type of deliberate, manual attention is important, he noted.

FFXVstraddles a weird line between an action and a role-playing game. You have full control during some battles. Other times the game moves the camera for you, often to showcase some gargantuan enemy. Mizoguchi said they knew the camera would have to be flexible, so they created basic, core camera code on top of which they could layer whatever additional functionality they needed for a given situation.

“By having this common functionality, the camera could receive control input by the player and control collision determination, while also making it possible to create cinematic framing for both still and dynamic moments and even blend the two together,” he said. “The camera is designed upon the premise that, at its basis, it belongs to the player.”

For all the game’s strengths, and for all this attention to detail, its camera is frequently cited as one of the game’s most problematic elements, by fans on places like Redditand GameFAQs, as well as by critics on sites like IGN, Polygon, Kotaku, Glixel, and countless others. The developers even attempted to address the camera issues in a day one patch, with debatable results.

The problems in this case seem to come down to sheer scope. It’s not just the size of the game, though, it’s any of the issues that developers face in fine-tuning the dozens of invisible factors that can affect the camera at any given point.

“Getting the battle system in FFXVinto its current shape required a long process of trial and error,” Mizoguchi said. “Even after we established a stylish battle in which the player can freely move around in the air, it still took time to adjust the camera so that it would not hinder the action as the places and situations changed. We continued to adjust right up to the time we submitted the master build.”

This is all so much more complex than any casual player might guess. And that’s by design. As Mizoguchi put it, “With a well-adjusted camera, players are able to comfortably play a game without noticing that a camera exists.” All the developers interviewed agreed that this is the number one identifier of a great video game camera.

It doesn’t take much to throw the camera out of whack -- a fact that, if nothing else, should make us all appreciate the great video game cameras in our lives all the more.

Mike Rougeau is a freelance journalist who lives in Los Angeles with his girlfriend and two dogs.