 |
| Kinect Sword (Very Rough Concept Model) |
This post serves as a more thorough explanation of the thinking behind the "Kinect Swords". It's designed to introduce people to the high-concepts involved, but would also be an interesting read for those that want to know where the idea came from.
I devised a gaming control method where we can finally treat a player's hands like hands without being so granular that we control individual fingers.
My control method is derived partially from how the computer mouse works. A simple computer mouse has three basic functions in a modern operating system.
- Navigation (of the 2D plane of the screen)
- Selection (left-clicking an object on-screen to mark the object as "selected")
- Interaction (right-clicking an object on screen to open a menu of options)
NAVIGATION
Let's assume that we know the rough locations of the player's hands in 3D space, such as is made possible with Kinect. I'll get more into the specifics of how to determine the 3D orientation of the player's hands later, but suffice to say, if we know both the 3D position and 3D orientation of the player's hands, we have the ability to "navigate" the virtual character's hands in virtual space.
What's more important is how the player navigates the character in a 3D environment. A computer mouse navigates the 2D plane of a screen, but when you think about it, human beings also navigate a 2D plane in real life, the surface of the earth. Sure we may jump, crouch, and climb, but in general "vertical" movement is very limited and the "lateral" plane is where we navigate most.
SELECTION
When you get right down to the most basic actions we do with other objects in real life, the very first thing we do is select an object to do something with. In the crib, if there's a new toy, we navigate (crawl) over to the toy and then "select" the toy by grabbing it.
Many objects have functions that don't involve anything other than variations on navigating and selecting the object. If you are putting a real-life paper in a folder inside of a filing cabinet, you find ("navigate to") the paper, grab ("select") the paper, find ("navigate to") the folder, and place ("deselect") the paper inside the folder. That's the "mental model" of how the modern computer's GUI-driven (Graphic User Interface) Operating System works. To move a file on a computer into a folder you hover over ("navigate to") the file, left-click ("select") hold (grab) the file, drag the file to ("navigate to") the folder, and release left-click ("deselect") inside the folder.
These types of mental models can be taken advantage of in games with 3D environments by understanding and utilizing a "grab" function (provided we're capable of "navigating" the character's hands freely).
INTERACTION
One of the goals of this control method is to make actions in a game feel more "natural" without significantly negatively impacting the gameplay experience. The purpose of doing this is partially to make the experience more immersive, but it's mostly so that it's much easier for new gamers to learn the controls by connecting each action in-game to the mental model of how that action would be done in real life.
Every commonplace object in the world has generally one non-contextual "interaction".
- A gun has a primary fire trigger,
- A pen has a button that enables/disables the pen tip,
- A can of soda can only ever be opened once,
- etc.
Any other functions of an object can be described as "contextual". The types of contexts are "broad" and "deep". "Deep" means that you must trigger other functions first to reach the desired function, and "broad" means that many options are available simultaneously at a given "depth".
For example, a keyboard has at first-glance a "broad" variety of functions for the depth of "wanting to use the computer". Further depths exist for the keyboard, but typically for a game a keyboard object is inseparable in concept from its connected greatly-simplified in-game computer (the object you actually use in-game is typically "the computer"), so we don't care about those functions. That is, many objects with "broad" functionality generally in a game have those functions below a certain "depth", and many of those depths and functions (such as emptying the Recycle Bin) might not be necessary in a game in the first place.
Most objects in games are not as complicated as a computer. Let's look at a few examples of a much more common game object, a gun.
Pistol Reloading
A pistol may have alternate functions such as reloading, but those can be simplified down to single contextual interactions by thinking about the mental model of how you accomplish those tasks. For example, reloading should be available at any time, but typically is compulsory once you run out of ammo in your current clip/magazine.
The act of reloading requires some amount of movement on the wielder's part, and the compulsory form of reloading might be a bit slower if you're attempting to reload with one hand than if you're reloading with two hands. This movement can be automated for the compulsory form of reloading (meaning it automatically happens if you attempt to fire while out of ammo), but the voluntary form should involve some kind of voluntary action by the player.
Since we know the locations of the player's hands, we want the player to be capable of (while holding the gun as normal in say their right hand) grabbing a magazine from their belt with their left hand and jamming it into the bottom of the pistol (slamming the new magazine into place), just like the real wielder of the pistol would.
Grenade Launcher Switching
Another alternate function for a gun might be switching between concussive, incendiary, and acid rounds for a futuristic grenade launcher. In a game (for the sake of visibility in the UI), this switch might be an unrealistically-large visible switch on top of the gun (perhaps sitting behind the ammo count display). If this gun were real, the wielder would operate this switch with their off-hand by grabbing the switch and moving it to a new position. In a game, since we know the locations of the player's hands, we want the player to be able to do the same thing using the mental model of how the grenade launcher might work in real life.
Assault Rifle Alt-Firing
Another alternate function might an always-accessible grenade-launching alt-fire on an assault rifle. Think about the mental model of how this gun might work in real life. While it's possible that another trigger might exist lower than the normal trigger, it might be far easier to handle an alt-fire trigger if it were somewhere else on the gun, such as farther down the barrel. If this gun were real, the other trigger would be triggered with the wielder's other hand, which is convenient since many guns are two-handed anyway. In a game using this control method, it would use the same mental model as the real thing and work the same way (which is the definition of intuitive). The player would hold onto the gun with their off-hand.
In General
All of these guns have "broad" functionalities that in games are typically assigned to separate buttons on a controller. I've shown that by thinking about the mental model of how these guns work, it's possible to reassign these functions to contextual "interactions" made possible by the ability to "navigate" and "grab" with the character's hands.
Isn't it surprising: for all of these examples, each hand is only ever doing one non-contextual thing at a time with the object it's grabbing.
KINECT SWORDS
The control method I've been talking about is a controller specifically designed to augment Kinect's abilities in order to get closer to what I call "Full Motion" (the tracking of the player's entire body to control a character). While you're reeling from the idea that a controller would be used with Kinect, realize that there's nothing functionally wrong with doing so. That's a lie perpetrated by Kinect's marketing tagline. The controller's called a "Kinect Sword", and it's essentially a wireless Wii Nunchuck with rumble, a magnetometer, and a gyroscope. You hold a Kinect Sword in each hand, and they act as hand-analogues that allow you to directly control your character's hands.
They're called "swords" because like wielding a real sword, the orientation of your hand is paramount. If you don't know the orientation of the player's hand, any virtual "sword" that the player wields would have to be always oriented along something else, such as the direction of the player's forearm, which would really suck in battle.
The Kinect Sword determines the orientation of the player's hand using the built-in magnetometer and gyroscope. The joystick on top of the Sword gives the player complete analog control of the character's movement in the lateral plane. The Sword has rumble to give the player a literal connection to the character's sense of touch (more on that later). Finally, a Gamecube-style analog trigger and a digital button on the Sword trigger the "grab" and "interact" functions we discussed earlier, respectively. A separate digital button is free for developer use.
The major differences between the Wii Nunchuck and Kinect Sword in form are that the "interact" button is positioned on the face of the controller (within easy range of the player's thumb), the "grab" button is an analog trigger, and the joystick is an analog thumbplate (similar to the thumbplate on the 3DS). The "interact" button is on the face of the controller to make repetitive presses of the "interact" button less tiring (in the same way the Wii Remote's 'A' button is less tiring for this purpose), however if the player prefers it, the "interact" button can swap roles with the other digital button (some find this more comfortable). The "grab" button is an analog trigger because there are degrees that you might want to grip something, such as if you're rappelling down a rope.
The Sword's rumble feature is not intended to be used in the same way rumble is used traditionally. Instead, rumble serves as a direct connection to the character's sense of "touch". Rumble would be used primarily as an indicator to the player that the character is touching something that can be grabbed or interacted with. Not all objects have to be "grabbed" to be "interacted" with (such as a computer keyboard) and some don't even have to be "interacted" with to be "touched" (such as a panic button on a wall). This would be a form of haptic (touch) interface that can even use varied rumble to inform the player of when the best time to grab something is.
PLEASE NOTE
Camera Control
If you wish to comment on this topic with concerns about controlling the player's camera, realize that now that the technology behind Avatar Kinect is solid, it's proven that Kinect can reasonably track the orientation of the player's head. By using a graduated head-orientation dead-zone similar to the graduated dead-zones Wii FPSs use for aiming the pointer, it's fully plausible for the player's head orientation to control camera-aim independently from gun-aim.
Object Control
If you wish to comment on this topic with concerns about a specific object that you believe does not fit into this "navigate-grab-interact" control paradigm, please first carefully consider the mental model of how that object works in real life. For example, if you're concerned about what aiming a gun would be like, realize that in real life aiming a gun involves navigating the gun to the position and orientation that you want the gun to fire from. For two-handed guns, aiming would rely on the imaginary line between the player's hands. This unintentionally opens up the ability to fire around corners (albeit blindly, although a cyber-link might be part of the game lore that allows you to see through one eye as if it were looking down the gun barrel).
Body Movement
If you wish to comment on this topic with concerns about the amount of body-movement that would be involved with the Kinect Swords, consider that maybe your main problem is with Kinect itself and not the Kinect Swords in particular. If so, that kind of comment probably doesn't belong in this particular discussion, as odds are you're already hard set against this system since you're set against Kinect.
The Kinect Swords serve as a way to get "core" FPS gamers up and moving, but also serve as a way to introduce "new" gamers and "casual" gamers to the idea of an FPS with something much closer to an NUI (Natural User Interface) than a traditional FPS can offer. While it is true that you will have to jump, crouch, and lean with actual body movements now with Kinect, the system could be implemented in such a way that those more common body-actions are possible even while you're on the couch (that is, Kinect would detect if you're on the couch and act accordingly).
Realize that most of the time in real life, you aren't required to hold your hands out in front of you for most actions you do in reality, so there's no reason a game would expect you to hold your hands out constantly in a game. That is, it's tiring to do a real action with your arms out constantly in real life, so it doesn't make sense that a game would expect you to have your arms out constantly. That said, for things that it would make sense to have your arms out in reality, it would likewise make sense for a game to expect you to have your arms out. Thinking about how actions work in reality is again key to understanding how to design for Full Motion.
Menus
If you wish to comment on this topic with concerns about accessing menus and other subscreens, there's an auxiliary button (the "X" button) reserved for that purpose, which between both hands makes two auxiliary buttons. Additionally, not all subscreens have to be assigned to an auxiliary button. For example, an inventory subscreen could be accessed by "interacting" with the character's backpack (that is, by touching the virtual backpack with your real hand and tapping the "interact" button).
0 comments:
Post a Comment