How to Tell Kinect its 3D Location

The real 3D location of Kinect relative to the TV is very important for several aspects of the Kinect Swords system, particularly "head-look", where the orientation of your head affects the game camera angle. With the TV's dimensions and real location relative to Kinect, you can determine where the player's body is relative to the TV, including where exactly on-screen the player is facing. Without this knowledge, the system would have to assume "forward and level" is the center of the head-look deadzone. With this knowledge, (among other abilities now available) you can put the head-look deadzone truly in the middle of the TV screen.

I mentioned in the Kinect + Swords presentation and in my Kinect Preliminary Analysis that Kinect is capable of determining its location in 3D space relative to the TV, but haven't explicitly stated how it can do this... until now.

Kinect location calibration process as seen by the player
(click image to see full-size)

The vertical bar is used to find the approximate "pixel to inch" or "pixel to centimeter" ratio. This is the ratio of how big a pixel on-screen is compared to a real-world unit of measurement. Once the system has that, it can use the ratio to determine real-world distances based on acquiring those distances in pixels.

Assuming...

• the TV's aspect ratio is known and correct,
• the pixels of the TV are roughly square,
• the TV is set up so that the edges of the 360's video signal line up with the edges of the screen, and
• the TV and Kinect are both roughly level and facing the same lateral direction

...the following steps can be taken to determine Kinect's real 3D location relative to the TV:

1. Change the width of the centered yellow bar until it's as wide as Kinect is in real life. 
2. Compare this pixel width to the known real width of Kinect to obtain the approximate pixel-to-inch ratio.
3. Use the pixel-to-inch ratio and pixel width of Kinect to determine the real width of the TV.
4. Use known TV aspect ratio to obtain the real height of the TV.
5. Change the height of the red bar until it it's approximately the same height as the distance from the bottom of the screen to the top of the Kinect sensor (if Kinect is above the TV, this bar is blue, at the top of the screen, and measures distance upward).
6. Use this pixel distance to the top of Kinect and the pixel-to-inch ratio to determine the real world vertical offset (how far Kinect is above/below the TV).
7. The depth offset of Kinect (away from the TV) is usually trivial, but if it's significant, this can be configured separately with a similar method as step 5.