I'm working on a 3D game engine and I'm having some trouble with the math behind player orientations / rotations.
My coordinate system is right-handed with (1,0,0) = right,(0,1,0) = up and (0,0,-1) = forward.
I have the following data available for the player character:
viewOrientation - A quaternion representing the way the player is currently facing
upDirection - A vector representing the up-direction of the player, by default it's the same as the world's up vector (0,1,0), but the player is able to walk on walls as well, in which case the up direction would be perpendicular to the surface below the player
forwardDirection,rightDirection - If the up-direction is changed, these change respectively. By default all three direction vectors correspond to the world axes
upRotation - Quaternion representing the rotation from the world's up vector to the upDirection of the player
Now, what I need to do is:
- Limit the player's pitch axis to 90 degree up and down (Can't look further than straight down / up)
- Extract a movement direction from the player's orientation
As long as the player's direction vectors correspond to the world axes, both of these are easy to implement.
But problems arise if the up direction is anything else but (0,1,0).
Let's start with limiting the player's pitch axis.
Usually I would just transform the viewOrientation by the inverse of the upRotation, convert that to euler angles, apply the limit and then convert them back to a quaternion and transform them back via the upRotation.
Unfortunately this is prone to errors, since different euler angles can represent the same rotation. For example if I have the euler angles (pitch=0,yaw=0,roll=0), then pitch is obviously 0. On the other hand, if I have the euler angles (180,180,180), which is essentially the same rotation, I get the pitch as 180, which is obviously not what I want. Since I'm working with quaternions, and convert them to euler angles, I can't be sure which ones I get.
Are there any alternative ways to do this?
As for the second problem, it's supposed to be like this:
If the player looks straight forward, he's supposed to move at full speed in that direction. If he looks 45 degree downwards, he's supposed to move at half the speed, but no downwards force should be applied, so the direction vector should stay the same.
Problem is, I'm unsure on how to extract the direction vector from the viewOrientation. Again, usually I'd just grab the euler angles same way as above, remove the pitch and roll components, and use the yaw, but the above problem applies here as well.
I'm at a loss, any nudge in the right direction would be very much appreciated.