I’m currently working on a VR application that lets users manipulate objects using a controller. The idea is that users can translate, scale, or rotate an object by pointing at it with a “laser” from the controller. I’ve gotten the individual features to work nicely in isolation, but I’m running into some major headaches when trying to combine them.
Here’s what I’m trying to do: When the user points at an object and pulls the trigger, they should be able to drag it around in space. If they move the joystick left or right, the object should rotate around its local y-axis. And if they push the joystick up or down, it should scale based on the point where the laser hits the object. Simple enough, right?
The problem is that when I try to string these actions together, everything falls apart. Like, if I scale the object and then try to drag it, it doesn’t follow the laser as it should. Or if I translate the object, it stops rotating correctly or scales from the wrong point. It’s super frustrating because it feels like I’m missing something fundamental about how transformations should be applied together.
Here’s what I’m currently doing in my code: I save the object’s transform and the hit location (where the laser strikes the object) on the first update. Then I create a transform based on the current operation—like translating to the origin for scaling, then reverting back to the hit point. Finally, I multiply the previous transform by this new one and apply it to the object.
But I can’t shake the feeling that there’s something off in my approach. Should I be keeping track of translations, rotations, and scales separately and combining them at the end instead of applying them immediately? Or is there a better way to handle it that could allow all these transformations to work together smoothly without any hiccups? I’m hoping to hear from anyone who’s faced similar issues or has insights into how to manage these transformations in a VR context.
Sounds like you’re deep in the weeds with those transformations! I totally get how messy it can get when you’re trying to combine multiple actions in VR. From what you described, it looks like the main issue is how transformations are compounded, especially in 3D space.
Here’s a thought: instead of immediately applying translations and rotations to your object in real-time, you might want to maintain separate states for translation, rotation, and scale. You can track the changes based on user input and then apply them all at once after the user is done making adjustments. This way, you can prevent one transformation from messing up another.
For example, when the user starts pulling the trigger, you can lock the current state and start accumulating transformations (like translation, rotation, and scaling) based on the user’s joystick movements. Then when the user releases the trigger, you apply all the accumulated transformations together. This should help keep things more predictable!
Also, consider using a local coordinate space when transforming the object. When you want to rotate, make sure you’re rotating around the object’s local axis rather than the world’s axis. This is really important for ensuring that everything feels intuitive from the user’s perspective.
Lastly, it might help to visualize the transformations you’re applying. You can debug by drawing the transformation axes (like a gizmo) on your object while you’re manipulating it. This way, you can track exactly where the scaling and rotation are happening relative to the object’s local space.
Hang in there! Combining transformations can be tricky, especially in 3D, but you’ll get the hang of it with a bit more tweaking!
When combining translation, rotation, and scaling in VR, it’s generally best practice to manage these transformations separately and clearly. Instead of continually applying transformations on top of each other in an incremental manner, maintain isolated variables—such as position vectors, quaternion rotations, and uniform or non-uniform scaling factors. That means, each frame, you compute the final transform from these separate components explicitly. Specifically, translation and rotation can be tracked as separate states that you apply in a defined and predictable order, usually rotation followed by translation, and scaling can be managed relative to a consistent pivot (such as your initial laser hit point or object’s center) rather than recalculated every update based on the current cursor location.
Additionally, when scaling from the laser point, it’s crucial to calculate the scaling pivot in the object’s local coordinate system rather than world coordinates. Convert the initial strike point on the object to a local point and store it; this ensures that scaling behaves intuitively even after prior rotations or translations have occurred. Each frame, reconstruct the object’s transform by applying scale first, then rotation, and lastly translation. By keeping a clean separation between each transformation axis and consistently applying your transformations in the correct order (scale → rotate → translate), you’ll achieve stable and intuitive behavior, and prevent compounded transformation drift or undesired interaction behaviors.