Hello,

I am trying to design a program using a webcam, Arduino + absolute orientation IMU (Inertial Measurement Unit).

What I would like to build is an augmented reality application that, as the title says, has a series of 3D objects that are super-imposed on a live video feed. I want the objects to appear static as the webcam is rotated, and I will be using Arduino + a 6 axis absolute orientation IMU (BNO080) to track the rotation of the webcam. I’m running this on Windows.

I recognize that the IMU will have some drift, etc., which is (currently) a secondary issue. I am hoping I can do this using just the absolute orientation of the device, and placing the 3D objects at specific positions on a transparent sphere, which rotates to compensate for the rotation of the device so that the 3D objects appear fixed in space.

What I’d like to know, as the title says, is:

• Is this possible, or is there a fundamental flaw in this that I’m missing?
• Can you guide me to any similar examples, using processing, or any other solution (I do not want to use Android ARCore / Apple ARKit since they depend on surface mapping, which is a challenge here (smooth surfaces; nothing to get a tracking point / surface on)

I can get into details of how I’m envisioning the solution if anyone is curious; so far, I’ve used the Sparkfun IMU guide to make the serial cube visualizer. So I can get the IMU / Arduino to talk with Processing & pull in absolute orientation data (with almost no drift, and it remembers its orientation after power off, etc. It took some doing to get it calibrated, but once that was done, it’s pretty impressive!)

Please let me know if I need to offer any additional information. I’ve tried to be as clear as I can!
Thanks in advance

 

To add some more detail:
1) I plan to mount the IMU on the webcam, centered above the lens
2) The webcam will not translate. It is fixed, but has a limited rotation (yaw & pitch) (+/- 90 degrees around the X axis; +/- 30 degrees on the Y axis).
3) The Z-axis position will not change

Thank you!

P.S:. I've also cross-posted on the Processing forum, here:
https://discourse.processing.org/t/...c-on-a-webcam-feed-using-arduino-an-imu/10902

If there are any inputs there, I will link & quote them here as well. TIA