Render to Oculus not working + acceleration question

[kubiak54]

Hello there,
first of all thank you for this beautiful piece of code, amazing! :)
I am a student trying to develop "unlimited" positional system for oculus for my bachelor thesis.
I would love to attache RGB camera on Oculus and when Oculus gets out of positional tracker, computing 3D position with RGB camera and some SLAM algorithm (currently I am using S-PTAM with 2x 170 deg cameras on oculus).
Just the background.
Anyway, I am successfully running your codes and seeing Oculus in Rviz however, Render to Oculus does not work - it just makes white background on oculus screen and does nothing then.
It would be amazing if you could help me with this :)
And one more question, is there anyway to get acceleration values (and maybe other as well?) from Oculus IMU to rviz as well?
Thank you!!

Jakub

[kubelvla]

Hi,

we (CTU in Prague) bought the DK2 for similar purposes - the idea is performing robotic manipulation from the view of our mobile robot. Thanks a lot to the author of this plugin, it helps a lot.
At the moment, we struggle with similar problem, the plugin loads and seems to be running, but the oculus window "renders" only background color of the rviz scene.

It is probably caused by hardware incompatibility - we have an old Dell notebook with NVidia GF119M [NVS 4200M] graphics card and this one works fine - the oculus renders the scene correctly (yet at low FPS rate). However, we bought a better laptop for the oculus and unfortunately, there is the same problem Jakub describes. It is equipped with NVidia Quadro K2100M. Kernel and driver are identical on both systems (Nvidia 352.63, kernel 3.13.0-75, ROS indigo).

The problem with the graphics card goes probably deeper, even the basic rift config dialog demo fails to draw textures of the table (screenshot included) even though the oculus world demo runs without problems.

I also include screenshot of the screen with rviz, there is definitely some connection between rviz and oculus screen because the background color changes appropriately and at some special angles, one of the eyes changes to a color of an object present in the scene (it's more or less random).

In this screenshot, I rotated the oculus screen, but the behaviour is still the same.

Needless to say I tried all possible combinations of nvidia drivers, including the open source one, the behaviour is the same everywhere (only difference is that the open-source driver fails to run the standard ovr demos).

So, I realize that you probably won't be able to solve these hardware related issues, but if anyone has any hints what to try, I'll be happy to try them and report results. Thanks a lot,

Vladimir Kubelka (kubelvla@fel.cvut.cz)

P.S. Log from rviz:

log_rviz.txt

[kubiak54]

Hi Vladimir, hello from Slovakia, TUKE :)

After many days of struggle I found the problem.
The problem is in distortion (correction) of images for oculus - Ogre tries to do it trough its .compositor .material .frag (all the things in ogre_media folder) and this (I have no idea why) just is not working in my machine (and yours too as I can see).
To disable this distortion just comment this line "m_compositors[i]->setEnabled(true);" in ogre_oculus.cpp. Now everything should be working just fine except the images are not corrected - but working.
I am still researching how to make correction work though but did not find solution yet - will write on Ogre forum, many suggestions welcome.

Regarding my need of raw acceleration data, I managed to do this as well - here is the method for ogre_oculus.cpp sending these data:

Ogre::Vector3 Oculus::getRawSensorData() //dont forget to add this to header file as well
{
ovrTrackingState state = ovrHmd_GetTrackingState(m_hmd, 0.0);
ovrVector3f AngularVelocity = state.HeadPose.AngularVelocity;
ovrVector3f LinearVelocity = state.HeadPose.LinearVelocity;
ovrVector3f AngularAcceleration = state.HeadPose.AngularAcceleration;
ovrVector3f LinearAcceleration = state.HeadPose.LinearAcceleration;
ovrVector3f Magnetometer = state.RawSensorData.Magnetometer;
return Ogre::Vector3 (LinearAcceleration.x,LinearAcceleration.y,LinearAcceleration.z); // return what ever you would like
}

Regarding Oculus speed, I managed to get it working a lot smoother and better by setting frame rate of Rviz to 60 (or 75 - this is Oculus native frequency) and running window in separate X screen (very important!) according to this article https://codelab.wordpress.com/2015/04/02/proper-oculus-rift-dk2-setup-on-gnulinux/.

Hopefully this helps.
Jakub

[kubelvla]

Jakub,

thanks a lot, it works :)

Unfortunately, as a phd student, I don't have enough time to sit and learn computer graphics magic (I know nothing about), so this helps a lot! My plan was to get some student to do this as a project (and that kind of actually happened :) The lack of distortion is funny, but usable.

Anyway, if you get some advice from the ogre community, it would be great to post it here... Nevertheless, thanks a lot and cheers to Kosice!

Vlada

[rebhuhnd]

@kubiak54
Author here, any way you could generate a pull request with the modified ogre_oculus.cpp and ogre_oculus.hpp with the getRawSensorData function in it? Seems like a good thing to have if it's not intrusive.