I literally just ordered a GoPro Fusion last night so that I can create 3D captures of hiking trails using photogrammetry. I will then use the 3D trail models for reinforcement learning for my off road robot. [2]
The basic pipeline is images to Meshroom (perhaps with pre-processing since Meshroom doesn't seem to support 360 cameras), then instant-meshes to reduce the poly count of the mesh, Blender to map the texture on to the new low poly mesh, save as .glb file in Blender, then open in habitat-sim for machine learning. Blender was the hardest part to learn, but tutorials on youtube walk through the whole process.
Interestingly the GoPro Fusion is pretty cheap now, as GoPro has a new model which is apparently not much of an improvement. So while the new model is $499, the Fusion is currently $179 on Amazon.
Previously I have tried photogrammetry of trails with cell phone video. It works really well but the narrow field of view of the cell phone camera compared to the whole 360 degree scene means the resulting 3D reconstruction has lots of holes, and the model is only well reconstructed immediately around the trail - wider terrain is missed.
The 360 camera captures all angles, front and back, sides and top. It should make for some fun immersive video to throw on youtube but it will also be great for photogrammetry. I found this research paper [3] which supports my thinking that 360 cameras are useful for photogrammetry.
Would love to hear more about your project. Sounds very cool.
We're working on some computer vision problems (we plan to open-source soon). Perhaps there's an opportunity to collaborate? https://www.trekview.org/greenhouse/
They that looks like something I want to do too! I like the idea of my robot being able to identify all the native plants around it. My mom is a landscape designer and she knows the latin names for almost every plant we come across, so it's something I want to support in my robot. I'd love to collaborate! I will email you. :)
I literally just ordered a GoPro Fusion last night so that I can create 3D captures of hiking trails using photogrammetry. I will then use the 3D trail models for reinforcement learning for my off road robot. [2]
The basic pipeline is images to Meshroom (perhaps with pre-processing since Meshroom doesn't seem to support 360 cameras), then instant-meshes to reduce the poly count of the mesh, Blender to map the texture on to the new low poly mesh, save as .glb file in Blender, then open in habitat-sim for machine learning. Blender was the hardest part to learn, but tutorials on youtube walk through the whole process.
Interestingly the GoPro Fusion is pretty cheap now, as GoPro has a new model which is apparently not much of an improvement. So while the new model is $499, the Fusion is currently $179 on Amazon.
Previously I have tried photogrammetry of trails with cell phone video. It works really well but the narrow field of view of the cell phone camera compared to the whole 360 degree scene means the resulting 3D reconstruction has lots of holes, and the model is only well reconstructed immediately around the trail - wider terrain is missed.
The 360 camera captures all angles, front and back, sides and top. It should make for some fun immersive video to throw on youtube but it will also be great for photogrammetry. I found this research paper [3] which supports my thinking that 360 cameras are useful for photogrammetry.
[1] https://www.trekview.org/trek-pack/
[2] https://reboot.love/t/new-cameras-on-rover/
[3] https://www.int-arch-photogramm-remote-sens-spatial-inf-sci....