CAD Clones: Creating a Digital Twin for pib
For our humanoid robot pib, we have successfully created and implemented a digital twin. But what exactly is a digital twin and what are the benefits of having one?
What is a digital twin?
For our purposes, a digital twin is a specific kind of simulation: It is the virtual representation of a physical object which it can be tightly coupled to. The goal of the digital twin is for the physical robot and the virtual model to react to any interaction in the exact same way. This allows having an accurate approximation of the robot without the drawbacks of having a physical copy.
Our expectations of pib’s digital twin are for it to:
- Look and behave similarly to its physical representation
- Be controllable in the same way as its physical representation
- Be generated from our Onshape CAD files
- Be up to date
- Be somewhat easy to generate
What is the goal of having a digital twin?
Our goal was to have a purely digital version of pib that behaves similarly to the physical pib and that is easily and widely accessible. With that, we will be able to eliminate some of the hardware problems that come up while developing or interacting with pib.
With Cerebra, pib’s very own control software, anyone can simply open a web interface and take control of pib, moving its motors or using the voice assistant for example. We are working on the physical pib and developing Cerebra at the same time which can cause frustrations. When implementing a new element that ends up not working as it should, there are many possibilities – it could be the element itself, it could be the framework, the internet connection, the motor, the cable… so we have to spend a lot of time and effort trying to find the error when with a digital twin, it could be so much easier. If you can get a purely virtual version to behave and to be controlled in the same way as the physical one, these types of errors can be found and removed much faster.
Reinforcement Learning in a virtual environment
Every slider in Cerebra should steer the physical pib the same way as the virtual one so that the user can get a good idea of how pib behaves in real life. One major benefit of having a digital twin is the possibility of running reinforcement learnings or pre-training machine learning algorithms on virtual versions of your robot. For example, if we want pib to be able to pick up a cup and recognize it, we could train it on the physical robot and it might break down after the first attempt because the motors give in. Therefore, it is a lot cheaper and more time-efficient to do all the pre-training in a very similar virtual environment. This way, we can load a copy of pib into our simulation and run as many trainings as required until the robot knows how to pick up a cup. Next, we take the results and load them onto a physical pib which then will already know how to pick up the cup.
The digital twin is also a great benefit for our pib@school program where students print and build their own version of pib. It offers extra possibilities for a group of students to work on pib at the same time. Also, anyone who is mainly interested in controlling the robot can work with the digital twin without building a complete pib first.
The first obstacle: Going from .stl to .urdf
For us, it is very important that pib’s digital twin is generated from our CAD files. CAD stands for Computer Assisted Design and our program of choice is Onshape where you can access our 3D model of pib. The model is comprised of .stl files which we now want to turn into a simulatable copy of itself.
.stl files are used for designing parts and printing them – however, they aren’t ideal for usage in simulations. The .urdf file format (Unified Robot Description Format) promises a solution.
Keeping in mind that pib is an open source project and the CAD files change almost constantly, it makes it more difficult for the digital twin to always remain accurate. That is why we had to come up with an easy solution to create it. Our goal is to have an automated pipeline so we can just press a button to generate a new copy of our digital twin.
.stls are pretty much only numbers whereas .urdf files have a very specific format. They are .xml based. The .stl files are broken up into two parts: links and joints. For example, the forearm is a link and so is the upper arm. The joint is the connection between these two – it’s not a real part but the program knows there is a connection and it uses this information to make sure the parts rotate or move like they are supposed to. Putting all these joints and links together, you get a tree shape.
However, the “unified” part of .urdf does not always hold its promise. It works great for Gazebo but there are other simulation softwares that have their own file format such as Webots which is not directly compatible to .urdf. With MuJoCo, you cannot directly use .urdf files but they do offer some basic compatibility: The MuJoCo format can handle URDF by wrapping a header around. Going from .urdf to .porto requires a more elaborate conversion but luckily, there are tools to help with conversions and exports: onshape2robot offers an interface that converts Onshape assemblies to .urdf and urdf2webots converts these .urdfs to .porto files.
So what’s next?
We want people to be able to use these digital twins in a way that actually makes it easier for them to do reinforcement learnings. Therefore, we are working on improving the files, making it easier to export them and to use them.
Meaning: We are currently creating a digital twin for MuJoCo which means expanding the URDF by adding colliders. We are also setting up a pipeline for exporting digital twins from our CAD repo to the file types of various simulation programs. Next, we will publish the digital twin and gymnasium (RL-training) as open source.
For anyone wanting to try out pib’s digital twin, our Webots package is installed by default when downloading pib’s software. The software setup can be found here: GitHub – pib-rocks/pib-backend
We recommend either using a RaspberryPi or a virtual machine running Ubuntu.
What can I do?
We are always looking for new co-developers to join our community and support us in these endeavors. Not sure what you can do in our community? For example, one our community members has already optimized a .proto file for webots and even made it controllable through cerebra. Join us now on Discord to pick up your own project!
