Caves for reptiles can be made of lots of materials and the ones you buy are often made of:
Plastic, often molded into shape and covered by sand or other materials to give a nice surface.
Styrofoam that is cut and shaped by hand and covered by spray glue and some surface material.
Plastic resin and fibers (glass fiber or wood fiber), molded and cut/shaped by hand with some surface material glued/sprayed on.
Papier-mâché made into the shape you want and covered with spray paint and some sealer on top.
Wood is used, often cut into shape and glued, and often in multiple layers painted and with some type of glued-on surface material.
Larger or more complex caves are often made of a combination of materials.
I am not sure about what types of quality/safety control these products undergo. Many of them are made in low cost countries.
If you google "diy reptile cave" you will find lots of tips on how to make them yourself.