The short answer to this is rocks that crumbles or break apart easily might be a sign that the rock dissolves easily so they might leach unwanted minerals into the water.
Rocks in cold climates might freeze apart so they might be safer but you realy need to know what type of rock it is,in general rocks that breaks apart easily should be avoided.
The rocks you can find in streams and on beaches if they are rounded by water is fairly safe to use in your fishtank,rounded rocks look nicer in a tank too in my opinion.
Rocks you find in nature should be disinfected to avoid harmful pests entering your tank,a good way to do this is by boiling the rock for atleast 30 minutes and a bit longer if the rock is large(be very careful when you boil the rock as pressure might build up inside the rock and it might explode causing injury to you or your suroundings).