The short answer to this is rock that crumbles or breaks apart easily might be a sign that the rock dissolves easily, so it might leach unwanted minerals into the water.
Rocks in cold climates might freeze apart, so they might be safer - but you really need to know what type of rock it is. In general rocks that break apart easily should be avoided.
The rocks you can find in streams and on beaches, if they are rounded by water, are fairly safe to use in your fishtank. Rounded rocks look nicer in a tank too, in my opinion.
Rocks you find in nature should be disinfected to avoid harmful pests entering your tank. A good way to do this is by boiling the rock for at least 30 minutes and a bit longer if the rock is large (be very careful when you boil the rock as pressure might build up inside the rock and it might explode, causing injury to you or your surroundings).
