Interesting question. Generally speaking, you are responsible for something like that.
However, there may be a set of facts which would exempt you from responsibility. For instance, if the rocks hit an automobile, the damage to that automobile would likely be covered by the owner’s insurance. Additionally, if the rocks were the result of a failed cleanup by another entity, they may be responsible.
Either way, remind me not to ride behind you.