This question involves many complicated halachic issues. In this forum, we can only outline the basis for our ruling. Our analysis presumes your description of the events, as you need to know what to do from your perspective, and this does not constitute a ruling of a din Torah.
Certainly, your agreement to pay was a mekach ta’ut (a transaction based on misrepresentation) and does not bind you. However, even without an agreement, when one uses another’s property for his benefit, it sometimes obligates him financially. Your presence at the beach did not cause the proprietor any loss, and there is a concept that one who benefits from his friend’s property under such conditions is exempt from paying.[1] However, since the proprietor clearly disallows use of his beach without payment, and since you agreed to pay for its use, it is logical that you be expected to pay for the benefits you received.[2] How to appraise the value of those two hours, taking into account the upsetting circumstances of having to leave the beach abruptly, is subjective; you can evaluate it better than we can. The subsequent unpleasantness in the office does not factor in because it occurred after the possible obligation took hold, just as, generally, the trouble and expense of adjudication is not factored in.
The next question is whether the return of the money was a valid mechilla (relinquishment of rights), a present, or neither. (We are operating under your assumption that the person you dealt with was the proprietor; otherwise, it is even less clear that the mechilla would be valid under these circumstances.) The Rama[3] cites Rabbeinu Yerucham’s suggestion that mechilla out of anger is invalid, as it is not done in a thought-out manner. From the halachic discussion on the matter, it appears that the application of this position depends on the particulars of the case.[4] Here, mechilla occurred with an action[5] by someone who realized that he would not be able to subsequently retrieve the money. In addition, despite his anger, the proprietor probably understood that, after deceiving you, the honorable thing was to refund all the money. Therefore, there is a strong case for assuming that this angry mechilla was valid.
Even if you technically owe the money, the matter is not simple. The K’tzot HaChoshen[6] says that when one owes money but the creditor has not asked for it, there is no practical obligation to pay. Admittedly, some disagree,[7] and his reasoning does not seem to apply to a case where the creditor cannot request the money (e.g., he does not have contact information). However, even if we say that the mechilla is invalid, it just means that he can reverse his refusal to receive payment. The status quo, though, is that until then, one is not obligated to pay. Thus, you may be able to rely on the likelihood that he has not actively decided that he desires payment. Since contacting him might reignite hard feelings, it is not necessarily a good idea to try to find out.
When one is holding someone else’s money, he does not have to go to the other person’s location to return it unless the money came to him as a favor or through a promise to pay.[8] Therefore, even if you decide to pay, you can at least wait until you pass by the beach again.
Due to a combination of the factors we mentioned (and a couple, possible others that we omitted), we do not feel that you are required to make efforts to return any part of the fee.