There is no problem leaving the tour early. Please simply just let the guide know you are leaving so they are aware and not waiting for you before departing for the next stop.