This should fix #38.
The problem was that there were several different situations where the highlighting should be cleared, all involving different parts in the code (and even code from girara). I think it's better to have a single point where the clearing happens and one such common event indicating that the links should be cleared is when the inputbar is hidden, i.e. we have either aborted the action or followed/displayed/copied the link. So I have used a hide-event handler on the inputbar where the clearing happens.
While here I also did some refactoring.