If you have been asked by the editor of a peer-reviewed journal to referee a paper, you should be honored. That you were asked is testimony that you have risen to a position of some prominence in your field. If the editor did not believe that you could do the job, you would not have been asked. This is one price you pay for prominence.