Every actor feels different about it, I guess, depends also how loved or hated that specific film was that they're being associated with. But, from interviews, and from a logical point of view, you can see they usually do their best to pick different roles and change their image. For the good of their own career, and for their own pleasure as artists. Robert Pattinson is amongst the most famous examples of this, as he's been pretty vocal about not wanting to be associated exclusively to the infamous vampire.

I don't think they were particularly good in said movies. Those movies were just hyped up a lot, so they stick more in our memories

I agree! Hugh Jackman came to my country Brazil, and found his voice actor, it was a beautiful moment, and both said they felt honored to be part of Wolverine's history.

Loki’s Lil Butter Knife ?

Sometimes I do that too