Pacifism isn’t the same as being a doormat. It doesn’t say he attacked anyone, just chased them out with a whip.

He is also reported to call his disciples to strap on their swords at one point though, and at least Peter took him literally. So he obviously wasn’t a “pacifism at all costs” kind of person.