AITA for ditching a ride upon finding out the driver had an eyepatch?

I’m not one to make a big fuss about little things in the service industry. In the modern parlance, you might say I’m not a Karen, although I’m a “Care-In” Nurse so I suppose technically I am. Regardless, despite not liking making problems for people doing their jobs, I will do so if I think they are putting me in danger, as was the case tonight.

Basically, I just booked an uber to take me and my wife to the Citrus Store as we are looking to make some Moscow Mules tonight. As we got into the car, I noticed the driver was wearing an eyepatch. It took me a second to make sense of it, since the only times I had actually seen a human wearing an eyepatch in real life is when I played a pirate in a school play and also when allowed my step daughter to provoke a bee hive and she got stung on the eye. But after my wife and I exchanged a look, it hit me that the man was probably blind in one eye, temporarily or permanently.

I didn’t even close the door, I just said “this is unacceptable” and got out of the car and pulled my wife out with me. As we walked away, he rolled down the window and asked what was wrong. I asked him if he wanted the truth or if he wanted me to spare his feelings. He chose the former.

I told him it was a safety hazard for us to get in a car with a man with one eye obscured and that he should be ashamed of himself, and made a pretty mean pirate joke. He got pretty mad at that but said “thanks for the money” and drove off.

My wife said I was in the right, but that I should have just not said anything to him as he has a right to make a living, and that he probably was as safe a driver as any. And she said my joke was too far. I feel like she might be right, but for some reason I still feel justified. Am I a prick?

EDIT: the joke was “you must be crazy if you think we are boarding the Queen Anne’s Revenge with Italian Blackbeard at the wheel”