Don't worry. It is natural that a dog will not want to pee or poop in the same place he lives. Just take him out once every hour or two and give him a chance to go. Then praise him when he does. He will understand very quickly that is what you expect him to do. Once he gets the idea you can lengthen the time to about every 4 hours. As the days go by pay attention to your dogs body language. He will "tell" you when he needs to go outside. My dog just sits in front of me and gives me eye contact as if he is sending a message. When he does that, I know he is asking to go outside. Some dogs are vocal. He might growl, but don't take it the wrong way. A growl can just be his way of communicating. Pay attention to your individual dog and figure out how he "talks " to you.
"You can't teach an old dog new tricks."
May or may not be true. I don't think this applies to all dogs, and age may not be the same in all dogs.
3 is not what I would consider old for any dog. I would say there is hope, but it may be a little more work than with a puppy.
It is never to late to train any age animal.
Many rescue dogs of all ages are successfully house trained after being placed in homes for the very first time. They do need time to develop bladder and bowel control just as a puppy does.
It takes commitment and patience ... and if a accident does happen it is not the fault of the dog it is our fault for failing to properly supervise him/her.
Remember .. Love, Leadership, commitment, patience and language.
The love and loyality you will receive in return makes this all worth it.
One more hint. Every time you go outside talk to him and use the word "outside". Some dogs are good with language. If he learns the word "outside" then things are easier. You can just ask him "Do you have to go outside?" and he will tell you with body language yes or no.