By making a referral, you are helping someone else. Being careful to match up trustworthy people, you will be remembered, by both sides, the next time your services are required. And it make you a good referral as well.
Make one EASY warm call. Select someone you know will remember you, and call them up and ask them about their business. You may not end up doing business together, but worst case, you start to lose that fear of making calls and eventually you WiLL get some business.
When giving your elevator pitch, if you typically say, “A good client for me is someone in a small business,” perhaps a more effective statement would be to instead name a specific industry. That way, the people listening to you can immediately think of someone they know in that industry, making it easy for them to think of a referral for you.