I agree with your physician. Golf can be relaxing and enjoyable and does get you moving (if you don't use a vehicle), but it doesn't give you sufficient aerobic or cardiovascular exercise, which everyone needs, regardless of age. Beyond that, any exercise program should include resistance training to increase muscle strength (which declines by about 15 percent per decade during one's 60s and 70s) and exercises to increase flexibility, which can help prevent falls as you age.
My preference for aerobic/cardiovascular conditioning is walking. Fitness expert, Dan Bornstein says that a 20-minute walk four to five days per week is the minimum needed, but if possible, a brisk 45-minute daily walk is ideal. The exercise will enhance your heart and lung function and improve your endurance as well as help with weight control by burning calories. For older adults who have joint problems that might be aggravated by walking, I recommend swimming or water aerobics classes.
As for resistance training, the American College of Sports Medicine recommends that individuals over the age of 65 follow a regular workout routine- either with free weights or at the gym on weight machines - at least two days per week (but no more than four) with at least 48 hours between sessions. If you haven't done any resistance training before, you'll need instructions from trainers at a gym or a few sessions with a personal trainer to learn the exercises and how to do them safely. The idea is to work the muscles of your chest, shoulders, arms, back, abdomen and legs by doing one or two exercises per muscle group. As the exercises become easier, you can increase either the number of times you repeat each movement or the amount of weight you're working with.
To increase flexibility you can learn stretching exercises, take yoga classes or learn Pilates, a conditioning system that increases both core strength and flexibility.
Once you get going with your exercise program, you'll probably feel stronger, more fit and energetic, all of which should improve your golf game.
Andrew Weil, M.D.