Many golfers, touring pro and weekend player alike, are so concerned with the mechanics of the grip and the swing that they often overlook an important element of good golf: tempo or rhythm. By this I mean the rate at which the club is taken back and then brought down into the ball. The two phases of the swing, the backswing and the downswing, should be carried out at pretty much the same tempo. The club head will always travel much faster coming down than going back, but the hands will not. Thus they should establish the tempo. If you have a fast backswing the downswing should also be fairly fast; otherwise you will lose momentum and therefore distance. If your backswing is slow and the downswing too fast, you are likely to lose control of the club at the top of the swing and thus have trouble hitting the ball straight. So work at maintaining a uniform speed.
Another consideration is whether the present speed of your swing is right for you. Maybe you are using a slow, leisurely tempo when a fast one would work more effectively. Or vice versa. You must find the tempo best suited for you, and you will not find it accidentally. You must learn how fast you can take the club back and still maintain momentum, balance and control on the downswing. Instead of being satisfied with the tempo you are now using—and after all, who is ever satisfied with his golf game?—try experimenting on the practice tee to see if a change improves your shots.