Can a drive veer one way or another if your tempo is off, even if you've pulled the stick down and back up straight?
Yes. Downswing affects it the most. Fast downswing will cause the ball to veer left (hook) and a slow downswing will cause it to veer right (slice). Downswing tempo is most important. Backswing tempo affects distance. Imagine backswing as the % of power your getting. If your backswing is perfect your getting close to 100% of your clubs distance. Fast backswing your not getting all of it and going to come up short. This is required for approach shots where you don't want a full swing that would send you past the hole, example being 12y chip shot distance and your only 8y from the pin. Slow backswing doesn't seem to change much of anything with a perfect downswing, maybe 1 or 2 yards less than normal carry distance on a drive. Here's a breakdown on what I've noticed so far.
Backswing | Downswing
Perfect - Perfect = Normal distance in direction of white line
Perfect - Slow = Normal distance veering right of white line
Perfect - Fast = Normal distance veering left of white line
Slow - Perfect = Slightly less than normal distance in direction of white line
Fast - Perfect = Less to a lot less than normal distance depending on where in backswing height you started downswing.
Combinations as far as slow - fast, fast - slow, or very fast/very slow I'm not 100% sure on, they'll go very badly I know that much

.
Imagine the white line as a line drawn from your stick in the back position all the way to the forward position of your swing.