Sometimes they do that if it is pushed to the left when the PS2 is turned on... then when you go into the game... it thinks that "center" is off to the left... so when it is centered... voila! You're going right.

If it does it everytime... well, it might just be an internal problem (in the controller) and I would suggest a new one.