Car battery, alternator, or starter?

Lately my car won't start after leaving the car for about 6 hrs or so. It would crank but wont start and slowly if i retry the crank would start to slow down. and after awhile it would start to click like crazy. I've checkin the battery voltage which is fine 12v with it off and 14.3 with the car on. Also checked fuses and replaced a blown 15a fuse. But it still happens. Also if i drive my car around then it stays fine and normal operation its only when I leave my car for a couple hrs that I start to notice a difference. Perhaps Im missing somthing?