I have a 1990 Geo prism that starts sometimes but mostly the clutch has to be popped soleniod or starter?

I am not sure if I should replace the starter or the solenoid and I don't know how to tell which is bad. Once running it runs great it is just starting I have replaced the battery already and that is not it. I also checked the Alternator by taking off the positive while it was running and it doesn't turn off. Is there a simple test to decipher wether it is the starter or solenoid?