Looks to me like you took the starter apart, instead of taking it off.
Feel underneath, and see if there's a couple of bolts running vertically. Take them out and the rest of the starter will come off.
Starter is bolted directly to engine with two bolts. Solenoid is also bolted directly to engine. The only connection between the two is a shot electrical wire from the starter to the solenoid.