Wow, I have been dealing with this myself last year and now again.
Clean looking risers don't mean effective risers, and good water flow out the back plugs of the manifolds doesn't mean much at all.
I removed my manifolds Saturday morning and (they too looked fine) with them upright on the ground, connected a garden hose to a scrap piece of 1" tubing. Clamp this tube to the manifold inlet and the male garden hose fitting will fit the tube perfectly...clamp that too. Replace the rear plug and turn on the water.
I had nice flow in only two of the ports and total blockage in the other two, so like yourself, I was fine at lower RPMs, but once the engine was generating more heat, it couldn't cope. The rear and inner ones will be clogged.
While the manifolds are off do the following:
I had particulate rust accumulating under the thermostat housing too and this is a sign of trouble. Again Saturday, I flushed my block THOROUGHLY using the following method and it worked like a charm.
You'll need to plug off the 2 ports on the thermostat housing that feed the manifolds. I used short pieces of 3/4" tubing and plugs all clamped on. Disconnect the hose that comes from your transom, or the raw water inlet. Clamp another piece of 1" hose to that spot on the thermostat housing and again clamp the male garden hose into that. You have now created a pressurized water source into the block only that WILL NOT go out through the exhaust since that's not there any more.
DO NOT turn on the water unless you have at least one drain fitting out. The pressure could blow a soft plug out or God knows what else.
Remove the entire block drain fitting from one side of the block and root around in there with a stiff wire. Go up, sideways and any way you can to dislodge any rust flakes and then turn on the water. You won't believe the crap that will come out of there. Let it drain out, root around again, blast it out again and repeat until no more junk comes out.
Close that side up and do the other side. Be ready to do this a dozen times on each side. Once I was satisfied, I opened both drains again and let water run through for about 10 minutes. Replace drain fittings and close drains.
If you're wondering why you can't do this with the manifolds in place, it's because the water that does come out of the top of the manifold will go right down the throat of the exhaust and into the cylinders. You could pop your spark plugs out, crank the motor and blow that water out, but the likelihood that you'll be replacing the manifolds is VERY high, so you might as well take them off now.
Your bilge pump will handle all of this activity and at the end whatever you have blown out of there will accumulate in the bottom of the boat. My guess is you'll be shocked by the amount of stuff you got out.
Another thing to look at while your manifolds are off is to see if your exhaust flappers have broken up and fallen into your Y pipe. Someone with small hands and thin arms can reach down in the to the bottom of the Y.
Like yourself, I removed the damn thermostats and then dealt with too rich fuel mixture and the sooty and fouled spark plugs that come along with running too cool.
Blow out your block, replace your manifolds and risers and get back out there. It'll cost you around $600.00. Ask me how I know.
This is a good hot weather job as you will get soaked and your bilge will never be cleaner. Let me know how you make out.