Yeah I figured, meanwhile I googled a bit more and it seems there's no fix for this problem. I'm having the same here on Vista but in a slightly less severe way, sometimes I can switch to another app when Reaper stays maximized, I bring the other audio app window into focus and Reaper in the background switches to "offline" mode. The "Launch file" button in the render dialog always works. However, the only ways to get rid of this seem to be "don't try to use more than 1 audio app at a time" or "use XP". I recommend the latter, Vista is partially bombing us back into the 80s audiowise.