All "alternative" frequencies that are offered this way make the APB bus change too. That has interesting consequences. As I already expected the wifi system can't handle the APB bus being other than 80 MHz, the UART needs tweaking (ok, not that much a problem) but also the timers no longer run at frequencies as expected. And last but not least, I expect all sort of issues with the built-in peripherals.
What I am looking for is purely the cpu core running at a higher speed, while leaving the APB at 80 MHz. It may or may not be possible with the PLL and clock dividers, but apparently noone succeeded there (yet).