I ran into this with my blogs too. It looks like WPSEO is replacing wp_title so that your theme can fill it in normally, but the themes are outputting wp_title followed by the blog name. So WPSEO adds the site name title, and then the theme adds it again.
Force rewrite seems to wipe the theme's <title> tag entirely and build a new one in wp_head.