If the Sun rotates around the center of the Milky Way, is the entire solar system moving around too?

Yes, because the planets orbit the Sun and where the Sun goes, so do the planets. The speed of the Sun+solar system is 250 kilometers/sec relative to the center of the Milky Way, and about 20 kilometers/sec towards the star Vega relative to the reference frame of the nearby stars which, along with the Sun are orbiting the center of the Milky Way at 250 kilometers/sec.