Misleading title.
~~If my thing was public in the past, and I took it private, the old public code is still public.~~
That's... How the Internet works anyway.
Edit: See Eager Eagle's better explanation below.
TL;DR - be careful who you allow to fork your private repos. And if you need to take a public repo, which has forks, private, consider archiving the repo and doing all the new work in a new repo. Which is arguably the reasonable thing to do anyway.
Still a misleading title. This isn't a way to break into all or even most of your private repositories.