Yeah, "fall into the sun" was sort of hyperbole. If it truly got out into space and was going fast enough to escape Earth's gravity, it would start orbiting with earth's orbit plus some delta. Out of all the possible angles it could leave the earth, there are probably 2 angles where it would directly hit the sun One is the angle that cancels out all the orbital velocity of the earth and sends it directly at the sun, the other is the one that does the same but sends it directly away from the sun. Of all the possible trajectories on the surface of a sphere, only those two tiny solutions would end up with it contacting the sun, everything else would result in an orbit.

Of course, given enough time, it's pretty likely that if it isn't collected by a planet, it will eventually end up in the sun. There isn't much friction in space, but there's a tiny bit: solar wind, micrometeoroids, etc. Eventually its orbit would decay and it would stray too close to the sun.