The details are rather complicated and you would really have to 'do' the calculations to get a good idea for what is going on, but the basic idea is that you only get circular orbits if the force field acting on you is constant in the 'radial' direction. This is not the case for gravity which varies as the 'inverse-square' of the distance.