Occasionally. Sometimes.  Neither often nor rarely but at unspecified times. (Compare to Few and far between or Once and a while.)

“From time to time I like to drive to the city.”