Herein, we report a patient who had an isolated sixth nerve palsy due to a petrous apex cholesterol granuloma. The sixth nerve palsy appeared acutely and then spontaneously resolved over several months, initially suggesting a microvascular origin of the palsy. Subsequent recurrences of the palsy indicated a different pathophysiologic etiology and MRI revealed the lesion at the petrous apex. Surgical resection improved the compressive effect of the lesion at Dorello's canal and clinical improvement was observed. A relapsing-remitting sixth nerve palsy is an unusual presentation of this rare lesion.