In this work we study the effect of thermal processing of exfoliated graphene on mica with respect to changes in graphene morphology and surface potential. Mild annealing to temperatures of about 200°C leads to the removal of small amounts of intercalated water at graphene edges. By heating to 600°C the areas without intercalated water are substantially increased enabling a quantification of the charge transfer properties of the water layer by locally resolved Kelvin probe force microscopy data. A complete removal on a global scale cannot be achieved because mica begins to decompose at temperatures above 600°C. By correlating Kelvin probe force microscopy and Raman spectroscopy maps we find a transition from p-type to n-type doping of graphene during thermal processing which is driven by the dehydration of the mica substrate and an accumulation of defects in the graphene sheet.