Type 2 diabetes mellitus (T2DM) is a global public health epidemic. Diet and lifestyle changes have been demonstrated as effective measures in managing T2DM and preventing or delaying the progression from prediabetes to diabetes, yet the relationship between diet, prediabetes and diabetes is still not entirely dear. The present study aimed to further elucidate the relationship between diet, diabetes and especially prediabetes. A total of 1542 participants of the cross-sectional, population-based Cooperative Health Research in the Region of Augsburg (KORA) FF4 study (2013/2014) were included in this analysis. Dietary intake was derived using a method combining information from a FFQ and repeated 24-h food lists. Glucose tolerance status was assessed via oral glucose tolerance tests in all participants without a previous physician-confirmed diagnosis of T2DM, and was classified according to the 2003 American Diabetes Association criteria. Crude and fully adjusted multinomial logistic regression models were fitted to examine associations between diet and prediabetes, undetected diabetes mellitus (UDM) and prevalent T2DM. After adjusting for major covariates, fruit was significantly inversely and total meat, processed meat, sugar-sweetened beverages and moderate alcohol significantly associated with UDM and/or prevalent diabetes. Sex-specific analyses showed that in men, coffee was significantly inversely (OR 0.80;95 % CI 0.67, 0.96) and heavy alcohol significantly (OR 1.84;95 % CI 1.14, 2.95) associated with prediabetes. Our findings on diet and T2DM are consistent with current literature, while our results regarding coffee, heavy alcohol consumption and prediabetes highlight new possible targets for primary prevention of the derangement of glucose homeostasis.