The implementation of an animal welfare assurance programme for dairy cattle in South Tyrol (Eastern Italian Alps) faces particular feasibility constraints due to the outstanding volume of travel associated with routine on-farm audits of remote mountain farms. Therefore, this study aims to estimate the inter-rater reliability of the expert’s and farmers’ welfare outcome assessment regarding recommendations to involve milk producers in animal welfare assurance within South Tyrolean dairy farming. A formal training programme containing a classroom session and an on-farm observation became mandatory for all 188 participating farmers, which was offered by the expert, applied as reference standard. On-farm data collected on the farmers’ cows (dataset of 1719 dairy cows) were compared at animal level. Cohen’s kappa, respectively, weighted kappa, examined for several welfare indicators, range from slight to moderate agreement(k=0.018-0.416;Kw=0.163-0.310). These findings are further confirmed by results at farmlevel (ICC=0.018-0.577). Continuous repeatability checks as part of routine audits are therefore proposed to substantially reduce the variability between the raters and to avoid significant bias in the welfare outcome assessment. In this way, the competence for regular and standardised monitoring could be increasingly transferred to dairy farmers in order to reduce the need for costly and time-consuming inspections by external auditors, which are in long-term perspective also harmful to the alpine environment. Additionally, the promotion of welfare assessment as an instructive management tool would intensify farmers’ commitment to the assessment process.