The interplay of social structure and cooperative behavior is under much scrutiny lately as behavior in social contexts becomes increasingly relevant for everyday life. Earlier experimental work showed that the existence of a social hierarchy, earned through competition, was detrimental for the evolution of cooperative behaviors. Here, we study the case in which individuals are ranked in a hierarchical structure based on their performance in a collective effort by having them play a Public Goods Game. In the first treatment, participants are ranked according to group earnings while, in the second treatment, their rankings are based on individual earnings. Subsequently, participants play asymmetric Prisoner’s Dilemma games where higher-ranked players gain more than lower ones. Our experiments show that there are no detrimental effects of the hierarchy formed based on group performance, yet when ranking is assigned individually we observe a decrease in cooperation. Our results show that different levels of cooperation arise from the fact that subjects are interpreting rankings as a reputation which carries information about which subjects were cooperators in the previous phase. Our results demonstrate that noting the manner in which a hierarchy is established is essential for understanding its effects on cooperation.