To achieve the aim of “building the rule of law in a socialist country”, local governments play a crucial role. In practice, local governments take the main responsibility to implement the law. At the same time, local governments face some practical problems because of the dual role of local governments in the structure of governmental regulation. The goal of “building the rule of law” requires the rule of law in the whole country, including the lower governments. To this end, it is necessary to attach the importance to the unique function of the local governments, and shape the local institutional autonomy so that they may participate in this process. As a normative study on the system of the Chinese Constitution, the paper will illustrate the meaning of “central unified leadership” and then suggest that at the local level, institutional incentives should be provided for the local initiatives.