Along with the deepening of the concept of corporate governance and the establishment of corporate governance structure and mechanism, the issues of corporate governance effectiveness increasingly become the focus of regulators and listed companies concerned. Based on the public data of listed companies from 2003 to 2013, using panel data methods, this paper investigates the relationship between two kinds of ownership agency cost and 12 important elements of corporate governance, which is included by ownership structure, operation mechanism, decision-making mechanism, monitoring mechanism and incentives mechanism. The results show that corporate governance elements can reduce agency costs effectively; but a few elements of corporate governance have a nullity. The top directors’ remuneration even shows a positive correlation with the first agency cost. Therefore, in order to further improve the effectiveness of corporate governance in our country, we should strengthen the correct corporate governance way of thinking, control the number of meetings of the board of directors, and improve the system of the board of supervisors.