Based on CGSS data and Heckman sample selection model, this paper makes a systematic study on wage differentials and influencing factors in state owned and non state owned sectors. The regression results show that the staff s rate of return on education in state owned sectors is higher than that of the non state owned sector, and rate of return on education is positively correlated with the level of education. Compared with the non state owned sector, the college degree or above graduates rate of return on education is higher in state owned sector. The decomposition of wage differential shows that the proportion of market discrimination is higher than that of characteristic differences accounted for in the full sample of the two sectors. And compared with the full sample, the total wage difference and characteristic difference of college degree or above graduate are higher, but the proportion of coefficient difference is lower. That is to say, the phenomenon of wage differentials in highly educated groups is more obvious, and there are still some severe labor market segmentation and employment discrimination in two departments. Therefore, the government should strive to eliminate barriers to labor mobility, to deepen the market oriented reform of the state owned sector, to develop a more reasonable salary mechanism and to promote sustained, healthy and steady growth of Chinas economy.