China’s birth rate has declined significantly since the 21st century, and the increase in residents’ working hours caused by the upgrading of industrial structure might be an important reason for the decline in the birth rate. The research constructed weighted average working hours of China’s urban industry as the key explanatory variable, and used the empirical study of China’s provincial panel data from 2003 to 2019 to find thatafter controlling the influence of other factors, the increase in the average working time of urban residents significantly reduced the local population birth rate. Further research has showed that residents’ income and working hours have interactive effects on the birth rate. The higher the per capita disposable income, the more the negative impact of increased working hours on the birth rate would be. The income level has a certain compensation effect on labor leisure. The construction of instrumental variable “economic geography deviation degree”, the removal of minority autonomous regions and the robustness test based on Sys GMM method also reach consistent conclusions. The heterogeneity test found that the increase of average working hours of urban male and female residents would both negatively impact the local birth rate. Relevant departments should strictly regulate the working hour system on the basis of ensuring the continuous growth of residents’ income, ensure that employees have sufficient labor leisure, and provide necessary support and assistance to family childbearing to avoid a continuous decline in the birth rate.