In order to overcome the shortage of disaster condition records and the difficulty of identification to drought effect at each growth stage, based on daily meteorological data at weather stations, growth period data at agro-meteorological stations and rice yield data in counties of Sichuan Province from 1981 to 2015, with improved water budget index as drought indicator, the relationship between drought and yield at each growth stage of rice was analyzed by using grey correlation analysis method. The results show that the drought frequency at growth stages of rice in Sichuan from high to low was transplanting-tillering stage, tillering-jointing stage, booting-heading stage, heading-maturity stage and jointing-booting stage in turn. The spatial characteristic of rice drought frequency at each growth stage was low consistent with the spatial distribution of grey correlation degree between water budget index and rice yield. The high frequency of drought occurred in the middle or northeast of Sichuan Basin, while the high grey correlation degree located in the mountainous areas around the basin edge and Panxi area. With the process of rice growth, the effect of drought on rice yield weakened, and at transplanting-tillering and tillering-jointing stage it was the most obvious. The growth period with the greatest influence of drought on rice yield was different in each county of different planting regions, so the policies of drought alleviation should be strengthened according to the spatial characteristics of drought influence at each growth stage of rice. We should focus on the drought at tillering-jointing stage of rice in planting regions of the Chengdu Plain and the central hill of Sichuan Basin and transplanting-tillering stage in other five rice planting regions.