Building reservoirs affects the process of water and sediment transport in the basin of the dam. The impact of the Three Gorges Reservoir (TGR) on sediment transport in the downstream has appeared. After the TGR impoundment, the total sediments in the downstream have been reduced. Sediments grouped with d≤0.125 mm (fine particle) were increasing along the path, but the average amount was still less than the amount before the reservoir impoundment. Sediments grouped with d>0.125 mm (coarse particle) have been recharged in the Yichang to Jianli section, but downstream of Jianli has a trend of deposition. In specific, the coarse sediment in the Yichang to Jianli section in 2003-2007 was above the value before the TGR impoundment, but increasing of the sediments concerning both coarse and fine sediments in 2008-2014 less than that in 2003 to 2007. Sediment retaining by the Dam is the major reason for the reduction of sediment in the downstream. However, the retaining in different reaches of the River is impacted by coarsening of the riverbed, the discharge, flow process, conditions of lake functioning and recharging from the tributaries. The main conclusions from our study are: (1) In the Yichang to Shashi section, the riverbed is relatively coarse that limits the recharge of coarse sediment, where the supplied fine sediment is controlled by the discharge by the duration of flood and its magnitude. (2) In the Shashi to Jianli section, recharging of both coarse and fine sediments is controlled by amount of total discharge; (3) In the Jianli to Luoshan section, the flow recharge from the Dongting Lake plays the most important roles on sediment recharging. Sediments supply from riverbed is higher in flood years than that from the dry years. The coarse sediment tends to deposit and the deposit amount in dry years is larger than that in flood years. (4) In the Luoshan to Hankou section, the recharge of fine sediment is controlled by supply from the riverbed. The recharging in 2008-2014 is more than that in 2003-2007. In about 2010, the coarse sediments were transited from depositing to scouring that is probably caused by the increasing of continuous large flow days. (5) In the Hankou to Jiujiang section, more fine sediments appear to deposit. The coarse sediment is recharged by the scouring of the riverbed and there were more coarse sediments recharged from the riverbed in the flood years than that in the dry years. (6) In the Jiujiang to Datong section, the rate of the fine sediment recharge from Poyang Lake to that from the riverbed was 1:2.82. It is obvious that the sediment from scouring of riverbed contributed more to coarse sediment transportation and the contribution is mainly affected by magnitude and continuous days of large flow.