Nitrogen cycling associated with microorganisms is the most important biogeochemical process in eutrophic lakes. In this study, molecular biological techniques including quantitative PCR, PCR-DGGE, and high-throughput sequencing based on amoA and 16S rRNA genes were used to assess the composition and community abundance of ammonia-oxidizing archaea (AOA), ammonia-oxidizing bacteria (AOB), and nitrite-oxidizing bacteria (NOB) in freshwater and sediment samples of Lake Taihu collected during autumn. The key environmental factors affecting the distribution of nitrifying microorganisms were further analyzed. Results showed that the amoA gene abundance of AOA from the upper, middle, and bottom freshwater and surface sediment samples from the mesotrophic Meiliang Bay was lower than those from the less eutrophic central lake and that the amoA gene abundance of AOB from the mesotrophic Meiliang Bay was greater than those from the central lake. The AOA community composition in freshwater was similar in both Meiliang Bay and the central lake, it was the same for AOA in the surface sediment, and the key factors affecting AOA community abundance were nitrate, pH and dissolved oxygen contents, while the community abundance of AOB mainly affected by total nitrogen content in sediments were proven to differ considerably between the two sites. The case was the same for NOB with the key factor nitrite content. AOA of freshwater and surface sediments in Meiliang Bay and central lake included two genera, Nitrosopumilis and Nitrosotalea. AOB of the surface sediment included Nitrosomonas and Nitrosospira. NOB of the surface sediment included Nitrospina and Nitrospira, and Nitrospina was scarce in freshwater. The main environmental factors affecting the abundance of AOA and AOB in Lake Taihu were TN, total phosphorus (TP), and ammonium content. This study proved that the eutrophication indexes (such as total nitrogen, total phosphorus, ammonium, nitrate and nitrite) have significant influences on the abundance of AOA or AOB and community abundance of nitrifying microorganisms in both freshwater and sediments in Lake Taihu, which may help the understanding of the nitrogen cycling in the Lake Taihu ecosystem.