The present study aimed to explore the possible mechanisms underlying Dendrobium officinale leaf polysaccharides of different molecular weight to alleviate glycolipid metabolic abnormalities, organ dysfunction and gut microbiota dysbiosis of T2D mice. An ultrafiltration membrane was employed to separate two fractions from Dendrobium officinale leaf polysaccharide named LDOP-A and LDOP-B. Here, we present data supporting that oral administration of LDOP-A and LDOP-B ameliorated hyperglycemia, inhibited insulin resistance, reduced lipid concentration, improved β-cell function. LDOP-A with lower molecular weight exhibited improved effect on diabetes than LDOP-B, concurrent with increased levels of colonic short-chain fatty acids (SCFAs) i.e., butyrate, decreased ratio of Firmicutes to Bacteroidetes phyla, and increased abundance of the gut beneficial bacteria i.e., Lactobacillus, Bifidobacterium and Akkermansia. These results suggest that LDOP-A possesses a stronger effect in ameliorating T2D than LDOP-B which may be related to the distinct improved SCFAs levels produced by the change of intestinal flora microstructure.