Abstract: Since federated learning only selects a small number of clients to participate in each round, and there is often heterogeneity between clients in an IoT environment. In this case, this training approach leads to large differences in model convergence. In order to reduce the impact of heterogeneity, cluster sampling was introduced to select participating clients, using the gradient vectors calculated from the local data as client clustering features, dividing participating clients into multiple clusters. In this way, the representation of clients in the model aggregation process was improved and the data specificity of non-sampled clients was guaranteed. Experimental results show that compared with the existing sampling schemes, the training through cluster sampling can effectively improve the model convergence speed, and when the local data is in a highly heterogeneous situation, the model accuracy can be improved by about 5 percentage points.