User Daily Load Classification Method Based on Improved Deep Clustering Under Spark Architecture

Load clustering is one of the most important technologies in power system management. By clustering algorithms to mine users' electricity consumption patterns, power system managers can gain a better understanding and enhance the optimization of power system operation, thereby improving its efficiency and economy. At present, it is difficult for traditional load clustering methods to deal with massive and high-dimensional load data efficiently and accurately under the trend of load data quantification and complexity. In this paper, we propose a daily load classification method in a Spark distributed computing architecture, which is based on improved deep clustering. First, a convolutional neural network autoencoder is utilized to acquire the representative feature vectors of users and send them to K-means clustering layer for load clustering completion. Subsequently, the feature extraction model and clustering model are jointly optimized to form a deep clustering model. Secondly, considering the adverse effects of the edge load samples at the boundary of the load class on the neural network, a self-stepping learning technique is introduced and a new loss function is designed. Finally, the integration of big data technology with deep clustering algorithm and the utilization of Spark distributed computing platform enable the parallel computing of deep clustering algorithm. The results indicate that the proposed algorithm outperforms the traditional algorithm in terms of both clustering effect and processing efficiency.