Abstract: Aimed at the complex time-varying characteristics of heterogeneous ultra dense network (HUDN), the problem of non-orthogonal multiple access (NOMA) wireless resource allocation under HUDN is studied. In order to improve the quality of experience (QoE) of network users, the problems of user-base station matching, channel allocation and power allocation were considered comprehensively, and a wireless network resource allocation algorithm based on double deep Q-network (DDQN) was proposed and implemented with the goal of maximizing the SUM of the mean opinion score (MOS) of the whole network users. Simulation results show that, compared with DQN and Q-Learning algorithms, the proposed algorithm not only effectively improves the QoE of all network users, but also improves the utilization of wireless resources.