Non-orthogonal multiple access (NOMA) emerges as a superior technology for enhancing spectral efficiency, reducing latency, and improving connectivity compared to orthogonal multiple access. In NOMA networks, successive interference cancellation (SIC) plays a crucial role in decoding user signals sequentially. The challenge lies in the joint optimization of SIC ordering and power allocation, a task complicated by the factorial nature of ordering combinations. This study introduces an innovative solution, the Attention-based SIC Ordering and Power Allocation (ASOPA) framework, targeting an uplink NOMA network with dynamic SIC ordering. ASOPA aims to maximize weighted proportional fairness by employing deep reinforcement learning, strategically decomposing the problem into two manageable subproblems: SIC ordering optimization and optimal power allocation. We use an attention-based neural network to process real-time channel gains and user weights, determining the SIC decoding order for each user. A baseline network, serving as a mimic model, aids in the reinforcement learning process. Once the SIC ordering is established, the power allocation subproblem transforms into a convex optimization problem, enabling efficient calculation of optimal transmit power for all users. Extensive simulations validate ASOPA's efficacy, demonstrating a performance closely paralleling the exhaustive method, with over 97% confidence in normalized network utility. Compared to the current state-of-the-art implementation, i.e., Tabu search, ASOPA achieves over 97.5% network utility of Tabu search. Furthermore, ASOPA has two orders of magnitude less execution latency than Tabu search when N=10 and even three orders magnitude less execution latency less than Tabu search when N=20. Notably, ASOPA maintains a low execution latency of approximately 50 milliseconds in a ten-user NOMA network, aligning with static SIC ordering algorithms. Furthermore, ASOPA demonstrates superior performance over baseline algorithms besides Tabu search in various NOMA network configurations, including scenarios with imperfect channel state information, multiple base stations, and multiple-antenna setups. These results underscore the robustness and effectiveness of ASOPA, demonstrating its ability to ability to achieve good performance across various NOMA network environments. The complete source code for ASOPA is accessible at urlhttps://github.com/Jil-Menzerna/ASOPA.