Yes, your setup will work since the PyTorch binaries ship with their own CUDA runtime (as well as other CUDA libs such as cuBLAS, cuDNN, NCCL, etc.). The locally installed CUDA toolkit (12.0 in your case) will only be used if you are building PyTorch from source or a custom CUDA extension.
The NVIDIA drivers are backwards-compatible so the newer driver (I would guess 525.60) will also work.