verify_swm_set(): Decrease scope of parts_to_swms