The Magpies were unable to recover from conceding two goals in the first 25 minutes at Upcott Field.
It was a seventh successive league defeat for the Carlsberg Peninsula Premier Division's bottom team.
A Wayne Riley own goal and Phil Stokes' drive put Rangers 2-0 up.
Lee Rundle converted a penalty before the break but Cullompton restored their two-goal advantage through Andy Rollason's second-half effort.
Rundle scored again for Holsworthy but could not save them from another defeat.
The Magpies visit Elburton Villa on Saturday.