Planned Government changes could result in careers services facing "destruction", according to members of one of its own advisory groups.