I think the reason to remove the trigger is to be able to support the trigger solidly while you drive out the little pin holding the disconnector in place. You can see it with the trigger still installed in the gun, but it's best not to pound on the side of the trigger while it's just hanging on the trigger pin.
Now if you have a spare set of hands to hold the gun with the trigger still in it, and can make a backing support small enough and solid enough to handle the pounding on the trigger without transferring the pounding force to the frame or pin, and provide a clearance hole in the support for the disconnector pin when you drive it out, then it would be no trick to do the job with the trigger still in the gun.
Polishing the disconnector pad and the front of each magazine is indeed a way to smooth out the feel of a disconnector-equipped Hi Power. But if you shoot much between cleanings the feel will start to change as soot builds up on the sliding surfaces, and you'll have to polish each magazine you have now or get in the future. Which may or may not be a big deal.