There are 4 places that could be altered to make it fit looser:
The action pin
The barrel lug
The barrel breech
The receiver breech face.
I like to change the cheapest part.
If you can get the pin out, take a few thousands off the center of the pin where it contacts the barrel lug. None off the outer parts of the pin that fit the receiver.
I have altered the receiver to fit the barrel, when making a upper assembly with a barrel blank. That is a hard way to go.