When the law first came out, no letterhead was needed, at least per many ATF agents. Later, the 'letterhead requirement' came out as an 'interpretation' by the ATF, and everything had to be done on letterhead. I believe this was done in response to officers purchasing weapons and mags without letterhead, then reselling them to the general public. At the time, there was a lot of confusion about the law. I had an ATF agent tell me when I bought my LEO AR lower that it was mine forever, even if I left the department. That later changed, too, and it had to be turned in when I quit, along with my LEO mags.
Lots of agencies have it in their policy manual that you are always (24/7) acting as a LEO even when not working shift, thus even off-duty weapons qualified as 'for duty purposes'. Even my 45Super Para P14 was a 'duty' weapon, because I carried it off-duty (and in my cruiser glove-box when on).