I would normally say yes, if the song is in Japanese then subtitles are really helpful for English viewers, mainly because it helps make the impact stronger for the individual lyrics against the visuals.
I can see how you'd think it might be distracting, so I would say the best way to achieve the best of both worlds is to do the subtitles elegantly. Having black bars on top and bottom, with Japanese lyrics on the top bar and the English translation on the bottom one (in a nice font) could be one way to achieve that. Here's a sort-of example: http://www.youtube.com/watch?v=ZS_FYeoIBPo
If you really feel like the subtitles detract from the video, an alternative I've seen is to provide text in the video description (you know, the author's comment area on the site you plan on posting the video to) that has the translated lyrics written out. The only downside is that English viewers won't be able to easily associate specific phrases in the lyrics with the scenes in the video without having to constantly reference back to the lyric translation you've provided. I'd say this idea works better if you don't think the lyrics are as important to the AMV as the general feeling that the song gives off.
One last thing, a little off-topic but XD, subtitles left in from the actual clips of the anime, they're almost always distracting in an AMV. There have only been a few instances I've seen where the AMV author was actually able to cleverly complement the AMV with the original subtitles, but those instances are few and far between. So IMO, RAWs are a base standard for a good AMV. Just my two cents, heh. =P