Ground it down to give it that finished look. I'll be painting it all and this part will stick through the carpet so I want it to look as nice as possible. It is structurally sound no worries there.