Paper filter versus mesh filter...fight!

Question: Does it make a difference what kind of filter I use between the premanent metal one and the paper ones you can buy in bulk? I've used the metal one before but I found it left a bit of sludge at the bottom of my cup so I went back to paper.  -- Neil