Disadvantages of wheel spacers for track use?

[ToddPhilly]

That was going to be my next question...so they are difficult to swap on and off when changing wheels?

I wonder if my stock wheels would fit with a 15mm spacer, or would it push them out too far that they would rub.

I guess the other thing to keep in mind is that I'd only have to do this on the rears.  The fronts match.