I have a VPI platter and motor and of course it came with the rubber belt. Try this ... put your finger lightly on the belt "before" the capstan on the motor. Place it lightly there and you will not feel much, because the belt it being pulled by the motor. Now place your finger on the other side, where the motor is pushing the belt to the platter. I bet you will feel quite a bit more bounce and vibration on the belt.
That vibration is being transmitted to the platter and ultimately to the stylus. 2 solutions.
(1) place something solid and heavy there so it touches the belt and absorbs some of that vibration before it gets to the platter. Maybe a brass weight or something similar.
(2) replace the rubber belt with silk string. The string won;t vibrate like the rubber and silk doesn't stretch. Any other kind of string will stretch and cause speed anomalies. Caveat ... you can;t do this second method unless you have a speed controller because the diameter of the silk string is much smaller than the rubber belt and will sit lower in the groove of the motor capstan and your platter will spin at something other than 33 1/3. Second caveat - getting the length of the sting correct takes some trial and error, because you have to tie that simple knot in it just right.
In my opinion the silk sting sounds much better than the rubber belt. You do have to start the platter turning by hand though, but that's a slight inconvenience.
Enjoy