Slow viewport - 1 Million Poly Model is 200+ FPS but 4FPS in Edit-Poly mode

[LightRays]

I have a detailed house model with over a million polys. I can browse around the perspective viewport at 200+ FPS, fullscreen viewport at 1080p HD with a GTX 550 2GB (scene uses only 1GB). But when I select a certain element or a polygon or even a single Edge and then rotate the viewport, I get 4 FPS speed and everything is slow to move. I am getting 200+ FPS in normal viewing mode so why so slow in edit poly mode? I have Max 2012 with windows 7 64. All latest drivers, hotfixes, windows updates etc.