Issue: Changing the desktop resolution on Linux or in my case Kali has issues in ESXi. It either fails, goes blank or displays corrupted image until it reverts itself back to 800×600.
I stumbled upon this with a new installation in a testing network I was playing with, despite installing vmtools on the client and updating them numerous times I could not get the desktop to a more usable resolution over 800×600. Was it drivers for the video card, monitor? Well after 30 mins of googling I found a solution in the Kali forums (Thanks to SpeedyQuick.)
Increase video card memory in vSphere Client to 32MB from the default of 4MB. Then you should be able to resize to your desired resolution.
I’d imagine if you were having similar issues with Virtualbox this solution would also apply, one thing to note though is that Virtualbox seems to default to a higher amount so I have not seen this issue yet.