1) Start Registry Editor (Start -> Run -> Type 'regedit' -> Enter)
2) Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TerminalServer\WinStations\RDP-Tcp\PortNumber
3) On the Edit menu, click Modify, and then click Decimal.
4) Type the new port number, and then click OK.
5) If you have any firewall installed, add a rule to allow traffic over new port number that you enter (Must!) - Please ensure that this step is followed correctly else you will lose connectivity with your VPS.
6) Quit Registry Editor.
7) Restart Remote Desktop Services or reboot entire machine.