The domain name entered in the Upload To FTP Host dialog needs to be the URL for the home page of the site being uploaded.
If the site is uploaded to a subfolder on your domain, then the "Domain Name" entered in the dialog needs to be "www.mysite.com/subfolder". For In-Browser Editing (and other features) to work correctly, they need to know the URL for where the site is located.