If you install converter server on the machine to be converted, it will be converted to the destination, too. If you want the machine to remain as clean as possible, use the other approach (agent can be deinstalled automatically).
OTOH if you have difficulties with firewall configuration, you may prefer installing converter on the source machine. Thus it will only have to connect the destination VC/ESX, otherwise you will need converter server to connect to agent, and agent to connect to destination.