So I saw the new .exe detection got implemented, but for me, it was worse than before.
1) Closed qBittorrent before running setup.
2) Setup still insisted I have the exe running, and it was there, no idea how. So I killed it from Task Manager.
3) Install finished, ticked in the "Start" box. It started the client, but setup didn't go away. Clicking Finish with the box ticked or unticked had no effect. The client did start, but I just could not close installer. Used Task Manager to close that too.
Just keeping this thread in case the bug reports come in for the new setup/installer (so basically if this happens to others as well.)