Removing the Ledger (or other hardware wallet) connection from app.thorswap.finance then refreshing the page and trying again can be helpful if you are experiencing issues connecting your wallet.
Click the icon to the left of the URL and then the Reset permission button