Login to your account at https://paybyphone.com/receipts
Click the Profile button and scroll to the bottom of the page, click Delete Account.
Check "I want to delete my account" and then the Delete Account button one last time.
Note: Once you close the account, your transaction history will be unavailable: please print any receipts that you might need prior to deletion.