If your email address is being spoofed, it means they are just faking the 'return address'. This is like trying to pin someone down for faking a return address on an envelope in usps. As far as I know, there is nothing you can do to prevent this, besides getting a new email address and informing everyone not to accept mail from your old address, that it is unverified.
I had this problem once on an account some years ago. I wound up making a new account. After looong time, it stopped. I assume the faker was shutdown.
PM me and I will give you an email to contact me. Then you can forward to me one of these 'impersonating emails' others receive from you and perhaps I can verify if they are spoofing or accessing your account.
After deleting your contacts, the emails from 'you' to the same people continued?