You don't actually recover the key, which is cryptographically impossible. You just get ownership of the account.
Perhaps you can do it with Bitcoin's scriptSig, but it's easier to do it with a cryptocurrency like Ethereum. Or if you're designing a new system you might consider making it a part of the system.
Ownership of the account is determined by possession of the private key. Only the one with the key can spend from the account.
In Bitcoin the way to implement this would be by transferring any funds in the account first to a time-locked escrow account and then to a new account for which the challenger possesses the key. It's doable but probably not worthwhile; IMHO the effort would be better spent on ensuring that you don't lose the key in the first place.
An easier solution would be for challenger to supply a new account id, and if claim is successful, all the funds are transferred there. This can all be done via smart contract. Brilliant idea, love it!
How can the funds be transferred if I still have the private key? Again, the question is who controls the keys? How can I still have my keys but the smart contract can transfer the "account" to a claimant that may or may not be me?
Perhaps you can do it with Bitcoin's scriptSig, but it's easier to do it with a cryptocurrency like Ethereum. Or if you're designing a new system you might consider making it a part of the system.