By policy, don't make changes outside of Terraform. This will slow you down a little to start with, but developing the culture of infrastructure via terraform pushes is part of what makes IaC successful. Obviously you should maintain break-glass capability, but doing all changes via Terraform will get you what you want.
Its much better to think of them as two tools that complement each other. In Void Linux we use terraform for setting up resources on clouds and setting up SDN links between them. Provisioning hosts on the other hand is best left to a tool that is designed for that, and Ansible is a good tool for this. There are certainly points where I would rather have a single tool to rule them all, but I suspect in that case it wouldn't do either task particularly well.
If you absolutely had to pick one tool to do both use cases, I'd go with Terraform and have it invoke an ansible-like provisioning process.
While Ansible CAN do the stuff terraform does, it cant do it very well. But thats not its primary purpose. Ansible is for configuration management, but can do some orchestration, terraform is all about orchestration. Use the right tool for the job. Re changes outside of terraform, there should be none. Set up reporting to highlight these resources and highlight the need & advantages for using terraform to stake holders.