Was using my SO's laptop, I had been talking (not searching, or otherwise typing) about some VPN solutions for my homelab, and had the curiosity to use the new big copilot button and ask what it can do. The beginning of this context was actually me asking if it can turn off my computer for me (it cannot) and I ask this.
Very unnerved, I hate to be so paranoid to think that it actually picked up on the context of me talking, but again: SO's laptop, so none of my technical search history to pull off of.
Copilot is weird and can give out very weird responses that have little to do with your conversations.
And of course it might just grab context depending on what you do (e.g. clicking the copilot button might already do that).
I found it works best as GPT model if you disable the fancy stuff like search. It too easily looses track of what happened or completly goes off the rails.
(i believe disabling search is a beta feature in some regions, but its a hidden flag you can theoretically set, i made a tampermonkey script to add a button).
I hate the slow UI of Copilot, so i translate requests from a different GPT interface.