I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy.
A few interesting things I found along the way:
- watched model/capability discovery and routing happen in real time
- looked at what gets injected into context and sent with ghost completions
- found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env)
- found the SQLite session store behind Chronicle, including previous prompts/responses
- watched the model query that history through tool calls
I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.
Overall some interesting lessons around how their harness is implemented.
One thing I found that I thought was a fun addition, is using eBPF made this even easier. No need to fight with anyone that is using certificate pinning, mTLS or anything else, you just get the raw plaintext data straight of the wire (right before encryption and right after decryption) and works nicely for most of the agents and IDE's.
That will in practice give you everything from telemetry to prompts, and its funny to see just how much some of them collect/run that is not at all related to your own ask..
A handy alternative when certain applications tend to make it harder to apply a MiTM proxy and you can dump it straight into your own scripts/programs to filter out and store it in whichever format you want for more analysis.
Surprised that works, I thought TLS was done entirely in process space.
I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?
They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik.
Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning
I wish copilot was better at coding Java. It's like using ChatGpt 5.1, even with Fable 5 or Opus 5 as models.
The other issue with copilot is how episodic memory works. Copilot writes memories after a task is completed, which means a lot of context is lost from the intermediate exploration, success/failure steps (turns), for what? Codex's multithreaded model adds the turn outputs to episodic memory (both agents submit their episodic data to ... themselves for summary) which gives better insight when working on multi-step problems.
Nice deep dive, I always wondered how copilot worked compared to similar tools. I'm shocked at the lack of of a rule for env files, I at least thought with a tool more integrated with github as a whole that would be a default but alas.
Interesting to see how big companies make compromises with security for innovation and i feel that it's comprehensible and better that doing nothing.
But i guess it also show how we can see governance problems as real opportunity for involved peoples to build good systems with an agent native perspective.
They use those apps either because of network effects or because there are few alternatives.
A few interesting things I found along the way:
- watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env) - found the SQLite session store behind Chronicle, including previous prompts/responses - watched the model query that history through tool calls
I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.
Overall some interesting lessons around how their harness is implemented.
That will in practice give you everything from telemetry to prompts, and its funny to see just how much some of them collect/run that is not at all related to your own ask..
A handy alternative when certain applications tend to make it harder to apply a MiTM proxy and you can dump it straight into your own scripts/programs to filter out and store it in whichever format you want for more analysis.
I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?
They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik.
Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning
The other issue with copilot is how episodic memory works. Copilot writes memories after a task is completed, which means a lot of context is lost from the intermediate exploration, success/failure steps (turns), for what? Codex's multithreaded model adds the turn outputs to episodic memory (both agents submit their episodic data to ... themselves for summary) which gives better insight when working on multi-step problems.