Writing a proxy to store and replay web sites is extremely difficult. I thought I had a simple solution, which was to store the request and its sha-256, then any request which comes in having the same sha-256 will generate the same response. Sounds easy, but in reality the referrer header field in the request can be different every time for the same page. Ok. So we just remove the referrer field (actually, because of an error in the original spec for http, it is the "Referer" field), then create the sha-256, then put the referrer field back and send the request on its way.

Problem Solved!

Not exactly. What if you have a web page you can log into. Now the cookie in the header will be different every time you log in. If you try and do the same trick with the cookie field as you do with the referrer field, you cannot actually log in. I obviously need to think about this some more, and become one with the HTTP 1.1 specification.

Any recommendations are welcome.