Symmetri Developer Blog

This is an issue which caused me much grief, and the solution took quite a lot of research to uncover; it is worth putting it on the record for the benefit of others. As part of my last ASP.NET project, we provided users with a payment page that allowed them to take payments from credit cards using a payment gateway. The code (in C#) was using the WebRequest object to make a HTTP POST request to the gateway URL, and a WebResponse object was used to get the returning XML response from the gateway:

objWebRequest = (HttpWebRequest)System.Net.HttpWebRequest.Create(m_sSecurePayURL);
objWebRequest.Method = “POST“;
byte[] objByteArray = Encoding.UTF8.GetBytes(sURLData);
objWebRequest.ContentType = “application/x-www-form-urlencoded“;
objDataStream = objWebRequest.GetRequestStream();
objDataStream.Write(objByteArray, 0, objByteArray.Length);
objDataStream.Close();
objWebResponse = objWebRequest.GetResponse();

That’s all fairly standard functionality, and everything worked nicely on both development and testing servers. After going live with beta-testing, however, I got a panicked call from the client, saying that there were payments being made multiple times on the same credit card; customers were ringing up to complain about being double-charged. I checked the logs, and what I found was that the WebRequest.GetResponse() was throwing the following exception:

WebException: The server committed a protocol violation. Section=ResponseStatusLine - System.Net.HttpWebRequest.GetResponse()

The exception was only being thrown intermittently; the same payment would work only a few seconds later. Worst of all, despite the exception, sometimes the request was still being sent successfully to the payment gateway server. The user, of course, would see an error and retry the payment, unaware that the first payment had gone through. At the point of time that the exception was being thrown, the state of the request was completely opaque to my application; I had no way of knowing if the payment had gone through or not.

Things were looking pretty bad: I was faced with a bug that was intermittent, poorly understood, environment-dependent and generated deep within the bowels of an atomic .NET fx function over which I had no control; not to mention a bug that had unpleasant real-world implications for the client’s business.

I did my research, and after much searching and reading I found only one real solution on the Web, which was to add the following to the web.config file:

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing=“true“ />
</settings>
</system.net>

(See for example, https://channel9.msdn.com/forums/TechOff/257803-HttpWebRequestResponse-The-server-committed-a-protocol-violation-SectionResponseStatusLine/)

That looked promising, but it didn’t work, even when applying the setting in code using reflection. It wasn’t the header of the request that was a problem, it seemed, but something about the request itself that was causing the server to intermittently reject the request or response on the grounds that it did not conform to the HTTP protocol. Why exactly the same request should be allowed to go through just seconds later was a mystery.

As is often the case when developing commercial software, sometimes we have be more interested in solutions that explanations. Tinkering with the properties of the HttpWebRequest object, I came across a property called ProtocolVersion. I realized that with the .NET 2.0 version of WebRequest, I was by default using the HTTP/1.1 version of the HTTP protocol. It turns out that there are some key differences between HTTP/1.0 and HTTP/1.1; generally speaking, the older version is looser in its specification. So, finally, I was at a solution to my problem; force my request and response cycle to use the HTTP/1.0 protocol instead of HTTP/1.1:

objWebRequest = (HttpWebRequest)System.Net.HttpWebRequest.Create(m_sSecurePayURL);
objWebRequest.Method = “POST“;
byte[] objByteArray = Encoding.UTF8.GetBytes(sURLData);
objWebRequest.ContentType = “application/x-www-form-urlencoded“;
// set HTTP/1.0 as the protocol version to stop intermittent protocol violations
objWebRequest.ProtocolVersion = System.Net.HttpVersion.Version10;
objDataStream = objWebRequest.GetRequestStream();
objDataStream.Write(objByteArray, 0, objByteArray.Length);
objDataStream.Close();
objWebResponse = objWebRequest.GetResponse();

It worked like a charm, and all of us could finally heave a sigh of relief. Both the problem and the solution are quite obscure. What exactly caused the exception is still not clear to me, and I’ll never have the time to investigate and find out. Hopefully, though, this description might help someone who is stuck with this same issue.