Send a HTTPS POST request with C#

The other day I wrote about my little Drag and Drop application and mentioned I wanted it to send HTTPS POST requests to an existing PHP web application. I thought it would be a relatively trivial task, but it turned out not the be as easy as I thought. First let me show you the code, and then discuss the pitfalls.

Sending the request by itself is actually pretty easy. Observe:

// this is what we are sending
string post_data = "foo=bar&baz=oof";
 
// this is where we will send it
string uri = "https://someplace.example.com";
 
// create a request
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(uri); request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
 
// turn our request string into a byte stream
byte[] postBytes = Encoding.ASCII.GetBytes(str);
 
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
Stream requestStream = request.GetRequestStream();
 
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
 
// grab te response and print it out to the console along with the status code
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
Console.WriteLine(response.StatusCode);

Piece of cake, right? Even if you don’t know C# you should be able to follow this. As a side note, can you see how freakishly verbose this shit really is? I mean, how come ruby can implement this like this:

Net::HTTP.post_form(URI.parse('https://somplace.example.com'), 
   {'foo'=>'bar', 'bas'=>'off'})

Or python for that matter:

data = urllib.urlencode({"foo" : "bar", "baz" : "oof"})
urllib.urlopen("https://somplace.example.com", data)

Is it really necessary for me to declare all that shit and go through all the motions to perform something that other languages do in a single line? This is one of these reasons why I no longer think that the traditional Java/C# approach of static typing, and overwhelming verbosity is the way to go. The functionality here is the same – and yet Ruby and Python totally win on expressiveness and readability hands down.

But I didn’t really start this post (just) to rag on the verbosity of C#. The problem with the code above is that it doesn’t work if your certificate is not valid. Why would I be posting to a web page with and invalid SSL certificate? Because I’m cheep and I didn’t feel like paying Verisign or one of the other jerk-offs for a cert to my test box so I self signed it. When I sent the request I got a lovely exception thrown at me:

System.Net.WebException The underlying connection was closed. Could not establish trust relationship with remote server.

I don’t know about you, but to me that exception looked like something that would be caused by a silly mistake in my code that was causing the POST to fail. So I kept searching, and tweaking and doing all kinds of weird things. Only after I googled the damn thing I found out that the default behavior after encountering an invalid SSL cert is to throw this very exception.

Fortunately, there is a quick and dirty workaround. You simply need to subclass the ICertificatePolicy class from the System.Security.Cryptography.X509Certificates namespace and rig it so that it always validates the certificate, no matter what:

using System.Security.Cryptography.X509Certificates;
using System.Net;
 
public class MyPolicy : ICertificatePolicy
{
  public bool CheckValidationResult(ServicePoint srvPoint, 
    X509Certificate certificate, WebRequest request, 
    int certificateProblem)
  {
    //Return True to force the certificate to be accepted.
    return true;
  }
}

Once you do that, you just need to toss the following line somewhere in your code, before you actually send the request. This will swap your default CertificatePolicy class for our fake one with the validation hack:

System.Net.ServicePointManager.CertificatePolicy 
    = new MyPolicy();

Note that the compiler may whine about this approach being an obsolete, but since this is a pretty ugly hack in itself, I paid no heed to it. There might be a better way to do this in .NET 3.5 but since it worked I let it be for now.

So there is how you do it. A more relevant exception message would be a lot of help in this circumstance. Perhaps something among the lines of “invalid certificate”? But I’m just thinking out loud here. If you ever run into this issue, here is how to solve it.

[tags]c#, .net, https, http, post, https post, http post[/tags]

This entry was posted in programming and tagged . Bookmark the permalink.



37 Responses to Send a HTTPS POST request with C#

  1. George Gomez UNITED STATES Mozilla Firefox Windows says:

    Great article, specially the certificate section.

    I have a problem. The variable post_data, the one you say you will be sending, is not used anywhere in the code you provide.

    Thank you,

    GG

    Reply  |  Quote
  2. Luke Maciak UNITED STATES Mozilla Firefox Windows Terminalist says:

    @George Gomez: Oh, yeah – copy and paste blunder.

    byte[] postBytes = Encoding.ASCII.GetBytes(str);

    Should be:

    byte[] postBytes = Encoding.ASCII.GetBytes(post_data);

    I hope this clears things up.

    Reply  |  Quote
  3. George Gomez UNITED STATES Internet Explorer Windows says:

    Yes it does, thank you very much.

    Reply  |  Quote
  4. Joel Perez ARGENTINA Internet Explorer Windows says:

    Very Good!!

    I tried to use the code posted in http://geekswithblogs.net/rakker/archive/2006/04/21/76044.aspx, it’s good… but it fails when i need to connect to an HTTPS. So your article was perfect to solve that problem.

    Thanks!

    Reply  |  Quote
  5. Timwi UNITED KINGDOM Mozilla Firefox Windows says:

    I completely agree with your assessment that the design of the API is quite crap because it makes you write so much unnecessarily verbose shit.

    However, you are wrong in claiming that this is due to static typing. It is entirely a problem of .NET’s API. Even without sacrificing static typing, one could easily have a static class Http with a static method Http.Post() that allows you to do this in a single line of code, for example:

    Http.Post(“https://somplace.example.com”, new Dictionary() { { “foo”, “bar” }, { “baz”, “oof” } });

    I notice from your posted code that Ruby requires a completely superfluous call to URI.parse(). Python requires you to call urlencode() and then handle the encoded data, which is bad practice (incomplete encapsulation). Unfortunately .NET is even worse, but it shows that everyone makes mistakes in designing their APIs.

    Reply  |  Quote
  6. Luke Maciak UNITED STATES Mozilla Firefox Ubuntu Linux Terminalist says:

    @Timwi: Yeah, you are right. It’s the API that’s the issue here, not static typing.

    Although, I must admit that the two often go together – I mean static typing, and overly verbose API’s. Or maybe it’s just that C# was designed to look and feel like Java and so it inherited it’s verbosity.

    Reply  |  Quote
  7. Timwi UNITED KINGDOM Mozilla Firefox Windows says:

    Although, I must admit that the two often go together – I mean static typing, and overly verbose API’s. Or maybe it’s just that C# was designed to look and feel like Java and so it inherited it’s verbosity.

    I disagree. The WebRequest API is the only example I can think of where C# is so overly verbose. Except for that, I feel that C# has precisely not copied Java’s verbosity. I can’t think of any other examples that are grossly more verbose than necessary.

    Of course C# is still slightly more verbose than dynamically-typed languages like Python, but clearly not unnecessarily so: the static typing you get saves you more time in frustrating debugging than a few characters can save you in typing.

    Reply  |  Quote
  8. Just a quick note: automatically accepting a self-signed SSL cert is bad security if there is anything sensitive in your POST. Someday there could be a man-in-the-middle reading your POST and you wouldn’t even know it.

    As for excessively verbose API, Java’s regex API is like this too. Compile a string into a Pattern. Use the Pattern to make a Matcher. Then call various methods on the Matcher and check the results. When it comes to regex, Ruby and Perl do in 1 line what Java does in 6.

    Reply  |  Quote
  9. joe j. UNITED STATES Mozilla Firefox Windows says:

    I ran into this exact thing with Java and the solution is pretty much the same there: fool the underlying security class into accepting all certs. X.509 is pretty well thought out, but really difficult to implement correctly. Getting all parties to agree on the web of trust is hard enough in a closed environment like a lab, but sucks in large corps. Thanks for the post; I’m climbing the C# learning curve.

    Reply  |  Quote
  10. Pedro UNITED STATES Internet Explorer Windows says:

    Dude,

    I love this article. I laughed I concurred. It was definitely better than Cats!

    Reply  |  Quote
  11. Graham UNITED STATES Mozilla Firefox Windows says:

    Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());

    From MSDN: “ReadToEnd assumes that the stream knows when it has reached an end. For interactive protocols, in which the server sends data only when you ask for it and does not close the connection, ReadToEnd might block indefinitely and should be avoided.”

    TCP considers the “End” as the closing of the TCP connection, since it has no other way of knowing if more data is on the way. If the server never closes the TCP connection, your call will bock until it does. You should use the read function to read in the number of bytes specified by the ContentLength Header of the HTTPRequest. This way you are not dependent upon the closing of the TCP connection from the server side to continue.

    Reply  |  Quote
  12. Leandro BRAZIL Internet Explorer Windows says:

    What i really want to know, how can I send some parameters, and these parameters enter into a field of type “textbox” then I would take the request of this consultation and analyze the data.
    I can do the sample code but is doesn’t solution my problem.

    how I do that?

    sorry english.

    Reply  |  Quote
  13. Luke Maciak UNITED STATES Mozilla Firefox Linux Terminalist says:

    @ Leandro:

    It’s easy – just append all the parameters into a string like this:

    string post_data = "foo=" + TextBox1.Text;
    post_data += "&bar=" + TextBox2.Text;
    post_data += "&baz=" + TextBox3.Text;

    The format is “argument1=value1&argument2=value2″ and etc…

    Does that make sense?

    Reply  |  Quote
  14. Leandro BRAZIL Internet Explorer Windows says:

    @ Luke Maciak:

    I can do that, but just in pages that submit button that send me to a another page.

    The problem that i find now is:
    The event button of the site that I need to send data, is an funcion of javascript. Is it possible?

    Thank You again

    Reply  |  Quote
  15. Luke Maciak UNITED STATES Mozilla Firefox Linux Terminalist says:

    @ Leandro:

    I’m not entirely sure what you mean here. The code above will send a POST request to a URL of your choice. This is equivalent of a submitting a form. For example, when you hit a submit button on a HTML form it usually sends either POST or GET request to some other page (or to itself).

    These request must then be handled on the server side. If you are trying to build an interface for an existing web form you should probably send the POST request to wherever that form is going. Look at the HTML code for your form. It should be something like this:

    <form action="/some/page.html" method="post">

    The action attribute tells you where to send the POST request.

    Now if the form has some sort of client-side javascript check going on, you probably have to recreate that behavior in your code. A lot of that stuff is session related or anti-spam protection so there is really no clear cut answer here.

    I guess my question is – do you have access to the server or are you building this tool for an existing website? If you do have access to the server, you can probably create a web service type page for yourself which will accept POST requests and handle them appropriately.

    Reply  |  Quote
  16. Pingback: Collecting Hardware Information using C# and WMI « Terminally Incoherent WordPress

  17. Gupta INDIA Google Chrome Windows says:

    really nice article!
    i have a problem while using ‘&’ , ‘=’ symbols in post_data as a value. tried
    using quotes.
    ex: if i want send x&yz as one of the value in post_data i am able to receive only x. the y after & is interpreted as another parameter…

    Reply  |  Quote
  18. Luke Maciak UNITED STATES Mozilla Firefox Linux Terminalist says:

    @ Gupta:

    Yes, this is by design. The server sees all the GET and POST requests as a single string like this:

    foo=x&bar=y&baz=z

    From there it parses it to extract the necessary value pairs. If you want to send a vale that contains characters & or = you need to URL encode them:

    System.Web.HttpUtility.UrlEncode(argument)

    This will convert the special characters into html entities. When you need to retrieve the value back just run the corresponding UrlDecode over it to change it back.

    I hope this helps.

    Reply  |  Quote
  19. Thommy UNITED KINGDOM Internet Explorer Windows says:

    Just thought you’d like to know, I’ve looked this up to implement it for one of your sponsors.

    How good is that.

    Great post BTW

    Reply  |  Quote
  20. hardik INDIA Mozilla Firefox Windows says:

    is there any way to get the encoding type of the site.. for example in this sentance

    byte[] postBytes = Encoding.ASCII.GetBytes(str);

    you have ASCII encoding. so, how would i know if it is ASCII or UTF or other encoding…

    need help

    thanks… good guide

    Reply  |  Quote
  21. Petra GERMANY Mozilla Firefox Windows says:

    does anyone knows the error “premature end of file” after sending an HTTPS Post as described?

    Reply  |  Quote
  22. Careful refactoring… If you set the POST method after you populate the postBytes RequestStream, it compiles, but gives you the exception

    “Cannot send a content-body with this verb-type.”

    If you set it first, then set request.Method = "POST" again later, it wipes your data.

    Reply  |  Quote
  23. shopping THAILAND Mozilla Firefox Windows says:

    does anyone knows the error “premature end of file” after sending an HTTPS Post as described?

    Reply  |  Quote
  24. Jason UNITED STATES Mozilla Firefox Windows says:

    Any insight on the following?

    System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. —> System.DllNotFoundException: Unable to load DLL ‘security.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E) at System.Net.UnsafeNclNativeMethods.SafeNetHandles_SECURITY.EnumerateSec urityPackagesW(Int32& pkgnum, SafeFreeContextBuffer_SECURITY& handle) at System.Net.SafeFreeContextBuffer.EnumeratePackages(SecurDll Dll, Int32& pkgnum, SafeFreeContextBuffer& pkgArray) at System.Net.SSPISecureChannelType.EnumerateSecurityPackages(Int32& pkgnum, SafeFreeContextBuffer& pkgArray) at System.Net.SSPIWrapper.EnumerateSecurityPackages(SSPIInterface SecModule) at System.Net.SSPIWrapper.GetVerifyPackageInfo(SSPIInterface secModule, String packageName, Boolean throwIfMissing) at System.Net.Security.SecureChannel..ctor(String hostname, Boolean serverMode, SchProtocols protocolFlags, X509Certificate serverCertificate, X509CertificateCollection clientCertificates, Boolean remoteCertRequired, Boolean checkCertName, Boolean checkCertRevocationStatus, LocalCertSelectionCallback certSelectionDelegate) at System.Net.Security.SslState.ValidateCreateContext(Boolean isServer, String targetHost, SslProtocols enabledSslProtocols, X509Certificate serverCertificate, X509CertificateCollection clientCertificates, Boolean remoteCertRequired, Boolean checkCertRevocationStatus, Boolean checkCertName) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) — End of inner exception stack trace — at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context) at System.Net.HttpWebRequest.GetRequestStream() at kerberosTest._Default.Page_Load(Object sender, EventArgs e)

    Reply  |  Quote
  25. nothing VIET NAM Mozilla Firefox Windows says:

    does anyone knows the error “premature end of file” after sending an HTTPS Post as described?

    Reply  |  Quote
  26. c#freak UNITED STATES Mozilla Firefox Windows says:

    @ Luke Maciak:

    Hi
    Great post! Thanks much.
    I have a doubt, it there a way to POST a file to an URI, instead of a string (in this example post_data)

    Reply  |  Quote
  27. Tal ISRAEL Google Chrome Windows says:

    Hi Luke,
    Thanks for a great post, really helped me out (and made me laugh along the way).
    You might wish to attach a CookieContainer to the request and pass it onward to further requests on that domain.

    CookieContainer cookieJar = new CookieContainer();
    request.CookieContainer = cookieJar;
    .
    .
    .
    HttpWebRequest secondRequest = (HttpWebRequest)WebRequest.Create(ANOTHER_URI);
    secondRequest.CookieContainer = cookieJar;

    Reply  |  Quote
  28. Jiri Zapletal CZECH REPUBLIC Mozilla Firefox Windows says:

    Hi, could you tell me how it will look like the PHP script to process the request and response?
    Sorry for my English:)

    Reply  |  Quote
  29. Luke Maciak UNITED STATES Mozilla Firefox Windows Terminalist says:

    @ Jiri Zapletal:

    Really dude? It’s a regulat POST request. If you are using PHP then you do something like:

    <?php
     
    $foo = $_POST["foo"];
    $bar = $_POST["bar"];
     
    // do stuff with $foo and $bar
     
    ?>

    I mean, that’s all there is to it. The code above just sends a POST request and passes along the values of foo and bar.

    Reply  |  Quote
  30. Billa Internet Explorer Windows says:

    It didn’t work for me.. I got below mentioned error. when i try to access https site from my app server.

    From my app server port 80 is blocked in firewall. only port 443 is opened

    Unable to connect to the remote server Exception : System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 141.13.20.119:8080 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
    at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP)
    at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)

    Reply  |  Quote
  31. Crack Monkey UNITED STATES Mozilla Firefox Windows says:

    @ Jason:
    You are missing a DLL.. It says right at the beginning that it can’t find a required DLL.

    Reply  |  Quote
  32. mahendra varandani UNITED STATES Mozilla Firefox Windows says:

    Hi luke
    Tahnks for the article.. but i had a secure connection site,which accept the data only if the URL contain the username and password:
    eg:
    https://username:password@ip.com?post_data

    and it is giving me the error :
    The remote server returned an error: (401) Unauthorized.
    on line:
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    please assist me….

    Reply  |  Quote
  33. Shyama Deo CANADA Internet Explorer Windows says:

    Hello, I think your blog might be having browser compatibility issues.
    When I look at your blog in Safari, it looks fine but when
    opening in Internet Explorer, it has some overlapping.
    I just wanted to give you a quick heads up! Other then that, wonderful blog!

    Reply  |  Quote
  34. BenN UNITED STATES Internet Explorer Windows says:

    New code to do the same thing, without using the obsolete CertificatePolicy property –

    System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; };

    Reply  |  Quote
  35. Optimus13 UNITED STATES Google Chrome Windows says:

    Hi. Nice blog. It really helped me out .
    But I am facing an issue on POST -ing something to the SSL enabled server.
    Error message 401.3: You do not have permission to view this directory or page using the credentials you supplied.

    While debugging, when the POST is sent, your code is executed and it returns the ‘true’ value. On coming back to POST, it throws an exception as shown above. Any inputs?

    Thank You!

    Reply  |  Quote
  36. cusman Opera Windows says:

    BenN wrote:
    New code to do the same thing, without using the obsolete CertificatePolicy property –

    System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; };

    This works perfectly. No MyPolicy class required. Thanks

    Reply  |  Quote
  37. david Google Chrome Windows says:

    Came for the code, leave saying that I agree that c# is waaaaaay to verbose compared to other languages.

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>