基于套接字的具备带宽限制能力的HTTP客户端

Introduction 介绍

Communication over HTTP is a very common task. Most of the .NET programmers are familiar with classes like WebRequest, WebResponse or WebClient which are very useful, but sometimes one needs features they don't offer. In these cases, we could not have another option but to write our own HTTP client implementation based on TCP sockets. This article is an example of how to do so. The reader should know the basics of HTTP protocol.

通过HTTP通信是一种很常见的任务。大部分的.NET程序员熟悉类 WebRequest , WebResponse 或 WebClient ，这是非常有用的,但有时他们不提供需要的功能。在这些情况下,我们没有别的选择，但是我们可以基于TCP套接字写自己的HTTP客户端来实现功能。这篇文章就是说明如何这样做的。读者应该知道基本的HTTP协议。

Background 背景

My first reason for writing my own HTTP client library was the need to upload large files to the server and show the current upload speed. It didn't seem to be very difficult, but I found out standard classes load all output data into memory before send despite the face that the WebRequest class offers an output stream for POST data. HTTP protocol is not really complex, so why not just open a socket on port 80 and start communication?

我写我自己的HTTP客户端库的第一个理由是需要向服务器上传大文件和显示当前上传速度。它似乎并不是非常困难的,但我发现标准类把所有输出数据加载到内存中后才发送，尽管 WebRequest 类提供了一个输出流来POST数据。 HTTP协议并不复杂,所以为什么不打开套接字80端口并开始通信吗?

I didn't want to rewrite all my application but just to do some experiments first to ensure my idea is sustainable. I needed to create an abstract interface, which enables me to return to standard solution at any time. These are my classes:

我不想重写应用程序只是做一些实验,以确保我的想法是可持续的。我需要创建一个抽象接口,它让我在任何时间可以回到标准方案。这些是我的类:

HttpConnection - represents an abstract connection to the web server which can handle only one request at a time HttpConnection ——代表了到web服务的抽象连接，它一次只能处理一个请求

HttpSocketConnection - new client implementation based on TCP sockets HttpSocketConnection -新客户端实现，基于TCP套接字

HttpWebRequestConnection - implementation using standard WebRequest and WebResponse classes HttpWebRequestConnection ——使用标准 WebRequest 和 WebResponse 类实现

HttpMessage - container for HTTP request or response including headers and data stream HttpMessage ——包含HTTP请求或响应数据，内包含有报头和数据流

These classes are all you need to run simple HTTP requests with data uploading/downloading in both ways. I also prepared some code which enables control bandwidth used by network communication. This can be accomplished simply by slowing down read streams - first for the POST data and second for server response, so I created a stream proxy class slowing down read/write operations for a given stream held inside the proxy.

这些类都需要运行简单的HTTP请求,以双工工数据上传/下载。我也准备了一些代码使得控制网络通信所使用的带宽。这仅仅通过减速流可以实现——第一次用于POST数据，第二次为服务器响应数据,所以我创建了一个流代理类,用于对于一个给定的流内部的减速流读/写操作。

WatchedStream - Serves as a proxy for given stream calling events before every read/write operation WatchedStream ——作为一个服务于指定流的代理，给每个读/写操作之前调用事件

WatchedStreamEventArgs - Event arguments for stream transfer events WatchedStreamEventArgs ——流传输事件的事件参数

HttpAdvancedStream - Uses the events from base class to insert some wait time HttpAdvancedStream ——从基类使用事件来插入一些等待时间

BandWidthManager - Responsible for measuring time and computing transferred data to keep the desired speed BandWidthManager ——负责测量时间和计算传输数据所需的速度

类图1

If you would like to use the library to send POST requests, you could use some framework to create POST body. Of course I have created one, and it can prepare request body in two common formats: simple URL encoded and multi-part format.如果你想使用库发送POST请求,您可以使用一些框架创建POST体。当然,我已经创建了一个,可以准备两个常见格式的请求主体:简单的URL编码和多部分的格式。

HttpPostBodyBuilder - abstract class for POST body builders HttpPostBodyBuilder - - - - - -抽象类POST主体构造器

HttpPostBodyBuilder.Multipart - Prepares a request body in standard multipart format which allows you upload files HttpPostBodyBuilder.Multipart ——准备请求主体，主体代表多部分格式，这样允许您上传文件

HttpPostBodyBuilder.UrlEncoded - Prepares a request body in standard URL format HttpPostBodyBuilder.UrlEncoded ——准备标准URL格式的请求主体

MergingStream - This stream reads data from multiple given streams, which allows to prepare an entire request body without the need to load all necessary data into memory MergingStream ——此流从多个给定流中读取数据,它可以准备一个完整的请求主体而不需要把所有必要的数据加载到内存中

HttpUtility - URL encoding logic extracted from System.Web.dll. This allows you to avoid referencing this library and keep application under .NET Framework Client Profile. HttpUtility ——URL编码逻辑，从System.Web.dll提取。这可以让你避免引用该库，让应用程序只使用.NET框架的客户端Profile。

类图2

Using the Code 使用的代码

I have prepared and attached a client/server sample solution for you to test and better understand the library. There is an ASP.NET web site which allows you to upload image, then it does some graphics operations and sends the result back to the browser. To invoke the service, you can either use your standard browser or the WinForms client emulating browser behavior.

我有准备附加一个客户端/服务器示例解决方案用于测试，以便更好地理解库。有一个ASP.NET的网站，你可以上传图片,然后做一些图形操作并将结果并发送回浏览器。为了调用服务,您可以使用标准浏览器或WinForms客户机模拟浏览器的行为。

测试截图

This block of the code is crucial: 这段代码是至关重要的:

this.NotifyState("Converting file...", 0)
   // choose connection
     HttpConnection http;
     switch (httpMethod)
     {
     case 0:
     http = new HttpSocketConnection();
     break;
     case 1:
     http = new HttpWebRequestConnection();
     break;
     default:
     throw new NotSupportedException();
     }
// prepare request
     var url = "http://localhost:12345/Page.aspx";
     var postBody = new HttpPostBodyBuilder.Multipart();
     var fileStream = new FileStream
     (this.openFileDialog.FileName, FileMode.Open, FileAccess.Read);
     var advStream = new BandwidthControlledStream(fileStream);
     lock (this.bandWidthSync)
     {
     this.uploadSpeed = advStream.ReadSpeed;
     this.uploadSpeed.BytesPerSecond = 1024 * (int)this.upSpeedBox.Value;
     }
     postBody.AddData(
  "imageFile",
     advStream,
     Path.GetFileName(this.openFileDialog.FileName),
     GetMimeType(this.openFileDialog.FileName)
     );
     var bodyStream = postBody.PrepareData();
     bodyStream.Position = 0;
     var req = new HttpMessage.Request(bodyStream, "POST");
     req.ContentLength = bodyStream.Length;
     req.ContentType = postBody.GetContentType();
     req.Headers["Referer"] = url;
     // send request
     advStream.BeforeRead +=
     (s, e) => this.NotifyState("Uploading...", e.Position, bodyStream.Length);
     var response = http.Send(url, req);
     // get response
     var readStream = new BandwidthControlledStream(response.Stream);
     lock (this.bandWidthSync)
     {
     this.downloadSpeed = readStream.ReadSpeed;
     this.downloadSpeed.BytesPerSecond = 1024*(int) this.downSpeedBox.Value;
     }
     readStream.BeforeRead +=
     (s, e) => this.NotifyState("Downloading...", e.Position, response.ContentLength);
     this.convertedFile = ReadAll(readStream, (int) response.ContentLength);
     this.NotifyState("Done", 100, true);

As you can see, the code snippet is not quite short, but it does a lot of work: 正如您可以看到的,不是很短的代码片段,但它确实做了很多工作:

Chooses method to upload data (WebRequests/sockets) 选择方法上传数据(WebRequests /套接字)

Prepares request body with file to upload (file is loaded continuously during sending) 准备请求主体与文件上传(文件加载期间不断发送)

Prepares speed-limited streams (you can also choose not to use them, or change speed during transfer) 准备限速流(你也可以选择不使用它们,或者在传输时改变速度)

Uses BeforeRead events to inform user about progress 使用 BeforeRead 事件通知用户进度

Shows how to set request headers 显示了如何设置请求

Limitations 限制

Please don't consider this library as full replacement of HttpWebRequest and other classes. This is only a simple solution and cannot be used in any scenario because of its limits:

请不要认为这个库是完全替代 HttpWebRequest 和其他类。这只是一个简单的解决方案,不能用在任何场景,因为它有限制:

No support for HTTPS 不支持HTTPS

No support for proxy 不支持代理

No caching 没有缓存

Does not keep opened socket 不保持打开套接字

Only few HTTP statuses are treated correctly (100, 206, 302 and of course 200) 只有少数HTTP状态正确对待(100、206、302和100年)

Only seek-capable streams can be used as source for upload 只能用作上传源seek-capable流