锐英源精品开源心得，转载请注明：“锐英源www.wisestudy.cn，孙老师作品，电话13803810136。”需要全文内容也请联系孙老师。

基于ffmpeg的视频播放器和支持RTSP/RTMP的视频播放器

Introduction 介绍

This article is a sort of continuation of my previous article, which shows an implementation of a web camera control. Recently I created another control and would like to share my experience with community. It is a FFmpeg-based stream player control, which can do the following: 这篇文章是我延续以前的文章,显示了一个网络摄像头控件的实现。 最近我创建了另一个作品并且想与社区分享我的经验。 这是一个基于FFmpeg流媒体播放控件 ,可以执行以下操作:

Play a RTSP/RTMP video stream 播放一个 RTSP / RTMP 视频或本地视频文件

Retrieve the current frame being displayed by the control 检索当前显示在窗口上的帧

The control has no additional dependencies and a minimalistic interface. 没有额外的依赖和简约的界面。

Requirements 需求

The WinForms version of the control is implemented using .NET Framework 2.0 WinForms版本控件是实现使用.NET框架2.0

The WPF version of the control is implemented using .NET Framework 4 Client Profile 使用WPF的版本控件实现.NET框架4客户端配置文件

Both versions are built using the x86 platform target. 控件同时支持x86和x64平台的目标。

Background 背景

Streaming audio, video and data over the Internet is a very usual thing these days. However, when I tried to find a .NET control to play a video stream sent over the network, I found almost nothing. This project tries to fill up that gap.

串流音频、视频和在互联网传输数据上是一个很平常的事情。我发现几乎没有一个.NET控件可以 播放通过网络发送的视频流。 然而, 我试图用此项目来填补这一空白。

Implementation details 实现细节

If you are not interested in implementation details, then you can skip this section. 如果您对实现细节不感兴趣,那么你可以 跳过 这一节。

The implementation is divided into three layers. 实现分为三层。

The bottom layer is implemented as a native DLL module, which forwards our calls to the FFmpeg framework.底层被实现为一个本地DLL模块,转发我们对FFmpeg框架的调用

For distribution convenience, the native DLL module is embedded into the control’s assembly as a resource. On the runtime stage, the DLL module will be extracted to a temporary file on disk and used via late binding technique. Once the control is disposed, the temporary file will be deleted. In other words, the control is distributed as a single file. All those operations are implemented by the middle layer.

为了分发方便,原生DLL模块嵌入到控件的装配资源。 在运行时,DLL模块将提取到一个临时目录下，再通过后期绑定技术绑定。 一旦控件释放,临时文件将被删除。 换句话说,控件是作为单个文件分发的。所有这些操作都由中间层实现。

The top layer implements the control class itself. 顶层实现控件类本身。

The following diagram shows a logical structure of the implementation. 下图显示了一个实现的逻辑结构。

Only the top layer is supposed to be used by clients. 只有顶层应该是由客户使用。

The Bottom Layer 底层

The bottom layer uses the facade pattern to provide a simplified interface to the FFmpeg framework. The facade consists of two classes: the StreamPlayer class, which implements a stream playback functionality 底部层使用 facade 模式来提供一个简化的FFmpeg接口框架。 facade包含三个类:StreamPlayer类,它实现了一个流播放功能

   /// <summary>
      /// The StreamPlayer class implements a stream playback functionality.
      /// </summary>
      class StreamPlayer : private boost::noncopyable
      {
      public:    /// <summary>
      /// Initializes a new instance of the StreamPlayer class.
      /// </summary>
      StreamPlayer();  
 /// <summary>
      /// Initializes the player.
      /// </summary>
      /// <param name="playerParams">The StreamPlayerParams object that contains the information that is used to initialize the player.</param
      void Initialize(StreamPlayerParams playerParams);
 /// <summary>
      /// Asynchronously plays a stream.
      /// </summary>
      /// <param name="streamUrl">The url of a stream to play.</param>
      void StartPlay(std::string const& streamUrl);
      /// <summary>
      /// Retrieves the current frame being displayed by the player.
      /// </summary>
      /// <param name="bmpPtr">Address of a pointer to a byte that will receive the DIB.</param>
      void GetCurrentFrame(uint8_t **bmpPtr);
 /// <summary>
      /// Retrieves the unstretched frame size, in pixels.
      /// </summary>
      /// <param name="widthPtr">A pointer to an int that will receive the width.</param>
      /// <param name="heightPtr">A pointer to an int that will receive the height.</param>
      void GetFrameSize(uint32_t *widthPtr, uint32_t *heightPtr);
 /// <summary>
      /// Uninitializes the player.
      /// </summary>
      void Uninitialize();
      };

the Stream class, which converts a video stream into series of frames Stream类,它将一个视频转换成一系列的帧

/// <summary>
      /// A Stream class converts a stream into series of frames. 
      /// </summary>
     class Stream : private boost::noncopyable
      {
      public:
      /// <summary>
      /// Initializes a new instance of the Stream class.
      /// </summary>
      /// <param name="streamUrl">The url of a stream to decode.</param>
      Stream(std::string const& streamUrl);   
 /// <summary>
      /// Gets the next frame in the stream.
      /// </summary>
      /// <returns>The next frame in the stream.</returns>
      std::unique_ptr<Frame> GetNextFrame(); 
 /// <summary>
      /// Gets an interframe delay, in milliseconds.
      /// </summary>
      int32_t InterframeDelayInMilliseconds() const;  
 /// <summary>
      /// Releases all resources used by the stream.
      /// </summary>
      ~Stream();
      };

and the Frame class, which is a set of frame related utilities
和Frame类,这是一组帧相关的实用程序。

/// <summary>
      /// The Frame class implements a set of frame-related utilities. 
      /// </summary>
      class Frame : private boost::noncopyable
      {
      public:
      /// <summary>
      /// Initializes a new instance of the Frame class.
      /// </summary>
      Frame(uint32_t width, uint32_t height, AVPicture &avPicture);
 /// <summary>
      /// Gets the width, in pixels, of the frame.
      /// </summary>
      uint32_t Width() const { return width_;   
 /// <summary>
      /// Gets the height, in pixels, of the frame.
      /// </summary>
      uint32_t Height() const { return height_; } 
 /// <summary>
      /// Draws the frame.
      /// </summary>
      /// <param name="window">A container window that frame should be drawn on.</param>
      void Draw(HWND window);  
 /// <summary>
      /// Converts the frame to a bitmap.
      /// </summary>
      /// <param name="bmpPtr">Address of a pointer to a byte that will receive the DIB.</param>
      void ToBmp(uint8_t **bmpPtr)
 /// <summary>
      /// Releases all resources used by the frame.
      /// </summary>
      ~Frame();  
};

These tree classes form a heart of the FFmpeg Facade DLL module. 这些树类形成的FFmpeg立面DLL模块。

The Middle Layer  中间一层

The middle layer is implemented by the StreamPlayerProxy class, which serves as a proxy to the FFmpeg Facade DLL module. 中间一层是由StreamPlayerProxy实现,作为代理FFmpeg Facade DLL模块。

First, what we should do is extract the FFmpeg Facade DLL module from the resources and save it to a temporary file.首先,我们应该做的是提取FFmpeg Facade DLL模块的资源并将其保存到一个临时文件。

_dllFile = Path.GetTempFileName();
      using (FileStream stream = new FileStream(_dllFile, FileMode.Create, FileAccess.Write))
      {
      using (BinaryWriter writer = new BinaryWriter(stream))
      {
      writer.Write(Resources.StreamPlayer);
      }
      }

Then we load our DLL module into the address space of the calling process. 然后我们的DLL模块加载到调用进程的地址空间。

 _hDll = LoadLibrary(_dllFile);

      if (_hDll == IntPtr.Zero)

      {

      throw new Win32Exception(Marshal.GetLastWin32Error());

      }

And bind the DLL module functions to the class instance methods. DLL模块的功能,并将其绑定到类实例方法。

private delegate Int32 PlayDelegate();
      private PlayDelegate _play;   
// ...  
IntPtr pProcPtr = GetProcAddress(_hDll, "Play");
      _play =
      (PlayDelegate)Marshal.GetDelegateForFunctionPointer(pProcPtr, 
      typeof(PlayDelegate));

When the control is being disposed, we unload the DLL module and delete it. 当控件正在处理,我们卸载DLL模块和删除它

private void Dispose(Boolean disposing)
      {
      if (disposing)
      {
      if (_hDll != IntPtr.Zero)
      {
      FreeLibrary(_hDll);
      }  
 if (File.Exists(_dllFile))
      {
      File.Delete(_dllFile);
      }
      }
      }

The Top Layer 顶层

The top layer is implemented by the StreamPlayerControl class with the following interface. 最上面一层是由StreamPlayerControl实现,使用了以下接口。

 /// <summary>
      /// Plays a stream.
      /// </summary>
      /// <param name="url">The url of a stream to play.</param>
      /// <exception cref="ArgumentException">An invalid string is passed as an argument.</exception>
      /// <exception cref="Win32Exception">Failed to load the FFmpeg facade dll.</exception>
      /// <exception cref="StreamPlayerException">Failed to play the stream.</exception>
      public void Play(String url);
/// <summary>
      /// Retrieves the unstretched image being played.
      /// </summary>
      /// <returns>The current image.</returns>
      /// <exception cref="InvalidOperationException">The control is not playing a video stream.</exception>
      /// <exception cref="StreamPlayerException">Failed to get the current image.</exception>
      public Bitmap GetCurrentFrame(); 
/// <summary>
      /// Stops a stream.
      /// </summary>
      /// <exception cref="InvalidOperationException">The control is not playing a stream.</exception>
      /// <exception cref="StreamPlayerException">Failed to stop a stream.</exception>
      public void Stop();  
/// <summary>
      /// Gets a value indicating whether the control is playing a video stream.
      /// </summary>
      public Boolean IsPlaying { get; } 
/// <summary>
      /// Gets the unstretched frame size, in pixels.
      /// </summary>
      public Size VideoSize  { get; }

Usage 使用

First, we need to add the control to the Visual Studio Designer Toolbox, using a right-click and then the "Choose Items..." menu item. Then we place the control on a form at the desired location and with the desired size. The default name of the control instance variable will be streamPlayerControl1.

首先,我们需要将控件添加到Visual Studio设计师工具箱,使用右键单击,然后“选择项目…… ”菜单项。 然后我们将控件所需的位置和形式以及所需的大小。 控件实例变量的默认名称 streamPlayerControl1 。

The following code plays a stream using the supplied address. 下面的代码使用提供的地址播放流。

streamPlayerControl1.Play("rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov");

To get a frame being played just call the GetCurrentFrame() method. The resolution and quality of the frame depend on the stream quality.使用 GetCurrentFrame()方法得到一个帧。分辨率和帧的质量取决于流质量。

Bitmap image = streamPlayerControl1.GetCurrentFrame();

To stop the stream the Stop() method is used.使用stop()方法停止流。

streamPlayerControl1.Stop();

You can always check the playing state using the following code. 你可以使用下面的代码检查播放状态。

if (streamPlayerControl1.IsPlaying)

      {

      streamPlayerControl1.Stop();

      }

Also, the StreamStarted, StreamStopped and StreamFailed events can be used to monitor the playback state. 此外, StreamStarted , StreamStopped 和 StreamFailed 事件可以用来监控回放状态。

To report errors, exceptions are used, so do not forget to wrap your code in a try/catch block. That is all about using it. To see a complete example please check the demo application sources. 这都是关于使用它显示的报告错误,使用异常,所以不要忘记您的代码封装在一个try / catch块。 看到一个完整的示例,请查看演示应用程序的来源。

WPF version WPF版本

The FFmpeg facade expects a WinAPI window handle (HWND) in order to use it as a render target. And that is an issue in the WPF world, where windows do not have handles anymore. The VideoWindow class has been introduced to workaround this problem.FFmpeg facade期望WinAPI窗口句柄( HWND )为一个渲染目标。 WPF中的问题是,没了窗口句柄。 VideoWindow类是这一问题的解决方案。

<UserControl x:Class="WebEye.StreamPlayerControl"

      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

      xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 

      xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 

      mc:Ignorable="d" 

      d:DesignHeight="300" d:DesignWidth="300"

      xmlns:local="clr-namespace:WebEye">

  <local:VideoWindow x:Name="_videoWindow"

      HorizontalAlignment="Stretch" VerticalAlignment="Stretch"/>

  </UserControl>