精通
英语
和
开源
，
擅长
开发
与
培训
，
胸怀四海
第一信赖
精通
英语
和
开源
，
擅长
开发
与
培训
，
胸怀四海
第一信赖
锐英源精品开源心得，转载请注明：“锐英源www.wisestudy.cn，孙老师作品，电话13803810136。”需要全文内容也请联系孙老师。
I tend to reinvent the wheel out of boredom, and decided to write a streaming music capture program a few years back. It watched the caption on a browser window running a stream from last.fm and recorded everything coming out of the speakers. When the caption changed, it took the recorded chunk, parsed the previous caption to determine the song title, saved it off as a .wav file, and then started recording for the next song. It would then spawn a new thread and run Lame.exe from the command line to convert the .wav to an .mp3. It worked great other than the occasional email beep embedded in a song when I would forget to turn off my system sounds or something.
几年前我倾向于重新发明,并决定写一个流媒体音乐捕获程序。有一个浏览器窗口里运行了从last.fm来的流，且记录所有的扬声器输出。当标题改变时,它取出记录块，解析前面的标题来确定这首歌的标题,保存下来作为一个wav文件,然后开始记录下首歌。它会分配一个新线程来运行lame.exe转换wav到mp3。当我忘了关掉我的系统声音或其它事时，它会偶尔的把beep嵌入歌曲内，其它的都很棒。
Then I was working on another project and using Fiddler to debug some web traffic and noticed Pandora sending m4a files across. I did some more digging and found identifying XML data, and PandoraCapture was born. This article is a discussion of the use of FiddlerCore to capture the audio and XML streams, along with the UltraId3Lib library to add metadata to mp3 files.
然后我从事另一个项目使用Fiddler调试一些网络流，发现Pandora发送m4a格式文件。我做了一些更多的挖掘，发现识别XML数据,并诞生了PandoraCapture。本文讨论使用FiddlerCore捕捉音频和XML流，也附带UltraId3Lib库来添加元数据添加到mp3文件。
The source zip contains all of my code. It started as a single windows form and has been refactored from there, so it's not pretty. An Options form provides a GUI wrapper to the Settings class, which loads and saves all settings to the registry. The defaults are set up for my particular work style, not what they should probably be "in the wild". For instance, the default is to not set the app up as the system proxy. The reason for this is that outlook at our company often fails when fiddler takes over. So instead I have a firefox profile specifically for internet radio that is hardcoded to use the PandoraCapture proxy and everything else bypasses it. But for the general populous, setting the system proxy is probably the easiest way to get it set up.
源代码zip包含了所有代码。它开始作为一个单一窗口的形式，并从那里被重构，所以它不是很好。属性配置窗口提供了一个GUI包装器来配置设置项，配置功能加载和保存所有设置到注册表中。特别工作风格是默认值设置，而不是“随机”成立的。例如，默认没有设置应用是系统代理。这样做的原因是，当fiddler接手管理时经常出现故障。所以我有别的选择，用Firefox配置来连接互联网电台，连接是硬编码方式使用了PandoraCapture代理并且绕过其它别的一切。但对于一般的人来说，设置系统代理可能设置的最简单方法。
On to the guts. 试试胆量
The backbone of this app is FiddlerCore. The usage is fairly simple, as it is a .net object-oriented library and there's no p-invokes or anything like that to deal with. The following block initializes the proxy server: 这个程序的支柱是FiddlerCore。使用非常简单,因为它是一个没用p-invokes的.net面向对象库。以下块初始化代理服务器:
public void ResetProxy() {
Fiddler.FiddlerApplication.AfterSessionComplete -= FiddlerApplication_AfterSessionComplete;
Fiddler.FiddlerApplication.BeforeRequest -= FiddlerApplication_BeforeRequest;
Fiddler.FiddlerApplication.Shutdown();
var settings = Fiddler.FiddlerCoreStartupFlags.Default;
if (!Settings.SetSystemProxy) {
settings &= ~Fiddler.FiddlerCoreStartupFlags.RegisterAsSystemProxy;
}
Fiddler.FiddlerApplication.AfterSessionComplete += FiddlerApplication_AfterSessionComplete;
Fiddler.FiddlerApplication.BeforeRequest += FiddlerApplication_BeforeRequest;
Fiddler.FiddlerApplication.Startup(Settings.ProxyPort, settings);
}
Because I also use this routine when Settings.ProxyPort or Settings.UseSystemProxy is changed, I first shut down the proxy. This doesn't affect anything if the proxy hasn't started yet. I then tell Fiddler whether or not it will be the system proxy, set up the event handlers, and then start up the proxy.
因为我也用这个程序，当Settings.ProxyPort或Settings.UseSystemProxy改变时，我先关闭代理。如果代理尚未开始，这不影响任何东西。那么我告诉Fiddler它是否会成为系统代理，设置事件处理程序，然后启动代理。
The FiddlerApplication_AfterSessionComplete event handler is the only one that matters. I use FiddlerApplication_BeforeRequest simply to display a status showing URLs being requested. FiddlerApplication_AfterSessionComplete事件处理程序是唯一一个关注点。我使用FiddlerApplication_BeforeRequest简单地显示一个状态显示被请求的url。
Inside FiddlerApplication_AfterSessionComplete, I check the host name against a regex to determine if it is a Pandora or last.fm session. Since these are handled differently, they branch to separate routines. I pass the fiddler session to the routines so that they have all of the information they need.
FiddlerApplication_AfterSessionComplete内部，我检查主机名称，用一个正则表达式来确定它是否是一个Pandora或last.fm会话。由于这些处理方式不同，他们转移到单独的程序。我通过Fiddler会话的套路，让他们有所有他们需要的信息。
if (Regex.IsMatch(session.hostname, @"(^|\.)((pandora\.com)|(p-cdn\.com))$")) {
handlePandoraSession(session);
} else if (Regex.IsMatch(session.hostname, @"(^|\.)last\.fm$")) {
handleLastFmSession(session);
}
Inside handlePandoraSession, I determine whether the page being requested is an XML file. If so, I use an XPath query to determine whether it is the XML I actually want; namely, the one that holds the metadata and links for the audio files. I break the XML fragments out into a dictionary keyed off of the URL for the audio.
handlePandoraSession内部,我确定被请求的页面是一个XML文件。如果是这样,我使用XPath查询来确定它是我真正想要的XML,也就是说,一个拥有音频文件的元数据和链接。我拆开了XML片段到字典，该字典的键来自URL。
Now, it gets a little weird. When I get an audio file from Pandora, it doesn't actually match the URLs I have stored. The URL has a token on it, and that token changes slightly between what was received in the XML and what is sent by the client app. So instead of being able to actually do something like pandoraXmlFragments[url], I chop 75 characters off of the end of each URL, then compare the remainder to that same section of the incoming URL. The URLs are different lengths, so I can't hardcode that length or anything.
现在,它变得有点奇怪。当我从Pandora找到一个音频文件,它实际上和存储的URL并不匹配。URL有一个令牌,这令牌在得到的XML和客户端应用程序发送之间会略有变化。所以能够做类似pandoraXmlFragments[URL],我砍掉每个URL的末尾的75个字符,然后把剩下的相同部分和传入的URL比较。url长度不同,所以我不能硬编码长度或其它事。
foreach (string key in pandoraXmlFragments.Keys) {
if (session.oResponse.MIMEType == "text/xml") {
var doc = new XmlDocument();
doc.LoadXml(session.GetResponseBodyAsString());
if (doc.SelectSingleNode("/methodResponse/params/param/value/array/data/value/struct
/member/name[text()=\"audioURL\"]") != null) {
foreach(XmlElement node in doc.SelectNodes("/methodResponse/params/param/value/array/
data/value/struct/member/name[text()=\"audioURL\"]")) {
var audioUrl = node.SelectSingleNode("../value/text()").Value;
var fragment = (XmlElement)node.SelectSingleNode("../..");
pandoraXmlFragments.Add(audioUrl, fragment);
}
}
} else if (session.oResponse.MIMEType == "audio/mp4") {
string test = session.fullUrl;
if (key.Substring(0, key.Length - 75) == test.Substring(0, key.Length - 75)) {
In the last.fm handler, the exact URL is used, but that URL is then redirected to a different one. So I watch for 302 statuses and move the fragment from the original url to the new one in the dictionary so that I have the proper key when the audio comes down. 在last.fm处理程序,使用了精准URL,然后重定向到一个不同的URL。所以我看302的状态，并将原始url的片段移动到新的字典,这样当音频结束时，我有适当的键。
if (session.responseCode == 302) {
if (lastFmXmlFragments.ContainsKey(session.fullUrl)) {
string station = lastFmXmlFragments[session.fullUrl].Key;
XmlElement fragment = lastFmXmlFragments[session.fullUrl].Value;
lastFmXmlFragments.Remove(session.fullUrl);
lastFmXmlFragments.Add(session.oResponse.headers["Location"], new KeyValuePair<string, XmlElement>(station, fragment));
}
}
Once I have determined the proper entry, I use XPath again to parse each of the major pieces of metadata from it so that I can build a filename for the audio file. I then decode the response (because it may be gzip-ed or something) and save the file with FiddlerCore. 一旦我已经确定适当的条目,我再次使用XPath解析每个主要部分的元数据,这样我就可以构建一个音频文件的文件名。然后我解码响应(因为可能gzip-ed什么的)与FiddlerCore合作并保存文件。
var fragment = pandoraXmlFragments[key];
var title = fragment.SelectSingleNode("member/name[text()=\"songTitle\"]/../value/text()").Value;
var album = fragment.SelectSingleNode("member/name[text()=\"albumTitle\"]/../value/text()").Value;
var artist = fragment.SelectSingleNode("member/name[text()=\"artistSummary\"]/../value/text()").Value;
var filePrefix = string.Format("{0} - {1} - {2}", title, album, artist);
session.utilDecodeResponse();
var audioFile = Path.Combine(folder,string.Format("{0}.m4a", filePrefix));
var xmlFile = Path.Combine(folder, string.Format("{0}.xml", filePrefix));
session.SaveResponseBody(audioFile);
var xmlFragmentData = new StringBuilder();
using (var x = XmlWriter.Create(xmlFragmentData)) {
fragment.WriteTo(x);
x.Close();
}
File.WriteAllText(xmlFile, xmlFragmentData.ToString())
ProcessPandoraFiles(audioFile, xmlFile);
pandoraXmlFragments.Remove(session.fullUrl);
When I save the file, I save the XML fragment alongside it. Pandora sends m4a files instead of mp3, and m4a can't have tags (or at least not the same tags as mp3). So I spawn a new thread that runs FFmpeg from the command line and converts the m4a file to an mp3. One caveat here is that FFmpeg, by default, uses ID3v2.4, whereas the ID3UltraLib only currently supports ID3v2.3. So when I call FFmpeg, I have to pass -id3v2_version 3 on the command line to make sure that they are compatible.
当我保存文件,我独立保存了XML片段。Pandoram使用4a格式文件,而不是发送mp3,m4a格式不能有标签(或至少不像mp3相同的标签）。我从命令行生成了一个新线程，在线程里让FFmpeg运行,把m4a格式文件转换为mp3。这里有一点需要注意的是，FFmpeg默认情况下,使用ID3v2.4,而ID3UltraLib目前只支持ID3v2.3。所以当我在命令行上使用FFmpeg,我必须通过-id3v2_version 3,以确保他们是兼容的。
Once I have a valid mp3, either from Pandora via FFmpeg, or directly from last.fm, I start setting the metatags. UltraId3Lib is another .net, object-oriented library, so there aren't any p-invokes and I don't have to know the format of an mp3 or anything. I simply load up the file, set all of the tags I have, and write it back out.
一旦我有一个有效的mp3,无论从Pandora通过FFmpeg,或直接连接last.fm,我开始设置元标签。UltraId3Lib是另一个.net面向对象库,所以我不必知道mp3格式或调用任何东西。我简单地加载的文件所有的标签,并把它写出来。
var id3 = new HundredMilesSoftware.UltraID3Lib.UltraID3();
id3.Read(audioFile);
id3.Album = fragment.SelectSingleNode("member/name[text()=\"albumTitle\"]/../value/text()").Value;
id3.Artist = fragment.SelectSingleNode("member/name[text()=\"artistSummary\"]/../value/text()").Value;
id3.Title = fragment.SelectSingleNode("member/name[text()=\"songTitle\"]/../value/text()").Value;
foreach (XmlNode genre in fragment.SelectNodes("member/name[text()=\"genre\"]/../value/array/data/value/text()")) {
id3.ID3v2Tag.Frames.Add(new HundredMilesSoftware.UltraID3Lib.ID3v23GenreFrame(genre.Value));
}
var node = fragment.SelectSingleNode("member/name[text()=\"composerName\"]/../value/text()");
if (node != null && node.Value != null && node.Value.Length > 0) {
var composers = new HundredMilesSoftware.UltraID3Lib.ID3v23ComposersFrame();
composers.Composers.Add(node.Value);
id3.ID3v2Tag.Frames.Add(composers);
}
node = fragment.SelectSingleNode("member/name[text()=\"amazonUrl\"]/../value/text()");
if (node != null && node.Value != null && node.Value.Length > 0) {
id3.ID3v2Tag.Frames.Add(new HundredMilesSoftware.UltraID3Lib.ID3v23CommentsFrame(node.Value, "Amazon URL"));
}
id3.ID3v2Tag.Frames.Add(new HundredMilesSoftware.UltraID3Lib.ID3v23CommentsFrame(xml, "Pandora XML Fragment"));
id3.Write();
last.fm and Pandora have different sets of data, so not all of the tags can be set in both. For instance, I don't have a way to get the genre from last.fm, so I use the radio station name, simply so I can sort later. last.fm和Pandora有不同的数据集,所以不是所有的标签都可以设置。例如,我没有办法从last.fm获取流派,所以这样我只可以用电台的名称,后面简单地排序
I write out all of the tags other than the album artwork first. I want to make sure that the write doesn't fail just because of something screwed in the artwork, and I don't want to do any complicated exception handling. So I do a write before loading the artwork, then do a second write for the album art. To get it, I use an HttpWebRequest to download it from the URL specified in the XML file. I basically trap and ignore any exceptions that occur at that stage.
我写出所有的标签除了专辑作品。我想确保不失败仅仅因为一些艺术品方面的复杂性,我不想做任何复杂的异常处理。在加载艺术品前，第一次写,然后对专辑封面第二次写。为了得到它,我使用一个HttpWebRequest从XML文件中指定的URL下载它。我基本上可以忽略任何发生的异常。
var artPath = "member/name[text()=\"artRadio\"]/../value/text()";
node = fragment.SelectSingleNode(artPath);
if (node != null && node.Value != null && node.Value.Length > 0) {
var web = System.Net.HttpWebRequest.Create(node.Value);
try {
using (var response = web.GetResponse()) {
using (var stream = response.GetResponseStream()) {
using (var bmp = new Bitmap(stream)) {
var pic = new HundredMilesSoftware.UltraID3Lib.ID3v23PictureFrame(bmp, HundredMilesSoftware.UltraID3Lib.PictureTypes.CoverFront,
Album art", HundredMilesSoftware.UltraID3Lib.TextEncodingTypes.Unicode) pic.MIMEType = response.ContentType;
id3.ID3v2Tag.Frames.Add(pic);
id3.Write();
}
stream.Close();
}
response.Close();
}
} catch (Exception ex) {
}
}
Obviously, last.fm and Pandora have different XML formats, so each one uses different XPath queries. I like last.fm better, I think, because they are using a standard playlist namespace. However, this causes me problems with my XPath queries because I have to specify the namespace every time. So before I load the last.fm XML, I rip off the namespace attributes.
很明显,last.fm和Pandora有不同的XML格式,因此每一个使用不同的XPath查询。我喜欢last.fm,因为他们是使用一个标准的播放列表的名称空间。然而,我的问题就出现在了XPath查询上,因为我每次都必须指定名称空间。所以加载last.fm XML前，我去掉名称空间的属性。
// without ripping off the namespace
doc.LoadXml(xml);
var ns = new XmlNamespaceManager(doc.NameTable);
ns.AddNamespace("xspf", "http://xspf.org/ns/0/");
if (doc.SelectSingleNode("/lfm/xspf:playlist/xspf:trackList/xspf:track/xspf:location", ns) != null) {
// with ripping it off
doc.LoadXml(xml.Replace(" xmlns=\"http://xspf.org/ns/0/\"", ""));
if (doc.SelectSingleNode("/lfm/playlist/trackList/track/location") != null) {
So if I don't rip it off, all of my xpath queries get longer because I have to add the namespace prefix, and I have to always pass the XmlNamespaceManager around. Now, it may turn out at some point in the future that this bites me in the butt if last.fm ever adds some other type of tracklist or something in there, but I doubt that will happen.
如果我不去掉它,我所有的xpath查询就会变长,因为我要通过XmlNamespaceManager添加名称空间前缀,现在,增加曾经一些其他类型的专辑曲目调频，但我怀疑它可能在未来某个时候出错。
Another thing you may notice in the code is that I don't handle any of the ID3UltraLib exceptions. Some exceptions it will actually throw, and others it merely returns; basically, they are warnings. I don't care about them so I don't worry about them. Another thing you'll notice is that I spell out the full namespace for UltraID3Lib in most places. I tend to do that with any new library I use. It helps me start memorizing the structure, and helps me bypass the help file; I often find methods or objects on a library that I wouldn't have found otherwise as they pop up through intellisense.
在代码中你可能会注意到的另一件事是,我不处理任何ID3UltraLib异常。它会抛出异常,其他人只是返回;基本上,我不在乎他们警告，所以我不担心。你会注意到的另一件事是,我拼出的完整名称空间来使用UltraID3Lib。我倾向于做新库来使用。它帮助我开始记忆结构,并帮助我绕过帮助文件,我经常在库里找到方法或对象,否则我就不会发现他们，尽管可以通过智能感知弹出。
This app is also a good(?) intro to slightly complicated XPath queries, especially in the Pandora XML files. Pandora doesn't use tag names to differentiate the parts, but instead uses a member element with a name and value child elements. You have to look at the text of the name element to determine what the text of the value element applies to. So the XPath query member/name[text()=\"albumTitle\"]/../value/text() says to pull the text associated with the value child element of a member tag whose name element contains the text "albumTitle".
这个程序也是一个好的(?)介绍稍微复杂的XPath查询,尤其是在Pandora的XML文件。Pandora不使用标签名称来区分的部分,而是使用一个成员元素和子元素名称和值。你必须看看name元素的文本来确定适用于价值的文本元素。XPath查询成员/名字[text()= \ " albumTitle \ "]/ .. /value/ text()会提取关联文本，关联者是成员标签的有价值子元素，相关标签名称会包含文本“albumTitle”。
A final weirdness with last.fm is that sometimes the mp3 files I receive already have tags. I'm not doing any check for this, so I have a few last.fm mp3s that have a proper genre and then have the station name added to the list. 最后一个和last.fm相关的古怪的是，有时我已经收到了有标签的mp3文件。我没有做这方面的任何检查，所以我有一些last.fm的mp3s文件，这些文件有正确的流派，然后把站名添加到列表中。
This app is very specifically coded for last.fm and Pandora audio files. However, if you wanted to find out the APIs needed for any of the music search sites, you could capture any incoming audio file and then query the music services to gather tags. Also, you could capture the HTML page that is loading the audio and may be able to pull other data from there. For instance, if I parsed the last.fm html file, I could take the tags associated with that song and add those as genres.
这个程序是专门为last.fm和Pandora的音频文件编码，然而,如果你想找出所需的api来适配音乐搜索网站,您可以捕获任何传入的音频文件,然后查询音乐服务收集标签。你也可以捕捉HTML页面和其他数据加载音频。举个例子,如果我最后进行解析调频html文件,我可以标记与这首歌有关的流派并添加。
I've also found that this works on various video sites. Youtube, as a bad example, sends flv files across. Now, YouTube is a bad example because they send each chunk as a separate file. Of course, it's possible all of the video sites work the same way. I tested with another site and received a single complete flv file, but it was also a 1 minute video, so it may have just been shorter than the chunk size. But I imagine the chunks simply need concatenating, although I haven't tested it. After that, you would simply have to parse the HTML or look for a similar XML to tell you what the file was.
我还发现,这适用于各种视频网站。Youtube,作为一个坏的例子,发送flv文件。现在,YouTube是一个不好的例子,因为他们把每个块作为一个单独的文件。当然,有可能所有的视频网站都是以相同的方式工作。我和另一个网站测试,收到一个完整的flv文件,但它也是一个1分钟的视频,所以它可能只是短于块的大小。虽然我没有测试过，但是我想只需要连接块,之后,你只需要解析HTML或寻找一个类似的XML来告诉你什么是文件。