锐英源软件
第一信赖

精通

英语

开源

擅长

开发

培训

胸怀四海 

第一信赖

当前位置:锐英源 / 英语翻译 / 物联网增强现实AR技术应用
服务方向
人工智能数据处理
人工智能培训
kaldi数据准备
小语种语音识别
语音识别标注
语音识别系统
语音识别转文字
kaldi开发技术服务
软件开发
运动控制卡上位机
机械加工软件
软件开发培训
Java 安卓移动开发
VC++
C#软件
汇编和破解
驱动开发
联系方式
固话:0371-63888850
手机:138-0381-0136
Q Q:396806883
微信:ryysoft

锐英源精品开源心得,转载请注明:“锐英源www.wisestudy.cn,孙老师作品,电话13803810136。”需要全文内容也请联系孙老师。


物联网增强现实AR技术应用


Introduction

The problem we are trying to solve is to augment the IoT device's sensory preception, to complement it when one of the sensor is decomissioned or faulty. A likely example would be when we use our phone as the primary locator within an office building. When we misplace the phone, our location identified is not accurate ( example an IoT based coffee maker stationed at the location, where is phone is located would suddenly turn on to serve us coffee, but we are no longer at that location ). However with IoT platform using augmented reality, the IoT device would be able identify our location using a-priori knowledge and sensory preception from other IoT devices we interact with.

Inorder to solve the problem, we will be using the azure infrastructure as a PaaS inorder to build the platform which provides augmented sensory preceptions for the IoT devices connected to the platform.

我们试图解决的问题是增加物联网设备的感官优先级,以便在其中一个传感器退掉或出现故障时对其进行补充。一个可能的例子是我们将手机用作办公楼内的主要定位器。当我们错放手机时,我们确定的位置不准确(例如,驻留在该位置的基于物联网的咖啡机,手机所在的位置会突然开启为我们提供咖啡,但我们不再在该位置)。然而,在使用增强现实的物联网平台的情况下,物联网设备将能够使用我们与之交互的其他物联网设备的先验知识和先验感知来识别我们的位置。

为了解决这个问题,我们将使用azure平台设施作为PaaS,以构建平台,为连接到平台的物联网设备提供增强的感官先行。

Background

A background on microprocessors and Azure platform would be useful in understanding the crux of this refrence architecture tutorial. The tutorial is laid out as simple as possible, for the reader to grasp the essentials in building a PaaS solution for IoT devices.

微处理器和Azure平台的背景将有助于理解这个参考架构教程的关键。本教程布局尽可能简单,让读者掌握为物联网设备构建PaaS解决方案的基本要素。

Using the code

The code provided in the solution file is for the PaaS reference architecture. This is a POC (proof of concept) to build an IoT PaaS platform for augementing sensor data using predictive analytics.解决方案文件中提供的代码适用于PaaS参考体系结构。这是一个POC(概念验证),用于构建物联网PaaS平台,用于使用预测分析来标准化传感器数据。

架构

Arduino based sensor network

网络拓扑

For this POC we are going to use the arduino based network with motion sensors. Each transreceiver has a unique id assigned using the CHILD_ID within the arduino sketch depicted below.对于这个POC,我们将使用基于arduino的网络和运动传感器。每个收发器都具有使用下面描述的arduino草图中的CHILD_ID分配的唯一ID 。

Using Nrf24l01 Radio Transreceivers

Arduino sketch for the motion sensor to capture location using HC-SR501 motion sensor. Please refer to details on connecting the sensor at, http://www.mysensors.org/build/motion.运动传感器的Arduino草图使用HC-SR501 运动传感器捕获位置

#include <mysensor.h>
#include <spi.h>

unsigned long SLEEP_TIME = 120000; // Sleep time between reports (in milliseconds)
#define DIGITAL_INPUT_SENSOR 3   // The digital input you attached your motion sensor.  (Only 2 and 3 generates interrupt!)
#define INTERRUPT DIGITAL_INPUT_SENSOR-2 // Usually the interrupt = pin -2 (on uno/nano anyway)
#define CHILD_ID 1   // Id of the sensor child

MySensor gw;
// Initialize motion message
MyMessage msg(CHILD_ID, V_TRIPPED);

void setup() 

gw.begin();

  // Send the sketch version information to the gateway and Controller
gw.sendSketchInfo("Motion Sensor", "1.0");

  pinMode(DIGITAL_INPUT_SENSOR, INPUT);      // sets the motion sensor digital pin as input
// Register all sensors to gw (they will be created as child devices)
gw.present(CHILD_ID, S_MOTION);

}

void loop()    
{    
// Read digital motion value
boolean tripped = digitalRead(DIGITAL_INPUT_SENSOR) == HIGH;

Serial.println(tripped);
gw.send(msg.set(tripped?"1":"0"));  // Send tripped value to gw

// Sleep until interrupt comes in on motion sensor. Send update every two minute.
gw.sleep(INTERRUPT,CHANGE, SLEEP_TIME);
}

Controller Gateway Using Arduino使用Arduino的控制器网关

The sensor network communicates to the controller gateway via RF and the message packets are posted from the controller gateway to MQTT hosted on azure. 传感器网络通过RF与控制器网关通信,消息包从控制器网关发布到托管在azure上的MQTT。In order to accomplish it we are using the mqttclientgateway 为了实现它,我们使用hosted on https://github.com/FotoFieber/MySensors/blob/mqttclient/libraries/MySensors/examples/MQTTClientGateway/MQTTClientGateway.ino

Nodejs MQTT on Raspberry PI

Install nodejs on Raspberry PI and MQTT server on Raspberry PI在Raspberry PI上的Raspberry PI和MQTT服务器上安装nodejs

Follow the posts on, http://workingmatt.blogspot.com/2014/05/install-nodejs-on-raspberry-pi.html.

Install mqtt on nodejs by running,通过运行在nodejs上安装mqtt,

npm install mosca

npm install mqtt

var mqtt    = require('mqtt');
var client = mqtt.connect('mqtt://127.0.0.1');

client.on('connect', function () {
client.subscribe('MySensor');
client.publish('MyMQTT/#', '');
});

client.on('message', function (topic, message) {
// message is Buffer
// call rest api - to post location
client.end();
});

Azure PaaS platform for Augmenting Sensors用于增强传感器的Azure PaaS平台

数据

MDM

The platform MDM is to be implemented to maintain the devices and its profile information that can be used by analytics operation to augement the sensor data.

实施平台MDM以维护设备及其配置文件信息,分析操作可以使用这些信息来对传感器数据进行处理。

Event Ingestion事件摄取

Event ingestion is via a REST API exposed by the PaaS solution to be platform independent. The REST api layer is named mlidynamics.iot.io, this is a WEB API 2.0 published to Azure and hosted.事件摄取是通过PaaS解决方案公开的REST API实现的,与平台无关。REST api层名为mlidynamics.iot.io,这是一个发布到Azure并托管的WEB API 2.0。

I have hosted the API at http://mlidynamics-iot.azurewebsites.net/Help. We would publish it from our solution into the Azure subscription -

发布网站

We therefore need the IoT device connected to the internet to access this API and publish the event. Example Json is as follows,因此,我们需要连接到互联网的物联网设备才能访问此API并发布事件。示例Json如下,

{
"DeviceId": "90bec401-36da-4750-a07e-7c666767190f",
"Channel": "sample string 2",
"Location": 3,
"Temperature": 4.1,
"Humidity": 5.1,
"PresenceDetected": 6
}

The events are then published to the event hub also hosted in Azure. The devices are secured via the device id's registered in the MDM. Security is off utmost importance in IoT devices used for patient monitoring and this architecture is built with that in consideration. The code kept in the attached solution is simplified for wide audience who would be reading this article. The solution can be expanded as per requirements for the implementation domain.然后,事件将发布到Azure中托管的事件中心。通过在MDM中注册的设备ID来保护设备。在用于患者监测的物联网设备中,安全性是至关重要的,并且该体系结构是在考虑的情况下构建的。附录解决方案中保存的代码已经简化,适用于阅读本文的大量读者。可以根据实现域的要求扩展解决方案。

[Route("Publish")]
public async Task<ihttpactionresult> Publish(TelemetryModel message)
{
// do processing here
var jsonString = JsonConvert.SerializeObject(message);

_client.Send(new EventData(Encoding.UTF8.GetBytes(jsonString))
{
PartitionKey = "0"
});

return Ok();
}</ihttpactionresult>

Notification hub is setup on Azure, which has the capability to ingest millions of events. As per this document, the events are generated from the nodejs hosted on the Raspberry PI which is the controller from the IoT sensor devices.Azure上设置了通知中心,可以摄取数百万个事件。根据此文档,事件是从Raspberry PI上托管的nodej生成的,Raspberry PI是IoT传感器设备的控制器。

流程图

Notification Hubs eliminate one major complexity: you do not have to manage the challenges of push notifications. Instead, you can use a Notification Hub. Notification Hubs use a full multiplatform, scaled-out push notification infrastructure, and considerably reduce the push-specific code that runs in the app backend. Notification Hubs implement all the functionality of a push infrastructure.通知中心消除了一个主要的复杂性:您无需管理推送通知的挑战。相反,您可以使用通知中心。通知中心使用完整的多平台,扩展推送通知基础架构,并大大减少在应用后端运行的特定于推送的代码。通知中心实现推送基础架构的所有功能。

Pros of using Event Hub使用Event Hub的优点

  • Log millions of events per second
  • Simple authorization mechanism
  • Time-based event buffering
  • Elastic scale
  • Pluggable adapters for other cloud services
  • 每秒记录数百万个事件
  • 简单的授权机制
  • 基于时间的事件缓冲
  • 弹性尺度
  • 用于其他云服务的可插拔适配器


Cons of using Event Hub使用事件中心的缺点

  • No all features from Service Bus Topic exist (but is acceptable)
  • The size of the event data is limited to 256KB (the size is pretty okay)
  • Number of partitions is limited to maximum 32
  • 没有来自Service Bus Topic的所有功能(但是可以接受)
  • 事件数据的大小限制为256KB(大小相当不错)
  • 分区数限制为最多32个

There is ample documentation available on the microsoft site to setup the azure notification hub. In this I have set the notification hub named "mlidynamics-iot-telemetry". This is the hub to which the REST API publishes the events too.Microsoft网站上提供了大量文档来设置azure通知中心。在此我设置了名为“ mlidynamics-iot-telemetry ” 的通知中心。这是REST API发布事件的中心。

private readonly EventHubClient _client =
              EventHubClient
                  .CreateFromConnectionString(ConfigurationManager.AppSettings[@"Microsoft.ServiceBus.ConnectionString"],
                      "mlidynamics-iot-telemetry");

Once these events are published to the notification hub, I am using stream analytics another Azure component to process the events in about realtime and stage it on the Azure SQL database.将这些事件发布到通知中心后,我将使用流分析另一个Azure组件来实时处理事件并将其暂存在Azure SQL数据库上。

Stream Analytics provides out-of-the-box integration with Event Hubs to ingest millions of events per second. Together, Event Hubs and Stream Analytics let you process massive amounts of real-time data so you can make business decisions in real-time.Stream Analytics提供与Event Hub的开箱即用集成,每秒可以摄取数百万个事件。通过事件中心和流分析,您可以处理大量实时数据,从而实时制定业务决策。

A simple stream query is written for this POC, to create location information sent by the remote IoT sensors using the arduino coupled to a RF transmitter and proximitry sensor. This query will create a frequency count of the sensor triggers when someone comes with its proximity.为该POC编写简单的流查询,以使用耦合到RF发射器和接收传感器的arduino创建由远程IoT传感器发送的位置信息。当有人接近时,此查询将创建传感器触发器的频率计数。

SELECT       DeviceId, Location, DateAdd(minute,-15,System.TimeStamp) as WinStartTime, 
     System.Timestamp as WinEndTime, Count(*) as EventCount      FROM [telemetry-input]
      group by TumblingWindow( minute , 15 ), DeviceId, Location

The stream analytics would aggregate and store the result in a Azure Sql storage. The schema used to capture the aggregrates,流分析将聚合并将结果存储在Azure Sql存储中。用于捕获聚合的模式,

USE [mlidynamics-iot]
  GO    
SET ANSI_NULLS ON 
 GO
    SET QUOTED_IDENTIFIER ON
  GO
    CREATE TABLE [dbo].[LocTelemetry] (      [DeviceId]     CHAR (54)     NULL,
      [Location]     BIGINT        NULL,      [WinStartTime] DATETIME2 (6) NULL,
      [WinEndTime]   DATETIME2 (6) NULL,      [EventCount]   BIGINT        NULL  );
      GO
  CREATE CLUSTERED INDEX [LocTelemetry]      ON [dbo].[LocTelemetry]([DeviceId] ASC);

Azure Ml (Machine Learning) for Predictive Analytics用于Predictive Analytics的Azure Ml(机器学习)

Using Azure ML (Azure Machine Learning) to predict the closest location based on the location in the time series sensor data from the generated by the stream analytics. The machine learning predicts the closest location in that particular time window. In this POC, I have broken the time window within a 15 minute interval. The features are built on a week, minute schedule for the training set required by the Machine Learning algorithm. The stream analytics creates a frequency count of sensor hits whenever someone comes within the proximity. The analytics built is a time series segmented within a 15 minute interval. Using this time and location we would be able to predict the closes location that a person whose schedule was used in the training. Using machine learning we are augmenting the sensor data with predicted location. The sensor didn't have a notion of the location but only notion of whether an event occured.

The Experiment Is Uploaded Into Azure Ml Gallery,使用Azure ML(Azure机器学习)根据流分析生成的时间序列传感器数据中的位置来预测最近的位置。机器学习预测该特定时间窗口中的最近位置。在这个POC中,我在15分钟的间隔内打破了时间窗口。这些功能建立在机器学习算法所需的训练集的一周,分钟时间表上。当有人进入附近时,流分析会创建传感器命中的频率计数。构建的分析是在15分钟间隔内分段的时间序列。使用这个时间和地点,我们将能够预测在训练中使用其日程安排的人的关闭位置。使用机器学习,我们用预测的位置增加传感器数据。传感器没有 实验被上载到Azure Ml Gallery

 http://gallery.azureml.net/Details/77f704bf44044ac4ba00bc93d2c6681e?share=1

物联网定位

The REST API that is exposed by Azure MLAzure ML公开的REST API

https://ussouthcentral.services.azureml.net/workspaces/84c646bc34b0432d82f03dd5c236b66f/services/
79793a0a274b44a7b74aa6336dacefaf/execute?api-version=2.0&details=true  

The request json that is to be used in the POST REST API call要在POST REST API调用中使用的请求json

 {
"Inputs": {
"input1": {
"ColumnNames": [
"wk",
"hr",
"loc",
"closest loc"
],
"Values": [
[
"0",
"0",
"value",
"value"
],
[
"0",
"0",
"value",
"value"
]
]
}
},
"GlobalParameters": {}
}

The response json that would be obtained from the REST API call将从REST API调用获取的响应json

{
"Results": {
"output1": {
"type": "DataTable",
"value": {
"ColumnNames": [
"wk",
"hr",
"loc",
"Scored Labels",
"Scored Probabilities"
],
"ColumnTypes": [
"Numeric",
"Numeric",
"String",
"String",
"Numeric"
],
"Values": [
[
"0",
"0",
"value",
"value",
"0"
],
[
"0",
"0",
"value",
"value",
"0"
]
]
}
}
}
}

The sample code that can be used to predict the location, by calling the REST API exposed by Azure ML通过调用Azure ML公开的REST API,可用于预测位置的示例代码

// This code requires the Nuget package Microsoft.AspNet.WebApi.Client to be installed.
// Instructions for doing this in Visual Studio:
// Tools -> Nuget Package Manager -> Package Manager Console
// Install-Package Microsoft.AspNet.WebApi.Client

using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Formatting;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;

namespace CallRequestResponseService
{

public class StringTable
{
public string[] ColumnNames { get; set; }
public string[,] Values { get; set; }
}

class Program
{
static void Main(string[] args)
{
InvokeRequestResponseService().Wait();
}

static async Task InvokeRequestResponseService()
{
using (var client = new HttpClient())
{
var scoreRequest = new
{

Inputs = new Dictionary<string, stringtable=""> () {
{
"input1",
new StringTable()
{
ColumnNames = new string[] {"wk", "hr", "loc", "closest loc"},
Values = new string[,] { { "0", "0", "value", "value" }, { "0", "0", "value", "value" }, }
}
},
},
GlobalParameters = new Dictionary<string, string="">() {
}
};
const string apiKey = "abc123"; // Replace this with the API key for the web service
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue( "Bearer", apiKey);

client.BaseAddress = new Uri("https://ussouthcentral.services.azureml.net/workspaces/84c646bc34b0432d82f03dd5c236b66f/services/79793a0a274b44a7b74aa6336dacefaf/execute?api-version=2.0&details=true");

// WARNING: The 'await' statement below can result in a deadlock if you are calling this code from the UI thread of an ASP.Net application.
// One way to address this would be to call ConfigureAwait(false) so that the execution does not attempt to resume on the original context.
// For instance, replace code such as:
// result = await DoSomeTask()
// with the following:
// result = await DoSomeTask().ConfigureAwait(false)

HttpResponseMessage response = await client.PostAsJsonAsync("", scoreRequest);

if (response.IsSuccessStatusCode)
{
string result = await response.Content.ReadAsStringAsync();
Console.WriteLine("Result: {0}", result);
}
else
{
Console.WriteLine("Failed with status code: {0}", response.StatusCode);
}
}
}
}
}

Testing the webservice, to predict the location based on the time series data we have. In this test, we are trying to find the location on the 1st week of the week, at the 1st hour and at the 30th second assuming the sensor in the study got triggered at about that time and the closest location assuming is the bedroom.测试Web服务,根据我们的时间序列数据预测位置。在这个测试中,我们试图找到一周的第一周,第一个小时和第30个秒的位置,假设研究中的传感器在大约那个时间被触发,假设最接近的位置是卧室。

输入数据

The results, identified the living room was the closest location based on the sensor readings.结果表明,客厅是基于传感器读数的最近位置。

Result: {"Results":{"output1":{"type":"table","value":{"ColumnNames":["sec","wk","hr","loc","closest loc","Scored Labels","Scored Probabilities"],"ColumnTypes":["Int32","Int32","Int32","String","String","String","Double"],"Values":[["30","1","1","study","bedroom","living","0.997014880180359"]]}}}}

友情链接
版权所有 Copyright(c)2004-2021 锐英源软件
公司注册号:410105000449586 豫ICP备08007559号 最佳分辨率 1024*768
地址:郑州大学北校区院(文化路97号院)内