A web app for IoT data visualization

As described in some earlier posts, I have a setup at home with IoT devices that publish measurement messages to a Raspberry Pi via MQTT. The RPi stores the data in a database and also forwards the messages to a cloud service (Adafruit IO).

In this post I have made a self-hosted data visualization web app that can be accessed from any browser-enabled device.

Overview

For the web front end and back end, I use my current favorite combination of frameworks for Raspberry Pi for web development – Flask and AngularJS. As I will involve a database and some charts, the total setup includes:

  • Flask as a microframework for a REST API and for hosting the initial SPA html page.
  • pymongo for accessing MongoDB via Python
  • angular-chart.js for AngularJS directives with Chart.js
  • AngularJS for creating the Single-Page Application

I have hosted the environment on a Raspberry Pi 3 with Raspbian OS, but any OS that has Python support should work.

The complete code for this project can be fetched from https://github.com/LarsBergqvist/IoT_Charts. To try this project out, fetch the code, install the JavaScript dependencies, install the Python dependencies and start the web server. Then, browse to http://<RASPBERRY_IP>:6001/ChartData.

About the project setup

Flask requires a certain folder setup:

  • / – the root folder where the Flask web server Python script is placed
  • /templates – the folder where html templates are placed (only one html file is needed for a SPA)
  • /static – the folder where the AngularJS controllers, CSS and third-party JavaScript dependencies are placed

As Flask will render html pages with Jinja2 that uses {{}} for placeholders, a workaround for AngularJS has to be done by changing the default {{}}-binding syntax to something else, [[]] in my case.

The application uses angular-chart.js that depends on AngularJS and Chart.js. I will use npm for installing these libraries, so the first step is to install npm on the RPi (if not already done):

sudo apt-get install npm

Then, create a folder named static, move to this folder and fetch angular-char.js, AngularJS and Chart.js:


npm install angular-chart.js --save
npm install angular --save
npm install chart.js --save

A subfolder to static called node_modules will be created where the JavaScript libraries are downloaded.

Creating the back end with Flask

Flask is used for creating a REST API. If not installed for Python 3.* already, use pip to install it:

sudo pip3 install flask

As pymongo also will be needed, we need to install that library as well:

sudo pip3 install pymongo

Now, in the root folder, create a new Python script:

There is a main entry <IP>:6001/ChartData that serves up the initial html template for the Single Page application. The REST API URI:s are built up from <IP>:6001/ChartData/api/<location>/<measurement>. This categorization is based on how I have defined my IoT data (see my previous MQTT IoT posts). For example:


ChartData/api/Outdoor/Temperature - gets the data from the temperature sensor located outdoors.

ChartData/api/GroundFloor/Humidity - gets the data from the humidity sensor on the ground floor.

The data can be fetched from MongoDB (how this data is stored is described in https://larsbergqvist.wordpress.com/2016/06/26/a-self-hosted-mqtt-environment-for-internet-of-things-part-3/) or from a fake repository (just to get some data for the front end development without a complete database).

The repository implementations are defined in two separate files:

These classes are based on an abstract base class that defines the common interface and the shared base method:

With these back end files in place, we can test the REST API. First start the Flask server:


python3 charts_server.py --fake

The –fake argument makes the implementation use the fake repository so that you don’t need a database with data to test the API. When browsing to http:<IP>:6001/ChartData/api/Outdoor/Temperature, you will get JSON back:


{
measurements: {
labels: [
'09 Jul 22:24',
'10 Jul 08:00',
'10 Jul 17:36'
],
values: [
23.3,
19.1,
30.1
]
}
}

As you can see, the measurements are divided into an array of values and an array of labels. This is because the api provides ChartData that will fit the front end charts easily.

The query parameter numdays can be used for fetching data older than 24 hours, e.g. <IP>:6001/ChartData/api/Outdoor/Temperature?numdays=3.

Creating the front end

The front end consists of a html page in the templates folder and a JavaScript controller in the static folder:

The controller makes asynchronous http requests to the REST API and updates the bound $scope properties with the result when the data has been fetched.

With the front end in place and using the fake data, you will get a highly responsive SPA app that display 3 data points for today and yesterday for each sensor that is defined (the same fake is used for each sensor).

fake_data_chart.png

With real data, you will get graphs like these, with a data point per recorded sensor value:

outdoorindoor_temp_graph.png

For each graph, there is a title that contains the sensor topic plus the latest value and its recording time.

If you want to play with real data, I have made a JSON export from my MongoDB database. It can be downloaded from:

https://github.com/LarsBergqvist/IoT_Charts/tree/master/DataBaseExport

Addendum, March 2017

I’ve started storing my sensor data in an InfluxDB time series database. For the chart app I have made an additional repository that fetches the data from this source. InfluxDB is now the default source for the charts app, but MongoDB or a fake repository can still be selected with startup arguments for the app.

The code is updated in GitHub:

https://github.com/LarsBergqvist/IoT_Charts

 

Advertisements

2 thoughts on “A web app for IoT data visualization

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s