Thesis: Device type classification of IoT devices on low-end dedicated hardware devices

Introduction

The concept of the Internet of Things (IoT) device has been around since about 1990, when John Romkey introduced arguably the first IoT device: an internet-connected toaster that he demonstrated to show what the Simple Network Management Protocol (SNMP) could do. The idea was that IoT devices would use the SNMP protocol to control their functionality.

Since then, the concept has become very popular. According to Ericsson, there were 14.6 billion IoT devices connected to the internet in 2021. IoT devices have made many people’s daily lives a little easier. However, they have also brought new privacy and security challenges.

One problem is that it is easy to misconfigure IoT devices. Another is that vendors do not always secure them properly, as security is not always a priority. For example, it is still common to find hardcoded or default passwords within the firmware of IoT devices. Misconfigurations and password vulnerabilities can provide hackers and indexers easy access to these devices. There are currently several websites that index vulnerable IoT devices. A good example is Shodan, a search engine for internet-connected devices. Shodan has a section called ‘images’, where screenshots of vulnerable computers and cameras are indexed. Anyone with a Shodan subscription can browse the index. What can be found there is shocking: we were able to look into people’s daily lives without any restrictions.

Approach

During the thesis research, we focused on how to help home users restore their privacy and security, which is often compromised by vulnerable IoT devices. We realised that it would be helpful in that regard to have a device connected to your home network that is capable of ensuring a certain level of privacy and security for your household. But what should such a device actually be capable of doing? Ideally, it should be able to automatically enforce specific security rules, in order to mitigate misconfigurations or vulnerability exploitation by limiting network access. That would remove the need for the user to interact with the system, meaning that the device would be a practical solution even for users without much technical expertise. Preferably, the rules should be based on the category of the IoT device, so that more fine-grained policies could be applied. For example, you might have a rule preventing external access to the control panel of a smart candle. That would prevent a hacker turning on the candle, thus creating a fire hazard.

We quickly realised that if we wanted to focus on this approach, we would need to have a cheap solution for home users. The main reason being that the solution needs to be attractive to use. Luckily, most home users already have a single device in their household that manages all their network traffic. Namely, the home router that you often get for free from your ISP. However, ISP routers are low-end hardware devices, meaning that they have limited computational resources.

With those considerations in mind, we decided that my thesis research should focus on one feature of the system for the automatic application of security rules—namely, detection of the categories of IoT device on a network.

Implementation

There are currently three ways of classifying an IoT device described in the literature, namely:

1. Detecting whether a device is an IoT device or not

2. Classifying the make and model of the IoT device

3. Classifying the category of the device

In our research, we focused on the third method. The other two methods have several limitations that would make it hard to use them to build a system capable of applying fine-grained security policies. The limitation of the first method is that it supports only one set of rules for all IoT devices in a network. That would make it impossible to define fine-grained policies. The second option would allow us to set device-specific policies. However, that would require us to have data on all devices worldwide. Going forward, we would also need data on all the devices manufactured in the future. Without such data, we would be unable to determine the make and model, as we would have never seen the device before. That would also introduce a serious scalability issue, making the system costly to maintain and deploy. Hence, we focused on the third method for classifying devices, which allows for security policies to be defined on the device category level.

Our research focused on five categories, one more than the current norm, namely audio, camera, automation (lightbulbs, sensors and smart plugs), smart hubs (devices that often handle state changes of lightbulbs, sensors, etc in the automation category), and televisions. To correctly classify such devices, we must overcome several challenges, the biggest being ‘freedom of implementation’.

Freedom of implementation implies a vendor being free to implement any features on a given device to make the device attractive to consumers. Unfortunately, all these extra features change the network traffic and their patterns, making it hard to classify the devices. To overcome that challenge, we turned our attention to the ‘primary task’ of each device type. The primary task is the core functionality that devices within a category have in common, which we should always be able to detect. We started by creating a hypothesis for every device category, answering the following question: what would the primary task look like at the network level?

The hypotheses were created manually by considering what the devices in a particular category would mainly do in terms of network traffic. The hypotheses were then partially verified by looking at their prediction accuracy in relation to various devices. A good example of such a hypothesis involves the network behaviour of devices in the 'camera' category. Such devices often stream snapshots or entire videos to a cloud, e.g. Google Cloud. They are therefore likely to transmit more traffic than they receive. Figure 1 shows an example of the traffic pattern associated with a camera. If we see such a pattern, the probability of the device being a camera is higher than if a different pattern is observed. The transmitted bandwidth is significantly higher than the received bandwidth, matching our hypothesis.

Figure 1: Traffic pattern associated with a camera.

We identified the network features that best explain each hypothesis. The network features were selected on the basis of how easily they can be extracted. Ease of extraction is important, because they need to be extractable using a low-end device without sufficient computational power to run complex algorithms. Examples of the network features we extract are the maximum packet size, the time between network packets, and how many devices the IoT device connects to on the internal network.

We extract the features from 24 hours of traffic taken from three datasets. The datasets are the UNSW IoT Traces, Your Things and a dataset on IoT information exposure published at the IMC 2019 conference. We chose 24 hours of traffic firstly because most devices have a capture duration of at least that length, and secondly because we anticipated that device prediction would take a day. The 24 hours of network traffic was split into six blocks of four hours. We chose to split the traffic that way after doing extensive tests with various split lengths. The six blocks of four hours showed the most promising prediction accuracy. Splitting the network traffic ensures that a single irregular event does not skew all the extracted features, as it would affect only a single four-hour block. An example of such an event is a firmware update. Based on the extracted features, we can train a machine learning classifier to predict the categories of IoT devices. During this step, it is important to ensure that some devices are left ‘unseen’ by the model, which means we exclude certain devices during the training phase.

The training phase of machine learning is where it learns to perform its intended task by analysing data presented to it. Generally, training is carried out using a training dataset, while the accuracy of the algorithm is measured using a test dataset. The test dataset is not used for training, and the two datasets should be completely separate. A common approach is therefore to split the available data into a training subset and test subset, where, say, 75% of the data is used for training. In our research, if the split is not made very carefully, data from one particular device can end up in both the training dataset and the test dataset, implying a serious risk that a device presented to the model during testing will have been ‘seen’ before. Keeping devices completely unseen is important for determining how the model will behave when it encounters a device that we have not seen before. That scenario needs to be explored because it reflects real-world use in a home network, since it is impossible to train a model using data on all existing devices worldwide. We therefore made sure that some devices remained unseen by always excluding traffic linked to certain devices from our training set.

Our approach correctly categorised 54 out of 74 unseen devices (73%). Furthermore, the model we created is suitable for deployment on low-end hardware. We verified that by running all experiments on a Raspberry Pi 4 with 4 gigabytes of RAM (figure 2). It takes 100 milliseconds for a full prediction while using, at most, 100 MB of RAM. The research ultimately demonstrated that five categories of device can be classified while keeping resource usage low.

Figure 2: The Raspberry Pi used for the experiments.

Future work

The approach described can be followed up in several ways. Most importantly, by developing a system capable of automatically blocking and allowing traffic routes based on the predicted device category. Such a system could be built as a module of the SPIN project developed by SIDN Labs, as SPIN already provides all the necessary base functionality. As well as developing such practical applications, the research could be followed up academically. We currently use five device categories, due to dataset limitations. However, there are far more than five device categories in the real world. Other categories include smart gardening equipment and smart kitchen appliances. To be able to include these categories, we first need to identify the types of device in use and to record the network traffic associated with them.

Further reading

My full thesis is available on the SIDN Labs website. Additionally, I would recommend reading the article on the Toast of the IoT: The 1990 Interop Internet Toaster about, arguably, the first IoT device.