How Do You Test the Internet of Things (IoT)?

The Internet of Things is just around the corner! It is an emerging technology that promises to make everyday life easier.

Imagine you that when you get home at night and start walking through different rooms in your house, the lights automatically turn on to show you the way. Or, what if the thermostat is automatically set for each room depending on who is in the room at that moment? Great! You’re comfortable and saving energy at the same time.

The world revolves around you. Coupons are automatically sent to your smartphone as you walk by a store you love and your car warns you about a parking spot that’s available a block away. This is time optimization.

Your health will be monitored and you will be informed of pending tests or medicines that you are missing in your bag before going into a trip. Also, in a critical situation, an ambulance can be immediately sent to your current location for a medical procedure. I believe that, as the IoT evolves, sicknesses will be diagnosed in their premature stages increasing the efficiency of medical procedures.

The IoT will have a strong presence in public administration situations as well, as it will allow for benefits such as better control on energy generated versus consumed, an increase on water quality delivered to residences and more efficient traffic management on the streets.

Each of these IoT scenarios will be set up by a few taps on your smartphone or wearable device.

What is the Internet of Things?

The IoT is simply defined as providing intelligence and connectivity to devices we use everyday, allowing them to interact with other devices as well as with us, in order to help us making the world a better place to live.

These “Things” (a combination of hardware and software) are connected to the Internet exchanging information all the time; this makes them behave in a specific way as if they were little robots programmed to perform a task.

How Do We Test the IoT?

Would a bunch of devices talking to each other really accomplish something? With the advances in technology, we have started to deal with situations we have not been used to, such as the ways we interact with these new “Things”. Everyday, new “Things” products are launched and manufacturers are making sure they are IoT-ready. These “Things” are becoming smaller, easier to use and capable of connecting to the Internet with the ability to post data on social networks.

We have to continue learning and prepare ourselves for what will occur in the future. For IT professionals, adapting to these new technologies actually means that we need to be very aware of industry trends, as we’re the ones developing the products and making sure they are properly working before they get into the final user’s hands. Nowadays, devices with serious glitches are not acceptable and competitors are always looking for an opportunity to launch similar products with better quality to increase their market share.

As such, traditional software testing/QA models are evolving along with IoT. The QA team must now focus on usability testing, simulating the environment where the device is to be used. This must be done to ensure that information is exchanged in a secure manner and devices are performing in a manner that fulfills Service Level Agreements.

The requirements to fulfill a QA position are naturally increasing as a result. The candidate must now have not only technical skills but also be willing take part in operations and business to work on testing the Internet of Things, as pointed out by Alex Martins when discussing 3 changes observed in enterprise QA and Testing. The deeper the candidate goes into device life cycle, the more accurate test scenarios he/she will develop. The adoption of test automation for regression test sets will be a key for succeeding in the world of the IoT.

The QA teams must plan tests scenarios that go beyond the use of mouse and keyboard to interact with these “Things“. They should now consider body movement, voice commands, touching and sensor utilization when designing the tests. They should also pay attention to distinct user profiles, different screen sizes and types, geo-location (lat/long/elevation), weather conditions, user mood (happy, sad, angry, etc.) and other considerations.

There are several elements in a software testing/QA model for the IoT, which I will now discuss. The main objective here is to assure that an IoT device operates as intended with no communication or performance hiccups in a secure environment.

Usability Tests

The devices must work as the user expects them to work. The most important point here is the user must recognize the device’s features and start using them right away without encountering any issues.

Within this test category, the QA teams should consider tests that meet functional/business requirements. They must also pay attention to features that may be missing on the device. The faster the user completes a task on the device, the higher will be its adoption by a broader community.

A simple way to run usability tests is to bring a group of people with different profiles (ages/skills) together and ask them to use the device and provide feedback on its usage. Although it may initially look like an approach that brings chaos, it is actually a powerful tool to understand the product’s potential acceptance in the market and to anticipate fixing any issues before its official launch.

Keeping the product vision in mind while running usability tests is a another point to consider. Questions like “What audience was this product designed for?” and “What value is this product bringing to its customers?” will help with the usability test strategy.

For example, if the device is for medical use then its outputs must be very precise and the speed at which it collects and reports data must be really fast. Another example might be a device that handles online data storage. It must allow for the fast upload/download of files with a fault tolerance mechanism in place to keep the files properly secured. In the case of a game, the focus should be on user entertainment and so on.

Connectivity Tests

The devices must properly “talk” to each other. Here, we must pay attention to the different communication methods and information exchanged between devices and users.

During the connectivity tests we have to consider the environments where the devices will be used (type of network, signal strength, weather, etc.) and validate if the device works in such conditions.

For example: If an IoT device is to be used inside a shopping mall, where cellphone reception is usually bad and data from the cellphone is required for communicating with the IoT device, we need to simulate a weak signal environment during connectivity tests. In case of devices with proximity sensors of 30 metre reach, we should use real sensors (or simulate them) to monitor its reach in the hopes of validating communication effectiveness.

In the connectivity test category, the QA team must also pay attention to how the device behaves in case its connection is dropped (no connectivity). There are a few questions that will help when designing test scenarios for it, such as “Will the device retain data up to the point the connection was dropped?” or “Is the user properly informed of the connectivity issue?”.

Regarding the information exchanged between users and devices, IoT products tend to implement APIs (Application Program Interfaces). These interfaces are programs responsible for receiving a message and replying to it with a new message. There are currently several tools available to test APIs. They can simulate a message being sent/received by a device, which helps validate if the information exchanged is accurate.

Performance (time it takes for the device to respond to user actions) is another hot topic. Although it may be related to the infrastructure behind the applications which support the device, performance will be related to connectivity in many cases,  given the nature of the IoT.

Security Tests

We must take care to avoid unauthorized access to devices or information. This test category is often overlooked due to market pressure for companies to quickly launch new products/releases. Another reason why this category is neglected is due to lack of security testing understanding on the part of the IoT manufacturers.

In IoT, the security testing is very critical and it can’t be missed as the device’s behavior affects our everyday lives and the device can actually be remotely configured through a wireless connection. Thus, imagine the mess this can cause if the device is wrongly configured intentionally!

For example, taking control of the entertainment system of a vehicle or cutting off its transmission can be done by an IoT system remotely. As such, security tests need to be done to ensure the system is accessed only by an authorized user and the information exchanged between the car and the system is not intercepted and modified (causing a different behavior by the car) by a hacking attack.

There are 2 main types of security testing:

  • Static Tests: This is done either manually or by code inspection tools. The objective here is to analyze the programming language/code developed for the device and identify if the programmer has followed the best coding practices and if the code doesn’t have any security flaws.
  • Dynamic Tests: These are tools that inspect the device during its normal operation. These tools will look for authentication issues, simulate hacking attacks, point out invalid use of device memory, etc.

 

Analytics Tools and Service Virtualization

The environment where the IoT devices are used can be very complex. From a testing perspective, it is important to have a testing environment similar to the real environment where the device will be used so the tests are accurate. A few tools will help companies with the real environment simulation:

Service Virtualization:

Service virtualization is a great way to simulate the pieces of the device’s real environment that are not available in the testing environment, so that development and testing teams can work on a realistic environment. Good service virtualization tools can actually record transactions from the real device environment and reproduce them on the testing environment, making conditions similar to the real environment. In several cases, building a testing environment that exactly replicates a real device environment is too costly, and this type of testing is therefore removed from the project.

A good example would be a company that has a vendor performing payment processing for its products. If this vendor doesn’t have a testing environment that processes payments for test purposes, the solution would be to use service virtualization tools to create the payment-processing feature on the test environment to allow for the design of more accurate tests.

Analytics:

Analytics are a tool that supports the decision-making process. The process for gathering analytics is collecting data from devices, users, applications, locations and other factors; these are then summarized in dashboards for executives to analyze the data and make decisions.

A simple example of analytics usage in the world of the IoT would be to collect data from places and type of devices used in each place so an optimal configuration of the testing environment by location can be developed. If the type of devices used in these places changes over time, then the testing environment can be reconfigured accordingly to produce more accurate test results.

 

Conclusion

IoT is coming and it is coming fast! Companies are already building strategies for these intelligent ”Things”. IoT is changing the way we do business and the demand for the QA professionals in this area is increasing exponentially. The upcoming months will drive the best usage of IoT, which means that you need to be ready with the most efficient methods of testing these solutions.

Header image courtesy of Flickr user “wilgengebroed”. Available via Creative Commons license 2.0.

This article was originally published on Jose Franca’s LinkedIn. To learn more, please connect with him on LinkedIn.

One Comment on “How Do You Test the Internet of Things (IoT)?”

  1. Appreciate several other insightful web site. The area in addition may I buy of which sort of facts printed in this sort of best method? I own a quest that we are basically at this point running for, and i also have been receiving the appearance away to get such information.. joomla website designing

Leave a Reply

Your email address will not be published. Required fields are marked *