- ✓Evaluating an IoT application requires testing it against the full set of requirements, including functional requirements (does it do what it is supposed to do?), performance requirements (does it do it reliably and within acceptable time limits?) and security requirements (is it resistant to the most relevant attack vectors?).
- ✓Integration testing is particularly important in IoT systems, where the application must interoperate correctly with sensors, actuators, gateways, cloud platforms and potentially other enterprise systems.
- ✓Interoperability challenges arise when IoT devices from different manufacturers need to work together, as there is no single universal standard and different devices may use incompatible protocols, data formats or security mechanisms.
- ✓Power consumption testing is a critical but often overlooked aspect of battery-powered IoT device evaluation: a device that performs well but drains its battery in a few days rather than the expected months or years will fail in deployment.
- ✓Post-deployment monitoring of IoT systems is essential because real-world performance often differs from laboratory testing in ways that are difficult to predict: robust logging, alerting and remote management capabilities should be built in from the start.
Listen to the full episode inside the course. Enrol to access all 80 episodes, plus assignments, tutor support and Student Finance funding.
Start learning →Alex: Hello and welcome back. Today we're looking at how you evaluate an IoT application once it's been built. Sam, evaluation in IoT seems particularly important because of the real-world consequences of failures.
Sam: The stakes are real. A bug in a desktop application causes inconvenience. A bug in an IoT system controlling industrial equipment, medical devices or safety-critical infrastructure can cause physical harm. The rigour of evaluation needs to match the potential consequences of failure.
Alex: What are the main dimensions of evaluation for an IoT system?
Sam: I'd structure it around four dimensions. Functional evaluation: does the system do what it was specified to do? Does it correctly detect the events it's supposed to detect, trigger the right responses, provide accurate information? Performance evaluation: does it do these things within the required timeframes and with adequate reliability? Security evaluation: is it adequately protected against the relevant threats? And integration evaluation: does it work correctly with the other systems it needs to interface with?
Alex: Let's dig into performance. What does that mean in an IoT context?
Sam: Latency is a key performance dimension: the time from an event occurring in the physical world to the system responding to it. For applications where fast response matters, like safety systems or real-time control, latency must be measured under realistic conditions including network variability. Reliability is another: what percentage of sensor readings are successfully transmitted and processed? In industrial environments, even a small percentage of lost readings can be significant. Scalability matters too: how does performance change as the number of devices increases? And for battery-powered devices, power consumption is a performance metric: the system must achieve its functionality within the power budget of the battery or energy harvesting system.
Alex: How do you approach security evaluation?
Sam: You work systematically through the attack surface. Can an attacker intercept data transmitted between devices and the backend? Are devices authenticated properly when they connect? Can the firmware on a device be modified without authorisation? Can an attacker spoof a device and inject false data into the system? Are default credentials changed? Is the backend API protected against common web vulnerabilities? For higher-risk applications, a formal penetration test conducted by specialists is worth the investment.
Alex: And integration challenges. What are the most common ones?
Sam: Different systems using different data formats is a perennial challenge: devices from different manufacturers may use proprietary formats that need to be translated. Different time zones and timestamp formats can cause data to be incorrectly ordered or correlated. Version mismatches between device firmware and backend APIs can cause silent failures where data is lost or misinterpreted rather than producing explicit errors. And enterprise system integration, connecting IoT data to ERP or CRM systems, often involves complex data mapping and workflow design.
Alex: Really thorough evaluation framework. Thanks, Sam. We'll close out Unit 10 with a look at IoT in industry next.