IOT Lab (CE-705) - Unit 3 Short Notes

UNIT 3: IoT Communication Protocols & Middleware (Practical Implementation Focus)

3.1 Introduction to IoT Communication Stack & Protocol Categories

Protocol Paradigms:

• Push vs. Pull:

  • Push: Data sent proactively by device/server (e.g., MQTT publish, CoAP notify). Efficient for event-driven updates.

  • Pull: Data requested by client (e.g., HTTP GET, CoAP GET). Client controls timing, simpler but less efficient for frequent updates.

• Pub/Sub (Publish/Subscribe) vs. Req/Res (Request/Response):

  • Pub/Sub: Decoupled. Publishers send to a topic; subscribers receive from topics via a broker. Scales well (e.g., MQTT).

  • Req/Res: Direct, coupled. Client sends request, server sends response. Simple but less scalable (e.g., HTTP, CoAP basic).

Protocol Selection Criteria (Trade-offs):

Role of Middleware/Brokers:

• Acts as an intermediary in Pub/Sub systems (e.g., MQTT broker).

• Decouples producers (publishers) from consumers (subscribers).

• Handles connection management, message routing (topic-based), QoS enforcement, and persistence.

• Enables scalability and fault tolerance.

[!TIP] Exam Focus: Be prepared to compare MQTT vs. CoAP vs. HTTP across the criteria above. Know the core architectural difference (broker vs. direct) between Pub/Sub and Req/Res.

3.2 Message Queue Telemetry Transport (MQTT) - Deep Dive & Lab

MQTT Architecture:

• Client: Any device/app that publishes or subscribes.

• Broker: Central server (e.g., Mosquitto, EMQX) that routes messages.

• Topic: Hierarchical string (sensor/temp/room1) used for message filtering.

• QoS (Quality of Service) Levels:

  | Level | Guarantee | Overhead | Use Case | | :--- | :--- | :--- | :--- | | 0 (At most once) | Fire-and-forget. May be lost. | Minimal | Frequent, non-critical data (e.g., temperature every sec). | | 1 (At least once) | Guaranteed delivery, possible duplicates. | Moderate | Important data where duplicates are acceptable (e.g., meter readings). | | 2 (Exactly once) | Guaranteed delivery, no duplicates. | High (4-step handshake) | Critical commands (e.g., lock door, payment). |

Session Management:

• Clean Session = True: No subscriptions or queued messages retained after disconnect.

• Clean Session = False: Subscriptions and queued QoS>0 messages are persisted for the client.

• Last Will & Testament (LWT): Pre-configured message (topic & payload) sent by broker if client disconnects ungracefully. Used for device failure detection.

Practical Implementation (Python with paho-mqtt):

import paho.mqtt.client as mqtt

# 1. Create Client

client = mqtt.Client(client_id="sensor1", clean_session=False)

# 2. Set LWT & Credentials

client.will_set("status/sensor1", payload="offline", qos=1, retain=True)

client.username_pw_set("user", "pass")

# 3. Connect

client.connect("broker.hivemq.com", 1883, keepalive=60)

# 4. Publish

client.publish("sensor/temp", payload="25.5", qos=1, retain=False)

# 5. Subscribe (Callback)

def on_message(client, userdata, msg):

    print(f"Received `{msg.payload.decode()}` from `{msg.topic}` topic")

client.on_message = on_message

client.subscribe("sensor/#", qos=1)

# 6. Loop

client.loop_forever()

Broker Configuration (Mosquitto):

• Key Config File: /etc/mosquitto/mosquitto.conf

• Enable Authentication: allow_anonymous false + password_file /path/to/passwd

• Enable TLS: listener 8883 + cafile, certfile, keyfile directives.

• ACLs (Access Control Lists): File defining topic read/write permissions per user/pattern.

Security:

• TLS/SSL: Encrypts entire connection. Mandatory for cloud/public brokers.

• Authentication: Username/Password (basic), Client Certificates (stronger).

• Authorization: ACLs on broker control topic-level access.

[!TIP] Common Pitfall: Confusing QoS levels. Remember: QoS 2 is expensive; use only where exactly-once is critical. LWT is set on connect, not publish.

3.3 Constrained Application Protocol (CoAP)

CoAP vs. HTTP (RESTful for Constrained Devices):

Message Format:

• Confirmable (CON): Requires ACK. Retransmitted if no ACK.

• Non-Confirmable (NON): No ACK. Fire-and-forget.

• Piggybacked: Response in ACK to a CON request (most efficient).

• Separate: Response sent in new message (for delayed processing).

Observe Mechanism:

1. Client GETs a resource with Option: Observe=0.

2. Server registers observer and sends current state.

3. When resource changes, server sends a new NON notification to client with updated Observe sequence number.

4. Client can cancel by sending a GET with Observe=1.

Practical Implementation (Python aiocoap):

import asyncio

from aiocoap import *

async def main():

    protocol = await Context.create_client_context()

    request = Message(code=GET, uri='coap://localhost/sensor/temp', observe=0)

    response = await protocol.request(request).response

    print(f"Initial: {response.payload}")

    async for resp in response.observation:

        print(f"Update: {resp.payload}")

asyncio.run(main())

[!TIP] Key Distinction: CoAP is request/response (like HTTP) but built on UDP with added observe (pub/sub-like). MQTT is pure pub/sub. CoAP is often used for device-to-device or device-to-gateway; MQTT for device-to-cloud.

3.4 Lightweight Application Layer Protocols & Alternatives

[!TIP] Rule of Thumb: Use MQTT for most cloud-bound telemetry/command. Use CoAP for very constrained devices or local multicast. Use HTTP only when interacting with existing web services.

3.5 Data Formats & Serialization for IoT

Practical Parsing (Python):

import json, msgpack, cbor2

# JSON

data = json.dumps({"temp": 25.5}).encode('utf-8')

# MessagePack

data = msgpack.packb({"temp": 25.5})

# CBOR

data = cbor2.dumps({"temp": 25.5})

[!TIP] Exam Question: "Why is CBOR preferred over JSON for CoAP payloads?" Answer: CBOR is a binary, concise format specified for constrained environments, drastically reducing payload size and parsing cost compared to verbose text-based JSON.

3.6 Hands-On Integration Labs & Mini-Projects

End-to-End Data Pipeline:

1. Sensor: DHT11 (temp/humidity) → Microcontroller: ESP32 (Arduino framework).

2. Code (ESP32): Read sensor → mqtt.publish("lab/sensor/dht", json_payload).

3. Broker: Local Mosquitto on Raspberry Pi (or cloud like HiveMQ Cloud).

4. Cloud/Server: Python script (paho-mqtt) subscribes, stores in DB (InfluxDB) or forwards.

5. Dashboard: Grafana queries InfluxDB or Node-RED subscribes to MQTT for real-time viz.

Actuation via MQTT:

• Device (ESP32): Subscribes to cmd/led.

• Client (Phone/Web): Publishes {"state": "ON"} to cmd/led.

• ESP32 Callback: Parses JSON, sets GPIO pin.

Simple IoT Gateway (Protocol Translation):

• Input: Serial data from Arduino (e.g., "T:25.5,H:60").

• Gateway (RPi Python): serial.read() → parse → mqtt.publish("serial/sensor1", json).

• Output: MQTT topic. Bridges physical (serial) to IP world.

Stress Testing & Metrics:

• Tool: mqtt-benchmark or custom publisher with varying QoS.

• Metrics to Measure:

  • Latency: Time from publish to subscribe receive.

  • Throughput: Messages/sec broker can handle.

  • Packet Loss: % of messages not received (especially QoS 0).

  • Impact of QoS: QoS 2 will show higher latency, lower throughput than QoS 0.

3.7 Security in IoT Communication (Applied)

Implementing TLS/SSL for MQTT:

1. Generate Certificates: Use openssl for CA, broker, and client certs.

2. Broker Config (Mosquitto):

   
   listener 8883
   
   cafile /path/to/ca.crt
   
   certfile /path/to/broker.crt
   
   keyfile /path/to/broker.key
   
   require_certificate true  # For mutual TLS
   
   

3. Client Connect:

   
   client.tls_set(ca_certs="ca.crt",
   
                  certfile="client.crt",
   
                  keyfile="client.key",
   
                  tls_version=ssl.PROTOCOL_TLSv1_2)
   
   client.connect("broker", 8883, 60)
   
   

Certificate Management on Constrained Devices:

• Challenge: Limited storage for cert chains, limited crypto hardware.

• Solutions:

  • Use certificate fingerprints (hashes) instead of full chains.

  • Use PSK (Pre-Shared Key) mode for DTLS (lighter than certificates).

  • Offload TLS to a gateway; device uses plain MQTT to gateway.

Basic Authentication & Authorization:

• Auth: Username/Password in client.connect().

• AuthZ (ACLs): Mosquitto ACL file pattern:

  
  user alice
  
  topic read sensors/#
  
  topic write cmd/alice/
  
  pattern read sensors/%u/%
  
  

Lab: Securing MQTT:

1. Set up broker with allow_anonymous false and password file.

2. Enable TLS on port 8883.

3. Configure ACLs to restrict user sensor1 to write only to sensor1/data and read only from cmd/sensor1.

4. Test: sensor1 should fail to publish to other/data.

3.8 Cloud IoT Platform Integration (Middleware Perspective)

Connecting to Cloud IoT Core (Generic Pattern):

1. Device Provisioning: Register device in cloud console → get certificate/credentials.

2. SDK/Library: Use vendor's SDK (AWS IoT Device SDK, Azure IoT SDK) or standard MQTT/HTTPS.

3. Connection: Device connects to cloud-specific endpoint using its credentials (X.509 cert or SAS token).

4. Topic/Endpoint Format: Cloud-specific (e.g., AWS: $aws/things/thingName/shadow/update).

Device Shadows / Digital Twins:

• Concept: JSON document ({ "state": { "reported": {...}, "desired": {...} } }) stored in cloud.

• Purpose: Sync device state even when device is offline. App sets desired state; device reports reported state; cloud computes delta and notifies device.

• Usage: device.shadow.update({"state": {"desired": {"led": "ON"}}})

Rules Engine (Cloud Side):

• Trigger: Incoming MQTT message on a specific topic.

• Action: Route/transform message to other cloud services.

  • Example: MQTT message sensors/temp → Rule → write to DynamoDB (AWS) / Blob Storage (Azure) / Pub/Sub (GCP) → trigger Lambda/Function.

Lab: Local Broker ↔ Cloud Bridge:

1. Local: Mosquitto broker on Raspberry Pi.

2. Cloud: AWS IoT Core with a bridge configured.

3. Bridge Config (Mosquitto): Define a connection to AWS IoT endpoint and topic mapping.

   
   connection cloud_bridge
   
   address <AWS-iot-endpoint>:8883
   
   cafile /path/to/AmazonRootCA1.pem
   
   certfile /path/to/pi-cert.pem.crt
   
   keyfile /path/to/pi-private.pem.key
   
   topic sensors/# out 1 "" ""
   
   

4. Result: Messages published locally to sensors/temp automatically appear in AWS IoT Core topic sensors/temp.

[!TIP] Cloud Comparison: AWS IoT Core has most mature Rules Engine & Device Shadow. Azure IoT Hub has strong device management & IoT Edge integration. GCP IoT Core (deprecated for new users) was simplest for Pub/Sub integration. Always check current service status.

\boxed{\text{Core Exam Topics: MQTT QoS & Architecture, CoAP vs. MQTT, Data Format Trade-offs (JSON vs. CBOR), TLS Implementation Steps, Cloud Shadow Concept}}