This folder contains several Sensor specific projects for x86/PC and ESP32-C3 platforms. Here is an overview about the contained sub folders:
- main-rust
A Sensor application for x86/PC written in Rust - main-rust-esp-rs
A Sensor application for ESP32-C3 written in Rust using esp-rs/esp-idf-sys and the Espressif IDF SDK - sensor-lib
A Rust library containing shared sources for all Sensor applications and the streams-poc-lib - streams-poc-lib
A static library providing C bindings for all functions needed in the SUSEE-Module and a test application written in C to test the library - main-streams-poc-lib-pio
A PlatformIO project to build the streams-poc-lib test application. This project demonstrates how to integrate the streams-poc-lib files in a PlatformIO project.
There are three different Sensor applications for different purposes:
| Application | Platform | Purpose | Progr. Language |
|---|---|---|---|
| streams-poc-lib test application | ESP32 | Test the streams-poc-lib functionality using its C binding | C |
| ESP32 Sensor DEPRECATED | ESP32 | Test the Rust code that builds the foundation of the streams-poc-lib without the limitations of a foreign function interface | Rust |
| x86/PC Sensor | x86/PC | Test the Sensor Rust code on x86/PC platforms and mock ESP32 Sensors for integration tests on x86/PCs | Rust |
In addition to the common CLI options described in the CLI API section of the main README file all Sensor applications provide CLI commands to manage the Streams usage:
-s, --subscribe-announcement-link <SUBSCRIBE_ANNOUNCEMENT_LINK>
Subscribe to the channel via the specified announcement link.
-r, --register-keyload-msg <KEYLOAD_MSG_LINK>
Register the specified keyload message so that it can be used
as root of the branch used to send messages later on.
-f, --file-to-send [<FILE_TO_SEND>...]
A message file that will be encrypted and send using the streams channel.
The message will be resend every 10 Seconds in an endless loop.
Use CTRL-C to stop processing.
-g, --random-msg-of-size <MSG_SIZE>
A random message of the specified length in bytes will be created,
encrypted and send using the streams channel.
A new random message will be resend every 5 Seconds in an endless loop.
Use CTRL-C to stop processing.
-p, --println-subscriber-status
Print information about the current client status of the sensor.
In streams the sensor is a subscriber so that this client status is called subscriber
status.
--clear-client-state
Deletes the current client status of the sensor so that
all subscriptions get lost and the sensor can be used to subscribe to a new Streams
channel.
TODO: -----------------------------
-------- WARNING ---------- Currently there is no confirmation cli dialog
----------------------------- use this option carefully!
The x86/PC Sensor directly accesses the IOTA Bridge. Use the following arguments to manage IOTA Bridge usage:
-b, --iota-bridge-url <IOTA_BRIDGE_URL>
The url of the iota-bridge to connect to.
Default value is http://localhost:50000
Example: iota-bridge-url="http://192.168.47.11:50000"
-o, --failover-iota-bridge-url <FAILOVER_IOTA_BRIDGE_URL>
Specifies a secondary iota-bridge used for failover.
In case the primary iota-bridge (specified by the CLI argument '--iota-bridge-url')
returns an erroneous http response, the sensor will try to use a secondary
iota-bridge instance specified by this argument.
The implemented failover-handling is very simple: The secondary iota-bridge is only
called in case of errors and only once (per error).
If this argument is not provided, no failover is done.
The error response behavior of the iota-bridge is designed to allow failover-handling
run on an application-server-connector. Therefore this argument must be used together
with the --use-lorawan-rest-api argument. In theory, it would be possible to use
--failover-iota-bridge-url while accessing iota-bridge API functions directly,
but this would break the communication between the iota-bridge and the Sensor.
Example: --failover-iota-bridge-url="http://192.168.47.11:50000"
As all Sensor applications running on ESP32 do not provide an interactive terminal, the x86/PC Sensor can be used to remote control the ESP32 applications. The x86/PC Sensor provides following CLI commands to manage the remote control functionality:
-c, --act-as-remote-control
Use this argument to remotely control a running sensor application on
an embedded device. For example this
> ./sensor --subscribe-announcement-link "c67551dade.....6daff2"\
--act-as-remote-control
will make the remote sensor subscribe the channel via the specified
announcement-link. This sensor app instance communicates with the remote sensor
app via the iota-bridge application. Please make sure that both sensor
app instances have a working connection to the running iota-bridge.
If sensor and iota-bridge run on the same machine they can communicate over the
loopback IP address (localhost). This is not possible in case the sensor runs on an
external device (embedded MCU). In this case the iota-bridge needs to listen to
the ip address of the network interface (the ip address of the device that runs
the iota-bridge) so that the embedded sensor can access the iota-bridge.
Therefore in case you are using 'act-as-remote-control' you will also need to use
the 'iota-bridge' option to connect to the iota-bridge.
The streams-poc-lib test application can only bee remote controlled if the Streams Channel has not already been initialized (further details can be found in the streams-poc-lib README).
The x86/PC Sensor application can also be used to act as a remote controlled Sensor or let's say to mock (or imitate) the test application for the Streams POC Library in the uninitialized mode. This is especially useful to test the IOTA Bridge and the Management Console without the need to run ESP32 Hardware. The CLI command to mock the Streams POC Library test application is:
-m, --act-as-remote-controlled-sensor
Imitate a remote sensor resp. an ESP32-Sensor awaiting remote control commands.
ESP32-Sensor here means the 'sensor/main-rust-esp-rs' application or the
test app of the streams-poc-lib in an initial Streams channel state.
This command is used to test the iota-bridge and the management-console application
in case there are not enough ESP32 devices available. The sensor application will
periodically fetch and process commands from the iota-bridge.
If the iota-bridge runs on the same machine as this application, they can
communicate over the loopback IP address (localhost). In case the sensor
iota-bridge listens to the ip address of the network interface (the ip
address of the device that runs the iota-bridge) e.g. because some ESP32
sensors are also used, you need to use the CLI argument '--iota-bridge-url'
to specify this ip address.
The --act-as-remote-controlled-sensor argument is especially useful to automatically initialize the x86/PC Sensor
in interaction with the
Management Console --init-sensor argument.
If you want to exit the Sensor application after the initialization has been finished you can use
the --exit-after-successful-initialization argument:
-e, --exit-after-successful-initialization
If specified in combination with --act-as-remote-controlled-sensor the command poll loop
will be stopped after a KEYLOAD_REGISTRATION confirmation has been send to confirm
a successfully processed REGISTER_KEYLOAD_MESSAGE command.
This argument is useful when the sensor app runs in automation scripts to allow the
initialization of the Sensor and the Sensor app should exit after successful
initialization.
The x86/PC Sensor application can also be used to test the lorawan-rest/binary_request endpoint of the IOTA Bridge
application. This is done using the --use-lorawan-rest-api argument:
-l, --use-lorawan-rest-api
If used the Sensor application will not call iota-bridge API functions directly
but will use its lorawan-rest API instead.
This way the Sensor application imitates the behavior of an ESP32-Sensor connected
via LoRaWAN and a package transceiver connected to the LoRaWAN application server
that hands over binary packages to the iota-bridge.
To test the Sensor Reinitialization workflow
the mocked DevEUI of the x86/PC Sensor needs to be reused.
This can be achieved using the --dev-eui argument:
-d, --dev-eui <DEV_EUI>
Use the specified LoRaWAN DevEui instead of a random value.
In case the sensor wallet file (wallet-sensor.txt) of the sensor has been deleted,
the default behavior is to use a random value as new DevEui.
The generated DevEui then is stored in the sensor wallet file later on so that the
DevEui is persisted for later use.
Using this argument the DevEui can be pre defined to have a static DevEui for test
purposes. This argument is ignored in case the DevEui has already been stored in the
sensor wallet file.
Example: --dev-eu=12345678
To reduce the LoRaWAN payload size, compressed streams messages can be used to communicate between a Sensor and the IOTA Bridge. Compressed messages do not contain Streams Channel IDs and other data that can be restored by the IOTA Bridge.
IMPORTANT NOTE: To restore the omitted data of a compressed message, the IOTA Bridge can process on the encrypted message because the omitted parts of the message are plain text metadata. This means that using compressed messages has no impact on data privacy. It would not be even possible to decrypt messages in the IOTA Bridge, because in the SUSEE project the encryption key never leaves the Sensor for security reasons.
The usage of compressed messages is only possible after one or more normal streams messages have been send using the IOTA Bridge. The IOTA Bridge then learns which Streams Channel ID is used by which Sensor where the Sensor is identified by its 64 bit LoraWAN DevEUI. Using LoraWAN, the DevEUI is available via the protocol automatically and does not need to be transferred as message payload.
The mapping of LoraWAN DevEUI to Streams Channel ID is stored in a local SQLite3 database managed by the IOTA Bridge.
In case the IOTA Bridge added a Sensor to its mapping database, the response of the
REST call that caused the new Sensor database entry will have a 208 - ALREADY_REPORTED http status.
All Sensor applications recognise this http response status and will only use compressed
messages further on. The Sensor applications store the state whether to use compressed
messages or not in their local client-state serialization files.
The process described above is called Sensor to Bridge Pairing.
The following sequence diagram shows the Sensor to Bridge Pairing in more detail:
In case a Sensor uses compressed messages and is connected to an IOTA Bridge that does not know its DevEUI the IOTA Bridge can not transmit the Sensors request to the IOTA Tangle. This can happen e.g. if different IOTA Bridge instances are used for the Sensor Initialization and for Sensor Processing.
In this situation Sensor to Bridge RE-Pairing allows to transmit the channel-id later on which results in a minimum of additional LoRaWAN payload. After the IOTA-Bridge receives the missing channel-id, the original compressed request is processed and the resulting response is returned to the sensor.
When the IOTA Bridge detects that a compressed request can not be processed, the relevant data of the request are stored in its local SQLite3 database.
The Sensor receives a response with status 422 Unprocessable Content together with a request_key
(response body) that can be used to address the unprocessed request later on.
Using the message/retransmit endpoint of the IOTA Bridge the Sensor can provide the request_key
and Streams Channel ID to the IOTA Bridge. The bridge then fetches the original request from the
database and processes the request.
The message/retransmit request of the Sensor is answered by the IOTA Bridge with the response
that results from the original request that has been addressed using the request_key.
The status of this response is 208 Already Reported to make shure that the Sensor uses
compressed requests further on.
The following sequence diagram shows the Sensor to Bridge RE-Pairing in more detail:
During the Sensor Reinitialization workflow a Sensor subscribes to a new IOTA Streams Channel while the Sensor DevEUI remains unchanged.
The Initialization Count property of the SUSEE http protocol allows IOTA Bridges to detect reinitialized
sensors and to update the cached IOTA Streams Channel-id to the new channel-id via the
messages/transmit endpoint.
The Initialization Count is stored in the Sensor wallet file and is incremented when the sensor subscribes to a new Streams Channel.
The IOTA Bridge stores the Initialization Count together with the channel-id in its local SQLite3 database to detect reinitialized sensors.
The Initialization Count is loged to the console by the Sensor applications and by the Management Console. Sensors can only be reinitialized 255 times. If this limit is reached a warning is loged by the Sensor and by the Management Console.
A design goal of the SUSEE application protocol has been, to only include the Sensor and the IOTA Bridge in the 'Sensor to Bridge Pairing' and the 'Sensor to Bridge RE-Pairing' process and to avoid any needed third party interaction e.g. a central management logic. Until the maximum number of Initialization Counts is reached no central management logic is needed.
In case the maximum number of Initialization Counts is reached there are several possible solutions to proceed:
- Allow a counter reset to zero. This includes the risk that IOTA Bridges with a very outdated cache will not detect a reinitialized Sensor which will lead to erroneous communication until another IOTA Bridge is used that detects the new Streams Channel-id.
- Use the
lorawan-nodeendpoint of all IOTA Bridge instances to delete the specific Sensor from the cache. This is the safest option, but a central management logic is needed to send the IOTA Bridgelorawan-noderequests.
The Sensor applications provided in this repository do not interact with LoRaWAN themselves. Therefore, the DevEUIs are mocked for test purposes. The following list explains how each Sensor application mocks the LoRaWAN DevEUI and how the DevEUI is transfered to the IOTA Bridge:
- Streams POC Library Test Application
The LoraWAN DevEUI is mocked using the base MAC address of the ESP32 MCU (EUI-48 - formerly known as MAC-48). Espressif provides a universally administered EUI-48 address (UAA) for each network interface controller (NIC) e.g. WIFI, BT, ethernet, and so on.
To be independent from the used NIC we mock the LoraWAN DevEUI using the base MAC address that is used to generate all other NIC specific MAC addresses.
To make sure the mocked LoraWAN DevEUI is received by the AppServer Connector Mockup Tool the DevEUI is prepended to the request data that are send via the socket connection. The AppServer Connector Mockup Tool application later reads the mocked DevEUI from the socket stream and uses it to access the iota-bridge/lorawan-restendpoints. - x86/PC Sensor
The LoRaWAN DevEUI is mocked using a persistent random value. The random value is stored in the wallet file together with the Streams Channel plain text seed. The x86/PC Sensor application does not use the AppServer Connector Mockup Tool to access the/lorawan-restendpoints of the IOTA Bridge. Instead, it uses the/lorawan-restendpoints directly in case the--use-lorawan-rest-apiCLI argument is used.