Automated BLE Testing

When developing low-power BLE devices, testing is very often done manually. The setup often includes a desktop or smart phone running an app such as Light Blue, and an operator tasked with stepping through a list of tests. Because this is time consuming, thorough testing often happens infrequently, often right before a big merge to {% c-line %}main{% c-line-end %}.

However, with a little knowledge of GitHub Actions and Lager, teams can run exhaustive tests every time code is pushed or on a scheduled job. This allows teams to catch bugs immediately, and because tests are continuously being run on actual hardware, identifying the commit that introduced the bug is easy.

To demonstrate we'll use the Heart Rate Sensor example from the nRF52 SDK. First we'll put together a simple GitHub Actions workflow that will build our project for us every time we push to GitHub.

{% c-block language="yaml" %}
#GH Action Job for building project using lager devenv commands
#and the lagerdata/devenv-cortexm-nrf52 docker image
build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout the code
        uses: actions/checkout@v2
        with:
          submodules: 'recursive'
      - name: Build Project
       uses: docker://lagerdata/devenv-cortexm-nrf52
       with:
         entrypoint: lager
         args: exec build
      - name: Upload the app hexfile for use in later Jobs
       uses: actions/upload-artifact@v2
       with:
         name: app_hexfile
         path: examples/ble_peripheral/ble_app_hrs/pca10056/s140/armgcc/_build/nrf52840_xxaa.hex
{% c-block-end %}

Next we'll add the appropriate Lager commands for connecting to and then flashing our board, in this case the nrf52840-DK development board.

{% c-block language="yaml" %}
#Jobs for connecting and then flashing
connect:
   runs-on: ubuntu-latest
   steps:
     - name: Connect to gateway
       uses: docker://lagerdata/lager-cli
       env:
         LAGER_TOKEN_ID: ${{ secrets.LAGER_TOKEN_ID }}
         LAGER_TOKEN_SECRET: ${{ secrets.LAGER_TOKEN_SECRET }}     #This is setup inside GitHub see <https://docs.lagerdata.com/ci/github_actions.html>
       with:
         entrypoint: lager
         args: connect --device nrf52 --force
flash: #Only proceed to this step if the build and connect jobs succeeded
   runs-on: ubuntu-latest
   needs: [build, connect]
   steps:
     - name: Download the app hexfile
       uses: actions/download-artifact@v2
       with:
         name: app_hexfile
     - name: Flash Device
       uses: docker://lagerdata/lager-cli
       env:
         LAGER_TOKEN_ID: ${{ secrets.LAGER_TOKEN_ID }}
         LAGER_TOKEN_SECRET: ${{ secrets.LAGER_TOKEN_SECRET }}
       with:
         entrypoint: lager
         args: flash --hexfile /github/workspace/nrf52840_xxaa.hex
{% c-block-end %}

Ok, we now have a very simple workflow that will build our project and flash our board every time we push to main.

But the real exciting part is when we start running actual hardware-in-the-loop tests on every push. To do this we're going to use Lager's python BLE library to run through the following tests:

1. Scan for device and verify advertising data
2. Connect
3. Run service discovery and verify Services, Characteristics, and Properties
4. Read from Characteristic
5. Enable Notifications and verify data
   

Although this sounds like a lot, we can accomplish this in just a handful of lines using Lager's python BLE library. We'll then call the test script we created from GH Actions using Lager.

First let's start by scanning devices in the area, looking for our device:

{% c-block language="python" %}
device = central.scan(name='Nordic_HRM')
if not device:
   raise SystemExit("Error Device Not Found")
{% c-block-end %}

Next we'll see if we can connect to our device and do a service discovery:

{% c-block language="python" %}
#Test Connection
with central.connect(device[0]) as client:
   services = client.get_services()
   if not services:
       raise SystemExit("No Services Found")
{% c-block-end %}

So far so good. Now we're going to verify our service, in this case the Heart Rate Monitor, by checking that it has the appropriate Characteristics, and that they have the correct properties.

{% c-block language="python" %}
#Verify HRM Service Characteristics
HRM_SERVICE = "0000180d-0000-1000-8000-00805f9b34fb"
HRM_BODY_SENSOR_LOCATION_CHAR =  "00002a38-0000-1000-8000-00805f9b34fb"
HRM_MEASUREMENT_CHAR =  "00002a37-0000-1000-8000-00805f9b34fb"
hrm_service = services.get_service(HRM_SERVICE)
if not hrm_service:
   raise SystemExit(f"HRM Service Not Found")
if not any(char.uuid == HRM_MEASUREMENT_CHAR for char in hrm_service.characteristics):
   raise SystemExit("Heart Rate Measurement Not Found")
if not any(char.uuid == HRM_BODY_SENSOR_LOCATION_CHAR for char in hrm_service.characteristics):
   raise SystemExit("Heart Rate Body Sensor Location Not Found")

for char in hrm_service.characteristics:
   if char.uuid == HRM_MEASUREMENT_CHAR:
       assert char.properties[0] == 'notify'
   if char.uuid == HRM_BODY_SENSOR_LOCATION_CHAR:
       assert char.properties[0] == 'read'
{% c-block-end %}

And finally, we'll read the Body Sensor Location Characteristic, enable notifications on the Measurement Characteristic, and do some light data verification on each.

{% c-block language="python" %}
#Verify Data on HRM Characteristics
val = int.from_bytes(client.read_gatt_char(HRM_BODY_SENSOR_LOCATION_CHAR),"little")
assert BLE_HRS_BODY_SENSOR_LOCATION_OTHER < val <= (BLE_HRS_BODY_SENSOR_LOCATION_FOOT)

try:
   timed_out, messages = client.start_notify(HRM_MEASUREMENT_CHAR, hrm_notify_cb, max_messages=5, timeout=10)
   if timed_out:
       raise SystemExit("Heart Rate Notifications Failed")
print(messages)
finally:
   client.stop_notify(HRM_MEASUREMENT_CHAR)
print("Heart Rate Measurement Service Verified")
{% c-block-end %}

Now, let's throw this into a python script called {% c-line %}main.py{% c-line-end %} and call it from our GitHub Actions file:

{% c-block language="yaml" %}
test_ble: #test basic BLE functionality
   runs-on: ubuntu-latest
   needs: [flash]
   steps:
     - name: Download ble test
       uses: actions/download-artifact@v2
       with:
         name: test_ble

     - name: Test BLE
       uses: docker://lagerdata/lager-cli
       env:
         LAGER_TOKEN_ID: ${{ secrets.LAGER_TOKEN_ID }}
         LAGER_TOKEN_SECRET: ${{ secrets.LAGER_TOKEN_SECRET }}
       with:
         entrypoint: lager
         args: python main.py
{% c-block-end %}

Without too much effort, we now have a pretty powerful script for continuously testing our firmware for basic BLE functionality. And as more requirements are added, it's straightforward to update the python script to test those as well.

‍

Checkout the full python test script for this project here:
https://github.com/lagerdata/demo-nrf52-hrs/blob/master/tests/main.py

‍

And the GitHub Action workflow file here:
https://github.com/lagerdata/demo-nrf52-hrs/blob/master/.github/workflows/build_and_test.yml

‍

Lager's BLE python library:
https://docs.lagerdata.com/gateway-lager/bluetooth_quickstart.html

‍

Interested in setting up automated BLE testing for your project? Request a live demo with us and we can show you how!