Bluetooth Beacon Proximity Detection in Linux

[ZerOne]

Introduction
The purpose of this thread is to outline and document the processes involved in
writing a Linux C Binary daemon that will listen for Bluetooth Beacons.

This binary will support both Eddystone (Google based) and iBeacon (Apple) based beacons.
Additionally it is hoped that the project will also be able to listen for other personal bluetooth devices
such as fitness trackers to determine what room a person is currently in.

This binary will ultimately be used to fire off events such as turning on / off lights in rooms
when a person has been detected entering or leaving an area (based on the beacon proximity)

[ZerOne]

Project Research

Beacon Transmitter Testing
I have obtained a BlueCoin Beacon in which I was hoping to test with.
https://elainnovation.com/blue-coin-id.html


Unfortunately the beacon seems completely unresponsive to any attempts to program it using
the ELA Device manager software (Tested with Windows, Android and iOS)
My thoughts are that the battery must be depleted and replaced in order for the beacon to function properly.

In place of using the above beacon, I am also using the following Android and iOS Apps for Beacon simulation,
Google Play Store :: Beacon Simulator
https://play.google.com/store/apps/deta ... l=en&gl=US
This app supports simulating both iBeacons and Eddystone beacons

Apple App Store :: eBeacon
https://apps.apple.com/us/app/ebeacon-b ... d730279939
This app supports simulating an iBeacon only

Finally, initial testing of the above Beacon transmissions will be tested using the Linux Command bluetoothctl
(This can be installed on Linux Debian using sudo apt-get install bluetooth)
https://ukbaz.github.io/howto/beacon_scan_cmd_line.html
NOTE: The above will not run using the Linux for Windows Subsystem on Windows 10.

[ZerOne]

Project Research :: Bluetooth / BLE C Code Examples
Official Linux Bluetooth protocol stack
http://www.bluez.org/

Bluez Example Tools
https://github.com/redboltz/bluez/tree/master/tools

Intel Edison C Examples
Intel Edison - Bluetooth BLE Beacon Proximity Scanning (Using Bluez Libs)
https://github.com/damian-kolakowski/in ... ter/scan.c

[ZerOne]

Scanned Devices (MAC Address and Device Name)
C8:DB:75:5E:A3:95 :: FitBit Charge3
DA:21:BC:88:FB:23 :: FitBit Charge4

IMPORTANT NOTES:
FitBit devices appear to broadcast when data syncronisation is required.
These devices will broadcast roughly every second, until data has been syncronised with a paired bluetooth device.

iPhone and Android Phones will generate random macAddresses when bluetooth is turned On, or every 15 minutes or so.
This limits the ability to use the Devices Bluetooth MAC Address to track the device.

[ZerOne]

Installing a Python Example that Displays Presence detection (Bluetooth 4.0 Low energy Beacon)
https://www.domoticz.com/wiki/Presence_ ... gy_Beacon)

The page assumes that you have Python, and the required libraries configured.
Testing on a fresh install of Linux Mint, I needed to install the following to get the Initial Python script working

Code: Select all

sudo apt-get install python pip3 python3-pip
sudo apt-get install python3-dev libbluetooth-dev libcap2-bin
sudo apt-get install -y libusb-dev libdbus-1-dev libglib2.0-dev libudev-dev libical-dev libreadline-dev
sudo aptitude install python-bluez
sudo pip3 install beacontools
sudo pip3 install beacontools[scan]
sudo apt-get install python3-requests
sudo pip3 install requests

Next Select a directory that you wish to store your bluetooth Python Scripts and execute

Code: Select all

wget https://raw.githubusercontent.com/jmleglise/mylittle-domoticz/master/Presence-detection-beacon/test_beacon.py

You may need to give python permissions to use the bluetooth device

Code: Select all

sudo setcap 'cap_net_raw,cap_net_admin+eip' "$(readlink -f "$(which python3)")"

To run the Script

Code: Select all

sudo python test_beacon.py

NOTE: If you get an error ImportError: No module named requests when running the script
then simply comment out line 33

Code: Select all

import requests

by adding a hash to the start of the line.

This will give an output similar to the following

Code: Select all

2021-01-08 13:11:12,767 - root - DEBUG - Ok hci0 interface Up n running !
2021-01-08 13:11:12,768 - root - DEBUG - Connect to bluetooth device 0
2021-01-08 13:11:12,775 - root - DEBUG - New Beacon 5c:30:d0:98:86:ae Detected 
2021-01-08 13:11:13,601 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -63. DATA (-113,)
2021-01-08 13:11:13,703 - root - DEBUG - New Beacon c8:db:75:5e:a3:95 Detected 
2021-01-08 13:11:13,882 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -58. DATA (-113,)
2021-01-08 13:11:14,156 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -58. DATA (-113,)
2021-01-08 13:11:14,697 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -64. DATA (-113,)
2021-01-08 13:11:14,974 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -63. DATA (-113,)
2021-01-08 13:11:15,523 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -68. DATA (-113,)
2021-01-08 13:11:15,798 - root - DEBUG - Tag 5c:30:d0:98:86:ae is back after 0 sec. (Max 0). RSSI -71. DATA (-113,)

[ZerOne]

This is another Python library which also scans bluetooth devices.

Webiste Information
https://www.raspberry-pi-geek.com/Archi ... ck-your-Pi

Website Source code for the Python iBeacon Scanner
https://github.com/switchdoclabs/iBeacon-Scanner-

Command Line Downloads for the above source code

Code: Select all

wget https://raw.githubusercontent.com/switchdoclabs/iBeacon-Scanner-/master/blescan.py
wget https://raw.githubusercontent.com/switchdoclabs/iBeacon-Scanner-/master/testblescan.py

I have found this library provides much more information regarding the scanned bluetooth devices, including
The Bluetooth Device MAC
The Bluetooth Device UUID
Major
Minor
RSSI

Example Output when an iBeacon is simulated on an iPhone using the following settings on the iBeaconBroadcast App on iPhones

Code: Select all

sudo python testblescan.py

63:57:58:28:e5:f4,594650a28621401fb5de6eb3ee398170,94,87,-59,-65

UDID: 59 46 50 a2 86 21 40 1f b5 de 6e b3 ee 39 81 70 None
MAJOR: 00 5e None
MINOR: 00 57 None
MAC address:  63:57:58:28:e5:f4
(Unknown): -59
RSSI: -63

And the iBeacon settings configured within the iBeaconBroadcaster App

PNG image.png

[ZerOne]

A C Example that compiles, but has not picked up ANY bluetooth devices
https://people.csail.mit.edu/albert/blu ... /c404.html

To compile, you will need the following
sudo apt-get install libbluetooth-dev

NOTE: The issue might be that the bluetooth device is not configured for scanning, or the device is locked.
Example -> Attempting to run the command :: sudo hcitool lescan
Results in :: Set scan parameters failed: Input/output error
Running :: bluetoothctl and turning scan on, then off, seems to have freed up the port...

However the c sample still does not return any scanned bluetooth objects.
---> May need to compare code with hcitool (lescan) option to determine what the code differences are in order to get the sample code working.

Some more examples C....
Bluez Lib Scantest
https://github.com/carsonmcdonald/bluez ... scantest.c

One of these GATT libraries that support modern D-BUS API is gattlib.
Here is a simple example based on this library for reading/writing a BLE device:
https://github.com/labapart/gattlib/blo ... ad_write.c

Bluez DBUS API is documented here: https://git.kernel.org/pub/scm/bluetoot ... t/tree/doc

[ZerOne]

The following code was modified from the following example
https://raw.githubusercontent.com/damia ... ter/scan.c

I have modified this code so as it does the following
1) It attempts to open the bluetooth receiver device, and if there are any problems, it attempts to perform a hciconfig hci0 down and a hciconfig hci0 up
2) It will calculate the start and end positions of the UUID in the received buf packet, and will display the UUID as a debug message
3) It will calculate the Device Major and Minor Values from the received buf packet, and will populate the uint16_t variables with these values
4) It will display the RSSI (Signal Strength) and TX Strength using the info_data[] array, using the last and second last int8 values respectively.

Github Repo for the following code..
https://github.com/mijxyphoid/blescanner/tree/main

Code: Select all

//
//  Intel Edison Playground
//  Copyright (c) 2015 Damian Kołakowski. All rights reserved.
//	Modified by Matti Jones 2020 Added UUID, Major and Minor an TX Power Reporting
//

#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <bluetooth/bluetooth.h>
#include <bluetooth/hci.h>
#include <bluetooth/hci_lib.h>

struct hci_request ble_hci_request(uint16_t ocf, int clen, void * status, void * cparam)
{
	struct hci_request rq;
	memset(&rq, 0, sizeof(rq));
	rq.ogf = OGF_LE_CTL;
	rq.ocf = ocf;
	rq.cparam = cparam;
	rq.clen = clen;
	rq.rparam = status;
	rq.rlen = 1;
	return rq;
}

int main()
{
	int ret, status;

	// Get HCI device.

	int device = 0;																	// Declare Variable for our bluetooth receiver device ID
	device = hci_open_dev(hci_get_route(NULL));										// Attempt to open our bluetooth receiver device
	if ( device < 0 ) 																// If there was an error opening our bluetooth device
	{
		char system_command[32];													// Stores our System Command we wish to execute
		int device_id = hci_get_route(NULL);										// Get the HCI Device ID (hci#)
		if (device_id < 0)
		{
			perror("Could not locate a HCI device.");
			return 0; 
		}
		else
		{
			printf("Bringing HCI Device hci%i Down...\n", device_id);				// Debug Message - Bringing Bluetooth receiver down
			sprintf(system_command, "hciconfig hci%i down", device_id);				// Build our system command to shut down our HCI device
			system(system_command);													// Execute our System Command
			sleep (1);																// Sleep for 1 second so as everything stabilises
			printf("Bringing HCI Device hci%i Up...\n", device_id);					// Debug Message - Bringing Bluetooth receiver down
			sprintf(system_command, "hciconfig hci%i up", device_id);				// Build our system command to restart our HCI device
			system(system_command);													// Execute our System Command
			sleep (1);																// Sleep for 1 second so as everything stabilises
			
			device = hci_open_dev(hci_get_route(NULL));								// Attempt to open our bluetooth receiver device again
			if ( device < 0 ) 														// If there was an error opening our bluetooth device
			{
				perror("Failed to open HCI device.");
				return 0; 
			}
		}
	}

	// Set BLE scan parameters.
	
	le_set_scan_parameters_cp scan_params_cp;
	memset(&scan_params_cp, 0, sizeof(scan_params_cp));
	scan_params_cp.type 			= 0x00; 
	scan_params_cp.interval 		= htobs(0x0010);
	scan_params_cp.window 			= htobs(0x0010);
	scan_params_cp.own_bdaddr_type 	= 0x00; 										// Public Device Address (default).
	scan_params_cp.filter 			= 0x00; 										// Accept all.

	struct hci_request scan_params_rq = ble_hci_request(OCF_LE_SET_SCAN_PARAMETERS, LE_SET_SCAN_PARAMETERS_CP_SIZE, &status, &scan_params_cp);
	
	ret = hci_send_req(device, &scan_params_rq, 1000);
	if ( ret < 0 ) 
	{
		
		hci_close_dev(device);
		perror("Failed to set scan parameters data.");
		return 0;
	}

	// Set BLE events report mask.

	le_set_event_mask_cp event_mask_cp;
	memset(&event_mask_cp, 0, sizeof(le_set_event_mask_cp));
	int i = 0;
	for ( i = 0 ; i < 8 ; i++ ) event_mask_cp.mask[i] = 0xFF;

	struct hci_request set_mask_rq = ble_hci_request(OCF_LE_SET_EVENT_MASK, LE_SET_EVENT_MASK_CP_SIZE, &status, &event_mask_cp);
	ret = hci_send_req(device, &set_mask_rq, 1000);
	if ( ret < 0 ) 
	{
		hci_close_dev(device);
		perror("Failed to set event mask.");
		return 0;
	}

	// Enable scanning.

	le_set_scan_enable_cp scan_cp;
	memset(&scan_cp, 0, sizeof(scan_cp));
	scan_cp.enable 		= 0x01;														// Enable flag.
	scan_cp.filter_dup 	= 0x00; 													// Filtering disabled.

	struct hci_request enable_adv_rq = ble_hci_request(OCF_LE_SET_SCAN_ENABLE, LE_SET_SCAN_ENABLE_CP_SIZE, &status, &scan_cp);

	ret = hci_send_req(device, &enable_adv_rq, 1000);
	if ( ret < 0 ) 
	{
		hci_close_dev(device);
		perror("Failed to enable scan.");
		return 0;
	}

	// Get Results.

	struct hci_filter nf;
	hci_filter_clear(&nf);
	hci_filter_set_ptype(HCI_EVENT_PKT, &nf);
	hci_filter_set_event(EVT_LE_META_EVENT, &nf);
	if ( setsockopt(device, SOL_HCI, HCI_FILTER, &nf, sizeof(nf)) < 0 ) 
	{
		hci_close_dev(device);
		perror("Could not set socket options\n");
		return 0;
	}

	printf("Scanning....\n");

	uint8_t buf[HCI_MAX_EVENT_SIZE];
	evt_le_meta_event * meta_event;
	le_advertising_info * info;
	int len;

	while ( 1 ) 
	{
		len = read(device, buf, sizeof(buf));
		
		if ( len >= HCI_EVENT_HDR_SIZE ) 
		{
			uint8_t ptype = buf[0];
			uint8_t event = buf[1];
			uint8_t plen = buf[2];
			uint16_t major = 0;
			uint16_t minor = 0;
			uint8_t counter = 0;
			int8_t string_start = 0;
			int8_t string_end = 0;
		
			meta_event = (evt_le_meta_event*)(buf+HCI_EVENT_HDR_SIZE+1);			// Populate our meta_event struct with data
			if ( meta_event->subevent == EVT_LE_ADVERTISING_REPORT ) 				// If we have an LE Advertising Report Packet
			{
				uint8_t reports_count = meta_event->data[0];						// Grab how many repoorts we need to process in this packet
				void * offset = meta_event->data + 1;
				while ( reports_count-- ) 
				{
					info = (le_advertising_info *)offset;
					char addr[18];
					ba2str(&(info->bdaddr), addr);
	
					// Debug Code - Display the Received Packet
					//printf("Packet: ");
					//while (counter <= plen) 										// Read the received packet in to our buffer
					//{
					//	printf ("\033[43;39m %02x \033[49m ", buf[counter]);		// Print out Hex Digit to the console
					//	counter ++;													// Move to the next byte to process
					//}
					//printf("\n");

					// Messy, need to clean up
					string_end = plen - 3;											// UUID Ends 3 Bytes before the last Byte in our packet
					if (string_end >= 16)											// If there are more than 20 Bytes to read
						string_start = string_end - 16;								// Then set our Start Byte for the UUID to 16 Bytes before the end
					else
						string_start = 4;											// Set our start byte to 4 Bytes in to the Packet
				
					counter = string_start;
				
					printf("UUID: ");
					while (counter < string_end) 									// Read the received packet in to our buffer
					{
						//uuid[(counter - string_start)] = buf[counter];			// Copy the byte to our UUID String

						printf ("\033[44;39m %02x \033[49m ", buf[counter]);		// Print out Hex Digit to the console
						counter ++;													// Move to the next byte to process
					}
					printf("\n");


					// Get the Device Major and Minor Values
					major = (buf[(plen - 3)] << 8);									// Get the MSB of the Major Value (4th Last Byte)
					major += (buf[(plen - 2)] & 255);								// Get the LSB of the Major Value (3rd Last Byte)

					minor = (buf[(plen - 1)] << 8);									// Get the MSB of the Minor Value (2nd Last Byte)
					minor += (buf[(plen)] & 255);									// Get the LSB of the Minor Value (Last Byte)
	
					//printf("%s - RSSI %d\n", addr, (char)info->data[info->length]);
					printf("MAC %s :: Major %i :: Minor %i :: TX Power %d :: RSSI %d\n", addr, major, minor, (int8_t)info->data[(info->length - 1)], (int8_t)info->data[info->length]);
					offset = info->data + info->length + 2;
				}
			}
		}
	}

	// Disable scanning.

	memset(&scan_cp, 0, sizeof(scan_cp));
	scan_cp.enable = 0x00;	// Disable flag.

	struct hci_request disable_adv_rq = ble_hci_request(OCF_LE_SET_SCAN_ENABLE, LE_SET_SCAN_ENABLE_CP_SIZE, &status, &scan_cp);
	ret = hci_send_req(device, &disable_adv_rq, 1000);
	if ( ret < 0 ) 
	{
		hci_close_dev(device);
		perror("Failed to disable scan.");
		return 0;
	}

	hci_close_dev(device);
	
	return 0;
}