Developing an IoT LoRa node

This project started off in Y13 Electronics class at school, but now it's become a learning platform to try things. It's scope has continued to expand, along with it's capabilities, as I've learnt new ways of doing cool stuff. This blog will detail said cool stuff...

Inception

We'll skip over most of this bit, as it wasn't that interesting (mostly just doing stuff as required for the report) and jump right into the post-Y13 era...

All that needs to be said is that the hardware was carefully chosen, and shippped to me. It consists of two identical units, both powered by an ESP32 with 4MB of flash storage, an onboard SSD1306 0.96-inch OLED display, Wi-Fi and Bluetooth support, and a Semtech SX1276 LoRa radio module (with external antennae).

Project structure, part 1

Initally, I did everything in the Arduino IDE, as that was (and is) the defacto-standard for all things microcontroller and DIY IoT. Once I had installed ESP32 support via the board manager, I was away to the races.

After trying a few LoRa libraries, I settled on one, and worked my way towards getting a very basic working example of comminication over LoRa. Thankfully,the library I ended up with has quite good documentation, so once I wrangled my understanding of C++ into doing what I wanted, it all worked pretty well.

This might be a good time to explain what LoRa actually is...

What is LoRa?

Long Range communication (LoRa) is really cool, as a technology. Typically, data is transmitted wirelessly by using a 'high' and 'low' frequency, and then transmitting at them to signal a bit. Or, a 'change' and 'no-change' frequency, or a 'long' and a 'short' transmission at the same frequency to indicate a 0 or a 1, etc. LoRa is quirky and wierd and super cool.

LoRa doesn't have set 'high' or 'low' frequencies. Instead, it transmits with a continuously changing frequency (which wraps around a central frequency). You can adjust the data rate and reliability by adjusting how much you change the frequency, called the bandwidth, or by changing how long you take to change the frequency, called the spreading factor.

LoRa has tons of other cool tweaks and features that makes it pretty robust and neat, at least for low data-rate applications, not the least of which is its range and sensitivity. For example, using a custom antennae, attached to the very same board that I've got, a LoRa connection has been demonstrated to reach over 700km! The other neat thing is that it can somehow break physics or whatever and make use of a negative signal-to-noise-ratio (SNR). I still don't understand how, but that's awesome anyway.

Project structure, part 2

Now we get to the good stuff. The project was becoming unwieldy to work on, partly due to choices made earlier on in development, but mostly due to a lack of knowledge. This changed when I learned of the Arduino Pro IDE! It was supposed to be designed for big, multi-file projects in a way that the classic Arduino IDE just... wasn't. Thus, with enthusiasm I downloaded the alpha... and it didn't really work. Which is understandable, it was an alpha after all, and I wanted to do a complex project for a 3rd party board, which is kinda-sorta hacked into the classic IDE anyway. Oh well, I guess I'll just- hang on. The classic IDE can do multi-file projects too!? Yup, that existed the whole time and I just didn't know about it!

So I set off doing some proper research, so that this time around I could build it to actually be extensible. This is the cool stuff now btw.

Side note: I will be demonstrating all this stuff in the Arduino IDE 2, as it is now in beta, and actually works. It's fully backwards compatible with the multi-file projects of yore, and besides it is actually what I'm now using to develop the project.

Multiple .ino files

This was a game-changer for project organisation, despite the few headaches it causes. By default, the Arduino IDEs use a .ino file to store all your code, effectively a .cpp file, and extract data from it to generate the real .cpp and .h C++ files for you.

This is very easily extensible, by simply adding more .ino files to your project directory. The Arduino IDEs will compile any and all suitable files they find. Your 'main' file has to match the project's directory name, and must provide the setup() (?) and loop() (?) functions, but after that it's all up to you!

So how have I actually done it? Let's go over each file together then...
I'll go in-depth with each file (or specifically each cool thing in them) further down.

LoRa_keephome.ino

This is the main file, and the entry point for the project.

LoRa_communication.ino

Unused at the moment, reserved for future use of the communication protocol. Actually, it is now doing stuff! Go check out this post on the details.

LoRa_display.ino

This handles all things display, controlling the SSD1306 display built into the units.

LoRa_functions.ino

This houses all the boring stuff relating to the practical side of LoRa communication between them, such as creating and encoding the data into JSON documents.

LoRa_json.ino

So far this just creates a JSON document, given a String.

LoRa_network.ino

This is pretty fun stuff, as it manages the networking connection side of things, such as creating a Wi-Fi access point or connecting to one, starting up the MDNS service, and handling web clients/requests.

LoRa_ntp_time.ino

This is mostly copy/pasted code, but it works to request and maintain sync with a local network time server, using the Network Time Protocol (NTP). It also gives me a nicely-formatted String of the current time, on demand.

LoRa_storage.ino

Did I mention the ESP32 also has a basic filesystem? Oh yeah, so there's that, and I make use of it to easily store and recall settings/data. It's much more convenient and better than trying to do stuff in the EEPROM (at least, that's what I think, I've never tried).

LoRa_system.ino

This has just got some general system stuff, such as the initModule() function. It also handles some things to do with the serial logging system.

Onboard display system LoRa_display.ino

Here's cool stuff to do with switch and enum.

enum UIs { UI_DEFAULT, UI_DETAILED, UI_EVERYTHING, UI_NOTHING, UI_RESET, LAST };

This little bit of code allows me to do really cool stuff. But first, what is it, why do I need it, and what makes it special?

What is it?

An enum (full name: enumerator) allows you to enumerate over it's elements. What that is and why is totally unimportant, and I don't actually use it like you're supposed to (I think) so we'll just ignore that and talk about how it works instead.

When you declare an enum as above, it allows you to create a variable, with a type of UIs, which can be assigned one of the values declared in the enum. For example, I have a variable declared as: enum UIs current_ui = UI_DEFAULT;, which says that the variable current_ui has a value of UI_DEFAULT.

But, what actually is UI_DEFAULT? That's not a base data type (int, float, bool)! So how does it get stored/processed? Some IDEs shed light on their magic in this case... nah I'll just tell you. It's an int.

The other way of doing this would be to declare a const int UI_DEFAULT = 0;, and then say int current_ui = UI_DEFAULT;, but that's messy and boring, and the compiler can do it for you in a much neater way. You can override this behind-the-scenes behaviour, but this is more useful I find. Basically, the big overview is: you can now assign and use textual names instead of literal data types, directly in your code.

Why do I need it?

So the onboard screen is really small. Like less-than-one-inch small. Like four-lines-of-text small. Not very useful if you want to put a bunch of info on it, and make it a fully functional device. I mean, the whole point of a fully digital screen is that you can completely change it's contents. How do you fix this? More screens! Uh, more virtual screens. As in, paginated information screens. Hopefully you've already figured that out from the names in my enum already. Thus, I have a 'default', 'detailed', 'fullscreen', 'hidden', and 'reset' UI display. But why do I have a 'last' screen too? Hang on to that, we'll see why when we discuss the actual implementation.

What makes it special?

As I've already said, you can use literal text as a data type. So what? How does that even work? Okay then, let's look at how I've used it:

void drawUI () {
	display.setTextAlignment(TEXT_ALIGN_LEFT);
	switch (current_ui) {
		case UI_DEFAULT:
			ui_height = 14;
			display.setColor(BLACK);
			display.fillRect(0, 0, OLED_WIDTH, ui_height);
			display.setColor(WHITE);
			display.drawString(0, 0, "KeepHome");
			display.setTextAlignment(TEXT_ALIGN_RIGHT);
			break;
		case UI_DETAILED:
			ui_height = 24;
			display.setColor(BLACK);
			display.fillRect(0, 0, OLED_WIDTH, ui_height);
			display.setColor(WHITE);
			display.drawString(0, 0, "Detailed UI");
			display.drawString(0, 10, getLocalIP());
			break;
		case UI_EVERYTHING:
			ui_height = OLED_HEIGHT;
			display.setColor(BLACK);
			display.fillRect(0, 0, OLED_WIDTH, ui_height);
			display.setColor(WHITE);
			display.drawString(0, 0, "EVERYTHING");
			display.drawString(0, 10, getLocalIP());
			break;
		case UI_NOTHING:
			ui_height = 1;
			display.setColor(BLACK);
			display.fillRect(0, 0, OLED_WIDTH, OLED_HEIGHT);
			display.setColor(WHITE);
			break;
		case UI_RESET:
			ui_height = 14;
			display.setColor(BLACK);
			display.fillRect(0, 0, OLED_WIDTH, ui_height);
			display.setColor(WHITE);
			display.drawString(0, 0, "Hold PRG to reset");
			break;
	}
	display.setTextAlignment(TEXT_ALIGN_LEFT);
}

I'm still working on this project, so the code above is literally WIP, but right now it's simple enough to use as an example to explain the cool stuff. Anyway, let's get to the explanation

Right at the top, you'll see a fancy switch statement, followed by several case blocks. This is part of what makes enum so cool to use. The initial switch just means that I'm going to be switching based on the value of current_ui. And what are the values of it? Why, as declared above in the enum! Thus, for each possible value, I have code to handle that case. Thus, when current_ui is UI_DEFAULT, that case will be triggered by the switch statement, and the code runs. At the break point, the execution exits the switch.

Overall, this could be acheived exactly the same with a bunch of if statements and a bunch of const int variables, but this is way neater (and easier to read IMHO). Also, it lets me call the whole function, drawUI(), without worrying about which UI view to display.

That just leaves one bit of the puzzle left for this segment: how do you actually switch the view being displayed?

cycleUI() and the PRG button

There's another function, handleButton() that isn't interesting, and all you need to know is that it calls cycleUI() when pressed.

void cycleUI () {
	if (timer_id > 0)
		t.stop(timer_id);
	current_ui = (UIs)((int)current_ui + 1);
	if (current_ui == LAST) {
		current_ui = UI_DEFAULT;
	}
	drawUI();
	display.drawRect(0, 0, OLED_WIDTH, ui_height);
	drawPacket(lastDoc);
	timer_id = t.after(500, drawUICallback, (void*) NULL);
}

There's actually a bit more code here than we need to look at, so let's zooooom in...

current_ui = (UIs)((int)current_ui + 1);
if (current_ui == LAST) {
	current_ui = UI_DEFAULT;
}
drawUI();

The first line looks like madness, so let's break it down a bit. Remember when I said that an enum is actually an int behind-the-scenes? Well we can make use of that. When we press the button, we want to cycle through to the next value. But of course, that's not how enum works. Go loop over an int if you want that. Oh wait- enum is an int! But that's hidden away a bit (they make you work for it).

We'll start right in the middle: (int)current_ui will cast it to an int. This just means, 'forget what type this variable was, it's now an int.' Of course, as it was an int in disguise all along, this works perfectly well for us. Now we can do stuff as you would an do with a regular number, such as adding one to it.

That explains the (int)current_ui + 1 bit, but now we've got a problem. We still want to use it as an enum. Luckily, that's easy enough, because we can cast it back again! Thus, we surround it with brackets, so that we make sure it's just one expression, and cast it back with (UIs). It might help to split the whole process into separate lines, so here's that too:

// cast from 'enum UIs' to 'int'
int view = (int) current_ui;
// increment the view, you could also do 'view++;'
view = view + 1;
// cast back to 'enum UIs'
current_ui = (UIs) view;

Now, what about the next few lines? Why do we have that extra if statement? Oh look, it's LAST again! I said we'll come back to that, so let's talk about it. We only have one button, to cycle forward through the views, so how do we loop back to the beginning again? How about checking if the current_ui is UIs.length or something? Hah I wish. That doesn't exist, because again, enum is special. Casting it to an int doesn't help here either. So instead, I found an ingenious solution (online of course) that is still reasonably elegant. We add another element to the enum, in the end position, right where we want to loop back around. Thus, if we are cycling through, and we end up with that one, before we go and display it (for which we haven't actually defined anything in drawUI(), so it wouldn't do anything anyway), we set current_ui back to the start, which is UI_DEFAULT.

And we're done. This let's us have a really simple, and elegant way to do multiple UI views! Most importantly though, it let's us write neat, simple, and elegant-ish code. For example, let's consider adding a new UI view:

1. Give your view a name in enum UIs {}, such as UI_CUSTOM
2. Define how your view will be displayed by adding case UI_CUSTOM to the switch statement
3. There is no step 3, all the rest of the code works perfectly now.

I hope that explains the cool stuff behind the display, and that you understand it (and what makes it neat). Let's move on to how I've built relatively self-contained modules, and their initialization.

The modules, and initialization LoRa_system.ino

I've done my best to write as modular code as possible this time around, and a big part of that is the module initialization system. It starts out in the setup() function in the main file, as required by Arduino. However, from that point on, it calls the following function several times (once for each module):

void initModule (void (*function)(), String module_name) {
	(*function)();
	printMessage("init", module_name + " initialized");
	drawMessage(module_name + " init");
	vTaskDelay(100);
}

It starts off pretty easy, so it returns void, it's called initModule, and- but then- What. What is (*function)()? Yeah it took me a while too. Let's break it down to what we do know...

function: starts off simple, it's something to do with functions.

*function: it's actually a reference to a function, i.e. memory address, i.e. a pointer

(*function): keeping what we've already got all as one package...

(*function)(): it's a function again? Nah, just doesn't take parameters

void (*function)(): and the function will return void

At this point, hopefully the next line, the first one in the function, makes sense now. We're calling the function that was passed as a parameter (or the memory address to the function). But this time, we're dereferencing the pointer, because C.

The remaining lines in the function, printMessage() and drawMessage() just log what's happened to the screen and the serial port. There's a 100ms delay afterwards, just to make sure the message is properly transmitted (another module is initialized afterwards and may fail, thus causing it to not transmit the message).

Thus, using initModule() is pretty easy, just like this:

void setup () {
Serial.begin(115200);
pinMode(0, INPUT); // PRG button
pinMode(2, OUTPUT); // Blue LED
vTaskDelay(100);
printMessage("system", "Booting...");

initModule(initOLED, "Display");
initModule(initLoRa, "LoRa");
initModule(initSPIFFS, "SPIFFS");
initModule(initNetwork, "Network");
...

Aaaand we've initialized: the display, the LoRa module, the SPIFFS file system, and the network! Do note, that we do not have brackets () after the function name. If we did, the function would be executed, and the result is passed as a parameter, not the actual function itself.

Is that all?

For now, that's all folks! Will there be more in the future? Yeah, probably. At this stage, this post is finished. I've got other things to write about (maybe even this website!), and if you want me to get into the more nit-picky details of how stuff works, rather than just touching on the cool stuff, I'm happy to oblige.

If there's any niggling little things left, please see the home page for contact information.