Mike Anderson here, an Engineer at Qualcomm Atheros, Inc. We recently posted a new open source project to Github that I’m pretty excited about, called boardfarm. It was created two years ago, and has been used daily to automatically flash new OpenWrt-based software and package files onto hardware, run basic sanity tests, and save results. It’s been a great help by catching kernel panics, memory leaks, networking bugs, and more.
When boardfarm is used along with continuous integration software, like the open source Jenkins, it becomes even more powerful because, for example, it can be triggered to run automatically every night to test new code. This allows issues to be caught immediately.
The name boardfarm originally came about because we setup several routers on one table to create a “farm” of devices. A “boardfarm” is really just a stable test bench setup. There are plenty of closed-source networking test benches that easily cost tens of thousands of dollars. Those devices usually use Virtual Machines connect to the LAN & WAN of routers and run networking tests. We’ve created a powerful alternative to that test equipment based on open-source solutions.
Using console/ssh servers and network-controlled electrical outlets, we made all these devices available remotely, over the local network. That allowed developers and testers in totally different cities to connect to the same routers and devices to test and debug issues.
A boardfarm for properly testing networking on a single OpenWrt router might look like this:
The networking connections in the above picture look like this:
Local Network -- eth0 Linux Computer eth1 -- LAN-Router-WAN -- eth1 Linux Computer eth0 -- Local Network
Basically, the Linux Computers (Raspberry PIs in the picture) connected to the router allow for networking tests like ping, curl, wget, iperf, and etc. For example, they can be used to run network traffic for hours (or days) to check the stability of the OpenWrt router.
All tests can be found in the “tests” directory of the project, and are written Python to be simple and readable, e.g:
def runTest(self):
board.sendline('ping -c2 www.google.com')
board.expect('2 packets received', timeout=10)
board.expect(prompt)
They simply type commands on devices, and expect certain output. You might recognize “sendline” and “expect” – this is just the Pexpect library for typing commands and expecting certain output.
Anyway, many more examples and details can be found at the boardfarm project site. We hope you’ll have a chance to setup a farm, try out the OpenWrt tests, and join the fun as we continue to expand test suites and improve open source networking!