Quality assurance
At SEGGER, we go the extra mile in testing the emUSB-Host stack.
Why is Quality Assurance extremely important?
USB is a worldwide standard used in every PC and in most peripherals.
It has been created in the mid-1990s and is developed by the USB Implementers Forum (USB IF).
Despite the fact that the standard clearly defines the behavior of both, device and host, as well as error correction codes (ECC) used, host and device communication does not always "simply work".
Though USB is a well defined standard, unfortunately many devices do not follow it to the letter. It seems, that many USB device manufacturers do not care about and do not test with anything but Windows hosts. Therefore issues which occur when using the devices with USB full-speed, low transfer sizes, different timings than Windows are rarely recognized by the manufacturers. Being compatible with Windows does not necessarily mean being compatible to the USB standard.
This creates a challenge for every USB host stack. It needs to be compliant with the USB standard, but at the same time work well even with devices that are not fully compliant. Limited resources in embedded systems make things even more challenging. The only way to make sure an embedded host stack fulfills all requirements is a clean design from ground up as well as intensive testing with different devices.
This makes intensive testing a must for any professional USB host stack.
Read on to find out about our procedures for testing the emUSB-Host stack.
Tests with USB memory sticks from different manufacturers
USB memory sticks are cheap and easy to use storage mediums. It seems most sticks on the market do not pass rigorous testing. From our experience, most manufacturers test their Mass Storage Devices with Windows hosts only. Other operating systems or hosts which do not support USB high-speed seem to be of no concern. Over the years we have come into contact with sticks where the NAND flash fails after relatively light usage, sticks which refuse to operate under USB full-speed, sticks which crash when single sectors are being read from them.
In our software we strive to accommodate and work around these issues. To achieve this, we maintain a large collection of USB sticks and other Mass Storage Devices (card readers, hard disk drives). We regularly add new devices to our collection, which we use to test emUSB-Host.
In the test scenario shown, nineteen USB sticks and four hubs are connected to a host microcontroller. The test ensures that individual hubs and USB sticks are enumerated without issues and that read and write operations succeed.
Tests with different device classes
emUSB-Host supports multiple device classes, to test each class module we use multiple devices from different manufacturers to make sure our host stack can handle any device on the market.
Over 2000 evaluation boards
emUSB-Host is designed to run on a multitude of different MCUs. Switching to a new MCU for a new project, or even a different architecture does not require modifications to the user application code.
SEGGER has a large collection of evaluation boards. emUSB-Host is tested on many of them.
Rigorous enumeration tests
Enumeration of USB devices is a complicated process. To test emUSB-Host with multiple thousands of enumerations and without having to hire a person exclusively responsible for connecting and disconnecting USB cables we have designed a special adapter for our J-Link which allows us to simulate this process. The adapter is able to electrically connect or disconnect the plugged in USB cables. This allows us to intensively test enumeration and device removal, the configurable timing allows us to guard against timing issues in the emUSB-Host stack.
Hub testing
Similar to many memory sticks many hubs are very unreliable. Some crash, some do not work in full-speed mode, some hubs while being one physical device actually have two hubs inside them.
To make sure our software works with any hub we acquired a multitude of USB hubs which we use for our tests.
Automatic emUSB-Host regression tests
For automatic regression testing we have designed a VBUS switch board, similarly to the small J-Link adapter it is able to simulate physical connection and disconnection, but instead of one cable pair it is able to do this for five. The VBUS switch is controlled via an emControl I/O board which supplies the board with power and is able to control which ports are connected. There are multiple devices connected to the VBUS switch. On channel zero there is a USB stick. Channel one is connected to an emPower board programmed as a composite device with two CDC interfaces. On channel two there is another emPower board programmed as an HID device. On channel three a mouse is connected. On the last channel we have the hub which can also be turned on and off to test connection or disconnection of all devices at once.
The automatic regression test software is controlled from a PC. It decides which devices are enumerated and which tests to run. Among enumeration tests read and write, asynchronous, operations with timeouts are tested with any packet sizes. Simultaneous operations on different devices are tested, canceling of running operations, operations under high CPU load, re-initialization of the stack, error handling and device removal during running operations is also tested.
The automatic test setup is steadily extended by adding new test scenarios for problematic corner cases and strange device quirks.
Below you can see a sample output of the automatic test software.
