UnPlugFest 53

[Christopher Collins <ch...@runtime.io>]

Hello all,

Last week I was in Atlanta, Georgia attending UnPlugFest 53.  UnPlugFest
is an event held three times a year at which participants get an
opportunity to test their bluetooth products.  My purpose for being
there was to test the Mynewt BLE stack (called "nimble").  Each day of
the event is partitioned into eight one-hour blocks.  For each block, an
automated system pairs up participants whose implementations support the
same bluetooth capabilities.  Since nimble is a full BLE stack (both
host and controller), I got the opportunity to test against a wide
variety of implementations.  In all, I was scheduled for 25 hour-long
test sessions against 19 other implementations.  I wanted to use
this email to briefly summarize what came out of this event.

Overall, I would say the event went very well.  Nearly everything we
tested worked as expected.  The biggest issue we encountered was our
lack of support for some functionality that other implementation took
for granted.  We knew about this going into the event, so while it was a
nuissance, it was not a surprise.

At the start of the event, nimble did not support any of the following
features:

    * L2CAP signalling.
    * L2CAP reassembly.
    * Connection parameters request link-layer procedure.
    * Features exchange link-layer procedure.
    * Version exchange link-layer procedure.
    * Channel map update link-layer procedure.
    * BLE security (at all layers).

During the course of the event, we were able to implement and
successfully test most of the above features.  By the end of the event,
the only two remaining features in the list were the last two (channel
map update and security).

Testing with other implementations exposed some bugs in the nimble
stack.

Bugs discovered and fixed:
    * Respond correctly to unsupported LL control requests (before the
      fix, we would sometimes not respond at all).
    * When a peer discovers our services, respond with 16-bit UUIDs if
      possible (before this fix, we would always send 128-bit UUIDs).
    * When a peer discovers our services, the final service we send
      should indicate an end handle of 0xffff.  This saves the peer from
      having to send a pointless follow-up request.
    * When a connection update procedure fails, the host won't allow
      another one to be initiated during the lifetime of the connection.
    * We don't send a follow up ATT read request during the GATT read
      long characteristic procedure.

Bugs discovered that remain open:
    * When a peer floods us with data packets, we would drop some of the
      packets.  We should acknowledge or nack every packet we receive.

The following list of issues occurred sporadically throughout testing.
These issues were inconsistent, and it was not clear if the nimble stack
was responsible for them, so they will require further investigation.

Unsolved mysteries:
    * The nimble stack terminates a connection immediately after it is
      established due to supervision timeout.
    * The nimble stack terminates a connection with a high slave latency
      parameter (e.g., 499).

Throughout the event, we tested against implementations from the
following organizations:
    * Apple
    * ARM
    * Broadcom
    * CASIO
    * Cypress
    * Dialog
    * EM Microelectronic Marin
    * Google
    * Greenpeak
    * Lapis
    * Marvell
    * MezzTronics
    * Microchip
    * Nordic
    * NXP
    * Qualcomm
    * Roche Indesign
    * Seed Labs
    * Silicon Labs

Thanks for reading,
Chris

Re: UnPlugFest 53

[Greg Stein <gs...@gmail.com>]

Very cool! ... it's great that these events occur. I recall doing the same
for WebDAV interop testing back around 1999.

It sounds like mynewt's support is good, and another event or two should
solidify it. Wonderful!

I saw SparkFun highlighted a bunch of Bluetooth products today, too. Nice
coincidence :-)

Cheers,
-g


On Fri, Feb 5, 2016 at 4:09 PM, Christopher Collins <ch...@runtime.io>
wrote:

> Hello all,
>
> Last week I was in Atlanta, Georgia attending UnPlugFest 53.  UnPlugFest
> is an event held three times a year at which participants get an
> opportunity to test their bluetooth products.  My purpose for being
> there was to test the Mynewt BLE stack (called "nimble").  Each day of
> the event is partitioned into eight one-hour blocks.  For each block, an
> automated system pairs up participants whose implementations support the
> same bluetooth capabilities.  Since nimble is a full BLE stack (both
> host and controller), I got the opportunity to test against a wide
> variety of implementations.  In all, I was scheduled for 25 hour-long
> test sessions against 19 other implementations.  I wanted to use
> this email to briefly summarize what came out of this event.
>
> Overall, I would say the event went very well.  Nearly everything we
> tested worked as expected.  The biggest issue we encountered was our
> lack of support for some functionality that other implementation took
> for granted.  We knew about this going into the event, so while it was a
> nuissance, it was not a surprise.
>
> At the start of the event, nimble did not support any of the following
> features:
>
>     * L2CAP signalling.
>     * L2CAP reassembly.
>     * Connection parameters request link-layer procedure.
>     * Features exchange link-layer procedure.
>     * Version exchange link-layer procedure.
>     * Channel map update link-layer procedure.
>     * BLE security (at all layers).
>
> During the course of the event, we were able to implement and
> successfully test most of the above features.  By the end of the event,
> the only two remaining features in the list were the last two (channel
> map update and security).
>
> Testing with other implementations exposed some bugs in the nimble
> stack.
>
> Bugs discovered and fixed:
>     * Respond correctly to unsupported LL control requests (before the
>       fix, we would sometimes not respond at all).
>     * When a peer discovers our services, respond with 16-bit UUIDs if
>       possible (before this fix, we would always send 128-bit UUIDs).
>     * When a peer discovers our services, the final service we send
>       should indicate an end handle of 0xffff.  This saves the peer from
>       having to send a pointless follow-up request.
>     * When a connection update procedure fails, the host won't allow
>       another one to be initiated during the lifetime of the connection.
>     * We don't send a follow up ATT read request during the GATT read
>       long characteristic procedure.
>
> Bugs discovered that remain open:
>     * When a peer floods us with data packets, we would drop some of the
>       packets.  We should acknowledge or nack every packet we receive.
>
> The following list of issues occurred sporadically throughout testing.
> These issues were inconsistent, and it was not clear if the nimble stack
> was responsible for them, so they will require further investigation.
>
> Unsolved mysteries:
>     * The nimble stack terminates a connection immediately after it is
>       established due to supervision timeout.
>     * The nimble stack terminates a connection with a high slave latency
>       parameter (e.g., 499).
>
> Throughout the event, we tested against implementations from the
> following organizations:
>     * Apple
>     * ARM
>     * Broadcom
>     * CASIO
>     * Cypress
>     * Dialog
>     * EM Microelectronic Marin
>     * Google
>     * Greenpeak
>     * Lapis
>     * Marvell
>     * MezzTronics
>     * Microchip
>     * Nordic
>     * NXP
>     * Qualcomm
>     * Roche Indesign
>     * Seed Labs
>     * Silicon Labs
>
> Thanks for reading,
> Chris
>