Device is not persisting configured number of bonds

[jefflongo]

Not sure if this is the right place to post this issue.. but I have a device with which I am bonding multiple devices. I have enabled CONFIG_BT_NIMBLE_NVS_PERSIST and configured CONFIG_BT_NIMBLE_MAX_BONDS to 5, but it appears that when bonding a 4th device the ESP32 loses the encryption key of one of the other 3 devices and I must repair it. So it appears I can only maintain 3 encryption keys in NVS. Any idea what I could be doing wrong or is this a bug?

[thekurtovic]

Just an idea, maybe adding a callback to check store status could provide some insight?
#295

Does it make any difference if you change max bonds/ccds to a higher value like 16?

[h2zero]

You may have had an old bond still stored, try erasing the flash and start again.

[jefflongo]

You may have had an old bond still stored, try erasing the flash and start again.

Supposing this is the case, I would expect that the stale bonds get deleted first. When I start repairing the other devices I can see one of the other devices becomes unpaired. I can keep doing this and cycling through only 3 devices being paired.

[jefflongo]

Just an idea, maybe adding a callback to check store status could provide some insight? #295

Does it make any difference if you change max bonds/ccds to a higher value like 16?

I have max bonds/ccds set to 8 now, no difference. Still only able to maintain 3 pairs. I tried checking out your store status branch. After working around a compile error (the storage for the default store status callbacks is not allocated leading to linker error), I noticed that when attempting to repair a device for which the encryption key was lost, the store status callback with the BLE_STORE_EVENT_OVERFLOW code was called repeatedly and very quickly, so much so that my log messages caused the watchdog timer to reset.

[thekurtovic]

Interesting, thanks for testing. I've seen my esp32's go beyond 3 bonds, strange that you are having an issue with it.
I take it there's no warnings/errors emitted from the controller?

FWIW here's where the event is originating
https://github.com/espressif/esp-nimble/blob/a4cd6027a544966d569db92c68e557f09e1f3d76/nimble/host/src/ble_store.c#L61-L71

Also forgot some documentation in my PR, but the callback should return either 0 or 1.

/**
 * Indicates an inability to perform a store operation.  This callback should
 * do one of two things:
 *     o Address the problem and return 0, indicating that the store operation
 *       should proceed.
 *     o Return nonzero to indicate that the store operation should be aborted.
 *
 * @param event                 Describes the store event being reported.
 * @param arg                   Optional user argument.
 *
 * @return                      0 if the store operation should proceed;
 *                              nonzero if the store operation should be
 *                                  aborted.
 */

[jefflongo]

Doing some more testing.. I was able to pair all 4 after erasing the flash. I'm not sure why that step was necessary though or if I will run into this issue again at some later time. I question the logic for deciding when/how keys in NVS get replaced.

Interesting, thanks for testing. I've seen my esp32's go beyond 3 bonds, strange that you are having an issue with it. I take it there's no warnings/errors emitted from the controller?

FWIW here's where the event is originating https://github.com/espressif/esp-nimble/blob/a4cd6027a544966d569db92c68e557f09e1f3d76/nimble/host/src/ble_store.c#L61-L71

Also forgot some documentation in my PR, but the callback should return either 0 or 1.

/**
 * Indicates an inability to perform a store operation.  This callback should
 * do one of two things:
 *     o Address the problem and return 0, indicating that the store operation
 *       should proceed.
 *     o Return nonzero to indicate that the store operation should be aborted.
 *
 * @param event                 Describes the store event being reported.
 * @param arg                   Optional user argument.
 *
 * @return                      0 if the store operation should proceed;
 *                              nonzero if the store operation should be
 *                                  aborted.
 */

Yeah no errors were getting emitted, at least at my configured WARN level or higher. That explains that problem then, I was returning 0 so that probably just resulted in an infinite loop/undefined behavior.

[thekurtovic]

Stupid question: you had already tried calling NimBLEDevice::deleteAllBonds() right?
Requiring a full erase is concerning

[jefflongo]

No, I have not. TBH I didn't know that function even existed. Unfortunately now that I have erased the flash, I'm no longer able to currently reproduce the problem. I think I will close this issue and reopen if the issue comes up again at some later time.