Setting up custom database for store forward fucntionality

[mohammedimrankasab]

Hi

I have gone through the https://docs.edgexfoundry.org/3.0/microservices/application/AdvancedTopics/#store-and-forward documentation for enabling the store forward functionality and the example describing the custom database store like jetstream.

we tried to setup Edgex with docker compose in local with mqtt as message bus so that we can configure a different database in the Database section of common configuration. After restarting, in the metadata service we are getting error saying "couldn't create database client: Unsupported database type" and may be below code is causing this.

// Return the dbClient interface
func (d Database) newDBClient(
    lc logger.LoggingClient,
    credentials bootstrapConfig.Credentials) (interfaces.DBClient, error) {
    databaseInfo := d.database.GetDatabaseInfo()
    switch databaseInfo.Type {
    case "redisdb":
        return redis.NewClient(
            db.Configuration{
                Host:     databaseInfo.Host,
                Port:     databaseInfo.Port,
                Password: credentials.Password,
                Timeout:  databaseInfo.Timeout,
            },
            lc)
    default:
return nil, db.ErrUnsupportedDatabase
    }
}

We need help on understanding on if some other custom database like Minio or Cassandra can be used to store the data for the Store forward use case? Also, a detailed example helps in setting up the store forward along with existing Redis/mqtt message bus would be really helpful.

[lenny-goodell]

We need help on understanding on if some other custom database like Minio or Cassandra can be used to store the data for the Store forward use case?

The documentation explains how to create your own Custom Storage and register it for use with Store and Forward. You have to create your own implementation of the StoreClient interface
https://docs.edgexfoundry.org/3.0/microservices/application/AdvancedTopics/#custom-storage
https://docs.edgexfoundry.org/3.0/microservices/application/ApplicationServiceAPI/#registercustomstorefactory

Also, a detailed example helps in setting up the store forward along with existing Redis/mqtt message bus would be really helpful.

Store and Forward is independent of the existing Redis/mqtt message bus. The documentation you referenced explains how to configure it.

Are you using the built-in HTTP or MQTT export function?
If so, all you need to do is enable Store and Forward as described in the documentation.
If not, your custom export function will need to call SetRetryData(payload []byte) on the AppFunctionContext (as described in the documentation) when the export fails. See here as example: https://github.com/edgexfoundry/app-functions-sdk-go/blob/main/pkg/transforms/http.go#L204

[mohammedimrankasab]

Thanks @lenny-intel for your reply. So, does that mean after implementing the custom storage we also need to extend the newDBClient function to add a new case for our custom database so that we do not get the "couldn't create database client: Unsupported database type" error?

I had asked particularly about the MQTT message bus because of the below documentation.

Database configuration section describes which database type to use, and the information required to connect to the database. This section is required if Store and Forward is enabled. It is optional if not using Redis for the EdgeX MessageBus which is now the default.

[lenny-goodell]

Nope, one you have registered your custom implementation with the custom factory method (as shown in the example) it will recognize the name use provided to RegisterCustomStoreFactory
Here is where it is looking for register custom implementations
https://github.com/edgexfoundry/app-functions-sdk-go/blob/main/internal/app/storeforward.go#L57