add http usage publisher using rpc, update readme

.env.sample

@@ -38,3 +38,6 @@ PG_EVENTS_PASS=<postgres password>
 PG_EVENTS_PORT=<postgres port>
 PG_EVENTS_HOST=<postgres host>
 PG_EVENTS_DB=<postgres database>
+
+# Usage Event HTTP Publisher (Optional)
+HTTP_PUBLISHER_ENDPOINT=<https://example.com>

README.md

@@ -7,6 +7,9 @@ you will need to make sure this container connects to the same docker network or
 <p>
 max packets per skfs are set to 0 by default and updated on activation. if not set by the tenant this will automatically be set as the same as your max packets set for the route id. if you wish to set these per device by the skfs, you need to set `device -> configuration -> variables`. click on add variable, set key name to `max_copies`, set the value as an integer for the amount of packets per device you want to purchase if it is under or above the default route id max packets for the device.
 </p>
+<p>
+if you find this useful give us a star or clone the repo and contribute.
+</p>
 
 ## Helium Openlns
 - Own your own helium OUI.
@@ -17,14 +20,22 @@ max packets per skfs are set to 0 by default and updated on activation. if not s
 ## Initial Helium Setup
 - [Helium Roaming Quickstart](https://docs.helium.com/iot/lorawan-roaming/#roaming-quickstart)
 
+## To Get Started
+- git clone this repository.
+- create an external attachable network eg. `docker network create core-infra` to attach chirpstack-docker and chirpstack-hpr to.
+- edit network section in the docker compose to attach to the same docker network created above in chirpstack-docker compose.
+- rename `.env.sample` to `.env` and edit with required settings applicable to your installation. you should name the hosts needed by there docker container name and not docker IP address as this can change between restarts causing breakage.
+- `docker compose up` should pull the current container from gh and start the container check for any errors. if you need to change things its usually best to recreate the containers `ctrl-c` to terminate current running container and disgard with `docker compose down --remove-orphans` make and save any changes as needed. once you have it right start docker in detached mode `docker compose up -d`
+
+
 ## TODO:
 1. possibly migrate sql to more friendly pythonic sqlalchemy ORM.
 
 ## Completed
 1. get python working with helium-crypto.rs and helium/proto to make changes directly over the wire.
 thanks to groot for getting the inital rpc working for signing changes [helium-iot-config-py](https://github.com/mawdegroot/helium-iot-config-py)
 
-2. external integrations to account for by device and by tenant dc usage. currently supports aws sqs & postgres.
+2. external integrations to account for by device and by tenant dc usage. currently supports aws sqs, postgres and http (using rpc).
 
 ## Create chirpstack integration database.
 - This step is no longer required, a sepreate table will be created in the regular postgres db to keep a synced record of

app/Publishers/HttpUsagePublisher.py

@@ -0,0 +1,27 @@
+import os
+import urllib3
+from . import tenant_usage_pb2
+
+
+usage_endpoint = os.getenv('HTTP_PUBLISHER_ENDPOINT', None)
+
+timeout = urllib3.Timeout(connect=2.0, read=7.0)
+HTTP = urllib3.PoolManager(timeout=timeout)
+
+
+def publish_to_http(usage_event: dict):
+    event = tenant_usage_pb2.TenantUsage()
+    event.datetime = usage_event['datetime']
+    event.dev_eui = usage_event['dev_eui']
+    event.tenant_id = usage_event['tenant_id']
+    event.application_id = usage_event['application_id']
+    event.dc_used = usage_event['dc_used']
+    payload = event.SerializeToString()
+
+    r = HTTP.request(
+        'POST',
+        usage_endpoint,
+        body=payload,
+        headers={'Content-Type': 'application/octet-stream'}
+    )
+    return r

app/Publishers/tenant_usage_pb2.pyi

@@ -0,0 +1,19 @@
+from google.protobuf import descriptor as _descriptor
+from google.protobuf import message as _message
+from typing import ClassVar as _ClassVar, Optional as _Optional
+
+DESCRIPTOR: _descriptor.FileDescriptor
+
+class TenantUsage(_message.Message):
+    __slots__ = ["datetime", "dev_eui", "tenant_id", "application_id", "dc_used"]
+    DATETIME_FIELD_NUMBER: _ClassVar[int]
+    DEV_EUI_FIELD_NUMBER: _ClassVar[int]
+    TENANT_ID_FIELD_NUMBER: _ClassVar[int]
+    APPLICATION_ID_FIELD_NUMBER: _ClassVar[int]
+    DC_USED_FIELD_NUMBER: _ClassVar[int]
+    datetime: str
+    dev_eui: str
+    tenant_id: str
+    application_id: str
+    dc_used: int
+    def __init__(self, datetime: _Optional[str] = ..., dev_eui: _Optional[str] = ..., tenant_id: _Optional[str] = ..., application_id: _Optional[str] = ..., dc_used: _Optional[int] = ...) -> None: ...

app/Publishers/tenant_usage_pb2_grpc.py

@@ -0,0 +1,4 @@
+# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
+"""Client and server classes corresponding to protobuf-defined services."""
+import grpc
+

app/UsagePublisher.py

@@ -32,6 +32,11 @@ def publish_usage_event(dev_eui, tenant_id, application_id, dc_used):
             info_log("Publishing usage event to PG: %s" % usage_event)
             publish_to_pg(usage_event)
 
+        case 'HTTP':
+            from Publishers.HttpUsagePublisher import publish_to_http
+            info_log("Publishing usage event to HTTP: %s" % usage_event)
+            publish_to_http(usage_event)
+
         case _:
             info_log("Provider %s not found" % provider)
             return

docker-compose-dev.yml

@@ -41,10 +41,11 @@ services:
       - PG_EVENTS_PORT=${PG_EVENTS_PORT}
       - PG_EVENTS_HOST=${PG_EVENTS_HOST}
       - PG_EVENTS_DB=${PG_EVENTS_DB}
+      # Usage Event Http Publisher (Optional)
+      - HTTP_PUBLISHER_ENDPOINT=${HTTP_PUBLISHER_ENDPOINT}
     command: bash -c 'cd /app && python app.py'
 
 networks:
   default:
     name: core-infra
     external: true
-

docker-compose.yml

@@ -40,6 +40,8 @@ services:
       - PG_EVENTS_PORT=${PG_EVENTS_PORT}
       - PG_EVENTS_HOST=${PG_EVENTS_HOST}
       - PG_EVENTS_DB=${PG_EVENTS_DB}
+      # Usage Event Http Publisher (Optional)
+      - HTTP_PUBLISHER_ENDPOINT=${HTTP_PUBLISHER_ENDPOINT}
     command: bash -c 'cd /app && python app.py'
 
 networks: