Skip to content

Memphis is an Open-Source, Real-Time Data Processing Platform

License

Notifications You must be signed in to change notification settings

zuhaibz/memphis-broker

ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Memphis light logo

Memphis light logo

Memphis UI

CNCF Silver Member

CNCF Silver Member

Sandbox - Docs - Twitter - YouTube

Discord Code Of Conduct GitHub release (latest by date)

Memphis is a next-generation message broker.
A simple, robust, and durable cloud-native message broker wrapped with
an entire ecosystem that enables fast and reliable development of next-generation event-driven use cases.

Memphis enables building next-generation applications that require large volumes of streamed and enriched data,
modern protocols, zero ops, rapid development, extreme cost reduction,
and a significantly lower amount of dev time for data-oriented developers and data engineers.

๐Ÿ“ธ Screenshots

Dashboard Station (Topic) overview CLI
Dashboard Station Overview drawing

โญ๏ธ Why

When your application requires a message broker or a queue for various reasons,
Implementing one will require you to -

  • Build a dead-letter queue
  • Create observability
  • Build a scalable environment
  • Create client wrappers
  • Handle back pressure. Client or queue side
  • Create a retry mechanism
  • Configure monitoring and real-time alerts
  • Deal with ACLs
  • Somehow create a cloud-agnostic implementation
  • Create config alignment between production to a dev environment
  • Spent weeks and months learning the internals through archival documentation, ebooks, and courses
  • Onboard your developers
    And the list continues...

Or, you can just use Memphis broker and focus your resources on tasks that matter

๐Ÿ‘‰ Use-cases

  • Async task management
  • Real-time streaming pipelines
  • Data ingestion
  • Cloud Messaging
    • Services (microservices, service mesh)
    • Event/Data Streaming (observability, analytics, ML/AI)
  • Queuing
  • N:N communication patterns

โœจ Features

Roadmap

v0.4.2

  • ๐Ÿš€ Fully optimized message broker in under 3 minutes
  • ๐Ÿ’ป Easy-to-use UI, CLI, and SDKs
  • ๐Ÿ“บ Data-level observability
  • โ˜ ๏ธ Dead-Letter Queue with automatic message retransmit
  • ๐Ÿ”ค Schemaverse - Embedded schema management for produced data (Protobuf/JSON/GraphQL/Avro)
  • โ›“ SDKs: Node.JS, Go, Python, TypeScript, NestJS
  • ๐Ÿณโ˜ธ Runs on your Docker or Kubernetes
  • ๐Ÿ‘จโ€๐Ÿ’ป Community driven

๐Ÿš€ Getting Started

Sandbox
Installation videos

Helm for Kubernetesโ˜ธ

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && \
helm install my-memphis memphis/memphis --create-namespace --namespace memphis

Docker๐Ÿณ Compose

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && \
docker compose -f docker-compose.yml -p memphis up

connect your first app

Build an event-driven food delivery app

High-Level Architecture

memphis.dev-logo

Local access

Via Kubernetes

To access Memphis using UI/CLI/SDK from localhost, run the below commands:

  - kubectl port-forward service/memphis-cluster 6666:6666 9000:9000 7770:7770 --namespace memphis > /dev/null &

For interacting with the broker via HTTP:

  - kubectl port-forward service/memphis-http-proxy 4444:4444 --namespace memphis > /dev/null &

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666 (Client Connections)
HTTP proxy: localhost:4444 (Data + Mgmt)

For Production Environments Please expose the UI, Cluster, and Control-plane via k8s ingress / load balancer / nodeport

Via Docker

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666

Beta

Memphis{dev} is currently in Beta version. This means that we are still working on essential features like real-time messages tracing, schema registry and inline processing as well as making more SDKs and supporting materials.

How does it affect you? Well... mostly it doesn't.
(a) The core of memphis broker is highly stable
(b) We learn and fix fast

But we need your love, and any help we can get by stars, PR, feedback, issues, and enhancements.
Read more on Memphis{dev} Documentation ๐Ÿ“ƒ.

Support ๐Ÿ™‹โ€โ™‚๏ธ๐Ÿค

Ask a question โ“ about Memphis{dev} or something related to us:

We welcome you to our discord server with your questions, doubts and feedback.

Create a bug ๐Ÿž report

If you see an error message or run into an issue, please create bug report. This effort is valued and it will help all Memphis{dev} users.

Submit a feature ๐Ÿ’ก request

If you have an idea, or you think that we're missing a capability that would make development easier and more robust, please Submit feature request.

If an issueโ—with similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)

Contributing

Memphis{dev} is an open-source project.
We are committed to a fully transparent development process and appreciate highly any contributions.
Whether you are helping us fix bugs, proposing new features, improving our documentation or spreading the word - we would love to have you as part of the Memphis{dev} community.

Please refer to our Contribution Guidelines and Code of Conduct.

Contributors โœจ

Thanks goes to these wonderful people โค:

License ๐Ÿ“ƒ

Memphis is open-sourced and operates under the "Memphis Business Source License 1.0" license Built out of Apache 2.0, the main difference between the licenses is: "You may make use of the Licensed Work (i) only as part of your own product or service, provided it is not a message broker or a message queue product or service; and (ii) provided that you do not use, provide, distribute, or make available the Licensed Work as a Service. A โ€œServiceโ€ is a commercial offering, product, hosted, or managed service, that allows third parties (other than your own employees and contractors acting on your behalf) to access and/or use the Licensed Work or a substantial set of the features or functionality of the Licensed Work to third parties as a software-as-a-service, platform-as-a-service, infrastructure-as-a-service or other similar services that compete with Licensor products or services." Please check out License to read the full text.

About

Memphis is an Open-Source, Real-Time Data Processing Platform

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 86.1%
  • JavaScript 11.0%
  • SCSS 2.6%
  • Other 0.3%