Skip to content

Tal Mizrahi's Final Project for Computer Science Bachelor's, Recognized as Outstanding for the Class of 2024

Notifications You must be signed in to change notification settings

TalMizrahii/Firmlight

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

46 Commits
 
 
 
 
 
 
 
 

Repository files navigation

HTML

Firmlight - Task Distribution System

Tal Mizrahi's Final Project for Computer Science Bachelor's, Recognized as Outstanding for the Class of 2024

DescriptionArchitectureAPISetupAuthor

Description

I developed a distributed task management system designed to efficiently assign tasks within user groups. The system includes a React.js client where users can create and manage groups, and a NestJS backend connected to MongoDB for securely storing user data. A second client, built in Python, will allow users to download and connect to the system with their existing credentials. Tasks will be distributed across group members, with only metadata and task statistics stored on the server, ensuring minimal data storage while maintaining robust task coordination.


You can browse to the web managment page here.

IMAGE ALT TEXT HERE

Architecture

The Firmlight system consists of the following components:

  1. NestJS Server: Manages users, groups, and task distribution.
  2. Reactjs Client: Web interface for users to manage groups and initiate tasks.
  3. Python Worker Client: Downloadable client for processing distributed tasks.
  4. MongoDB: Database for storing user data, group information, and task metadata.
  5. Redis: In-memory data structure store used for caching and as a message broker.
  6. BullMQ: Queue system for handling distributed job processing.
  7. cron-jobs: Scheduled tasks for monitoring and maintenance.
  8. JSON Web Tokens: An authentication for representing claims securely between The server and clients.
Final_arch drawio

Workflow

  1. Users register/login via the React client, with credentials stored in MongoDB.
  2. Users create or join groups through the React interface.
  3. When a task is sent to a group, the server uses BullMQ to queue and distribute it among group members.
  4. Python worker clients process the tasks and send results back to the server.
  5. The server updates task status in real-time using WebSockets.
  6. Task results are sent to the appropriate client (React or Python).
  7. Only task metadata and statistics are stored on the server to maintain efficiency.
  8. Cron jobs periodically monitor task progress and perform system maintenance.
Final_arch drawio

API

For detailed API documentation, please refer to our API Docs in the Swagger interface.

Final_arch drawio

Setup

# Clone the repository
git clone https://github.com/TalMizrahii/firmlight.git

# Navigate to the project directory
cd firmlight/FirmlightWorker

# Install dependencies
pip install

# Sign up using the web interface
Browse to https://firmlight.onrender.com

# Login to the worker and start to work!
Using the same credentials of the web managment page

Author

About

Tal Mizrahi's Final Project for Computer Science Bachelor's, Recognized as Outstanding for the Class of 2024

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published