Skip to content
/ JAMSched Public

Pseudo Real-Time Task Scheduler + Coroutine Engine + Tiny RPC on MQTT

License

Notifications You must be signed in to change notification settings

yma67/JAMSched

Folders and files

NameName
Last commit message
Last commit date

Latest commit

c16a2cf · Oct 24, 2020
Oct 23, 2020
Oct 23, 2020
Oct 18, 2020
Oct 24, 2020
Oct 24, 2020
Oct 23, 2020
May 5, 2020
Sep 26, 2020
Aug 16, 2020
Aug 22, 2020
Oct 2, 2020
Oct 22, 2020
Jul 21, 2020
Jul 21, 2020

Repository files navigation

JAMSched Build Status

Real Time Scheduler + Parallel Coroutine Engine for JAMScript C (Device) Side, with support of Remote Procedure Call and separate data-transfer path on Redis

Scheduling

Resources

Threads

  • 1 Kernel Level Thread for Real-Time/Interactive Jobs (jobs with deadline/time constraints)
  • n Kernel Level Thread for Batch Jobs (jobs without time constraints)

Coroutines

  • Capable of supporting shared-stack coroutines and stand-alone stack coroutines, scheduled with no difference
  • Automatic type deduction of C++ functions
  • Each task is allocated to as a coroutine, mapped to 1 kernel level thread
  • More than 1M tasks per GB
  • Switching time: ~30ns on Intel Core i7 10710U

Real Time Tasks

Description

  • Task with hard start/deadline
  • Scheduled on Real-Time Kernel Level Thread
  • All Real-Time tasks runs in a Single Kernel Thread

Scheduling Strategy

  • A static schedule of Real-Time tasks as a finite list of time intervals are used to schedule Hard Real-Time tasks
  • Each time interval has a RT-Task ID, beginning of an interval defines the start of the task with the ID
  • An ID equals to 0 defines a slot for Non-Real-Time tasks
  • Schedule of Real Time Tasks are downloaded for each cycle
  • Length of the cycle is defined by the end time of last interval in the finite list
  • Time Point defined in the interval are time points relative to the start of the cycle
  • Scheduler picks up Real Time task according to the ID associated with interval
  • 2 schedules with different distributions of Non-RealTime slots are provided, picked up according to some algorithm

Performance

Jitter
  • Jitter = (StartTimeOfRTTask - BeginOfTimeInterval), relative to the start of cycle
  • Jitter is within 20us, usually within 10us

Interactive Tasks

Description

  • Task with soft deadline
  • Scheduled on Real-Time Kernel Level Thread, possibly Non-Real-Time Kernel Level Thread if expired

Scheduling Strategy

  • If task has not passed its deadline, schedule it in Earliest Deadline First
  • Otherwise, if the programmer allows this task to degrade to a Batch Task, it will degrade.
  • Otherwise, it would be pushed into a Bounded LIFO Stack, and will be executed if there is no task in EDF queue
  • If the LIFO stack exceeded its capacity, it would pop out (cancel) the eariest-pushed task
  • Just like how we process our emails, assignments, and messages...

Batch Task

Description

  • Task without deadline
  • Scheduled on Non-Real-Time Kernel Level Thread
  • Capable of Parallel Execution

Scheduling Strategy

  • Only Batch Tasks could be executed in worker threads
  • FIFO with Work Steal (between Non-Real-Time Kernel Level Threads)
  • Work Steal is triggered if one Non-Real-Time Kernel Level Thread run out of tasks
  • Load Balancing: Tasks are distributed by the main scheduler to the Non-Real-Time Kernel Level Threads with least amout of tasks

Synchronization Primitives

Purpose

  • Synchronize at user-space, without blocking Kernel-Level thread
  • Blocking Kernel level threads could break the real-time constraints
  • Easier for Asynchronous Programming

Spin Mutex

Condition Variable

Mutex

Semaphore

Future

Timer as Timing Wheel

  • Powered by timeout.c: Tickless Hierarchical Timing Wheel

Precision

  • Ideally 30us, normally 50us

Remote Procedure Call (RPC)

This is a Toy-Level simple ADD-ON of RPC capability to the scheduler

  • Each RPC Invocation is matched to a Batch Task

Fault-torelance of this RPC depends on JAMScript JavaScript-Side

Protocol

  • JSON/Cbor on MQTT

Fault-torelance

  • Heartbeat monitoring
  • Multiple connections
  • Acknowledgement, retry and timeout of connection failure

Note

Please return by a pointer to memory allocated on heap by malloc/calloc to avoid memory leak.

Example

jamc::RIBScheduler ribScheduler(1024 * 256, "tcp://localhost:1883", "app-1", "dev-1");
ribScheduler.RegisterRPCall("DuplicateCString", strdup);
ribScheduler.RegisterRPCall("RPCFunctionJSync", [] (int a, int b) -> int {
    std::cout << "Sync Add of " << a << " + " << b << std::endl;
    return a + b;
});
ribScheduler.RegisterRPCall("RPCFunctionJAsync", [] (int a, int b) -> int {
    std::cout << "Async Subtract of " << a << " - " << b << std::endl;
    return a - b;
});
ribScheduler.RegisterRPCall("ConcatCString", [] (char *dest, const char *src) -> char* {
    printf("please return by a pointer to memory allocated on heap");
    return strdup(strcat(dest, src));
});

Named Local Invocation

Example of Registration

jamc::RIBScheduler ribScheduler(1024 * 256);
ribScheduler.RegisterLocalExecution("TestExec", [] (int a, int b) -> int {
    return a + b;
});

Example of Invocation

auto fu = ribScheduler.CreateLocalNamedInteractiveExecution<int>(
    // Interactive Task Attributes
    {false, 1024}, std::chrono::milliseconds(1000), std::chrono::microseconds(50),
    // Execution Name
    std::string("TestExec"),
    // Execution Parameters
    3, 4
);
// after some lines of code...
int val = fu.get();

Note

Also available for Batch Tasks

Exception

InvalidArgumentException

Although there is std::invalid_argument, to appreciate Prof. Gunter Mussbacher giving me an A- grade in ECSE223 - Model Based Programming, I would like to separately define the class to be InvalidArgumentException, which is the same InvalidArgumentException from ECSE223 course project.

General Workflow

Create an application

  • goto app folder, and make a folder for your application
  • add a CMakeLists.txt for configuration and link your favoriate machine learning libraries here
  • add a main.c/main.cpp, and other files you need
  • make sure your include your folder in app/CMakeLists.txt
  • if you are VSCode Users, goto VSCode Users - Reasons to Use

Create a module

  • goto src folder, and make a folder for your module
  • add a CMakeLists.txt for your library
  • add files you need
  • make sure your include your folder in src/CMakeLists.txt

Testing

CiteLab Coding Style (draft)

  • vscode formatter style: Visual Studio
  • avoid unnecessary memcpy/copy construct of large object
  • avoid unnecessary notify of threads on Condition Variable
  • avoid random comment outs without explanation
  • avoid unhandled memory allocations, like strdup
  • more to come...

Note to VSCode Users

Reasons to use

  • free yourselves from gdb, cmake... commands
  • looks cool

Build

Prereq

  • boost (>= 1.65, but preferred 1.73+)
  • paho-mqtt3a
  • hiredis
  • libevent
  • tcmalloc

Note

  • install CMake Tools, available from https://marketplace.visualstudio.com/items?itemName=ms-vscode.cmake-tools
  • configure, and build all project using the top right buttons
  • supported compiler: Clang 6.0.0, Clang 9.0.0, Clang 11.0.0, GCC 9.3.0, GCC 10.1.0
  • on macos, use cmake -D CMAKE_C_COMPILER=/usr/local/opt/llvm/bin/clang -D CMAKE_CXX_COMPILER=/usr/local/opt/llvm/bin/clang++ ../

Debug

  • Goto Run tab, and choose which app you would like to add/debug
  • To debug testcases, run VSCode task "clang-9 - test-debug"

Debug Your App

  • Add the following "chunk" in launch.json
  • <name>: name of your app folder
  • <exec>: name of your app executable
{
    "name": "clang-9 - <name>",
    "type": "cppdbg",
    "request": "launch",
    "program": "${workspaceFolder}/build/app/<name>/<exec>",
    "args": [],
    "stopAtEntry": false,
    "cwd": "${workspaceFolder}/build/app/<name>",
    "environment": [],
    "externalConsole": false,
    "MIMode": "lldb",
    "setupCommands": [
        {
            "description": "format lldbprint",
            "text": "-enable-pretty-printing",
            "ignoreFailures": true
        }
    ],
    "preLaunchTask": "build project",
    "miDebuggerPath": "/usr/bin/lldb"
}