add support for flux-cloud ui (#22)

* add support for flux-cloud ui this commands brings up the minicluster and starts the web interface/api for interactive use. Signed-off-by: vsoch <[email protected]>
converged-computing · Jan 25, 2023 · 26100d4 · 26100d4
1 parent 2e73ab6
commit 26100d4
Show file tree

Hide file tree

Showing 13 changed files with 216 additions and 106 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -14,6 +14,7 @@ and **Merged pull requests**. Critical items to know are:
 The versions coincide with releases on pip. Only major versions will be released as tags on Github.
 
 ## [0.0.x](https://github.com/converged-computing/flux-cloud/tree/main) (0.0.x)
+ - add flux-cloud ui to just bring up (and down) a user interface (0.1.16)
  - support for submit and batch, to run jobs on the same MiniCluster (0.1.15)
  - minikube docker pull needs message, update tests and typo (0.1.14)
  - wait until pods terminated and removed between applies (0.1.13)

diff --git a/_typos.toml b/_typos.toml
@@ -0,0 +1,4 @@
+# Ignore until we can skip regex
+# https://github.com/crate-ci/typos/issues/651
+[default.extend-words]
+"Fpr" = "Fpr"
diff --git a/docs/getting_started/commands.md b/docs/getting_started/commands.md
@@ -214,6 +214,32 @@ $ flux-cloud submit
 $ flux-cloud down
 ```
 
+## ui
+
+If you are interested in interactive submission on your own, either in the user interface
+or via one of our client SDKs, you can bring up the MiniCluster and it's interface with
+the Flux Restful API with `ui`:
+
+```bash
+$ flux-cloud ui --cloud minikube
+```
+
+If you have many sizes of MiniClusters, you'll need to specify the one that you want:
+
+```bash
+$ flux-cloud ui --cloud minikube --size 4
+```
+
+By default, it will use your single MiniCluster size.
+
+<script id="asciicast-ie6CeWWNIw3NnNpEYGKfRTFpr" src="https://asciinema.org/a/ie6CeWWNIw3NnNpEYGKfRTFpr.js" data-speed="2" async></script>
+
+Which then looks like this in the browser, available for submission via the interface itself
+or the restful API until the user presses control+c to close the port forward and delete
+the MiniCluster.
+
+![img/ui.png](img/ui.png)
+
 ## down
 
 And then bring down your first (or named) cluster:

diff --git a/docs/getting_started/examples.md b/docs/getting_started/examples.md
@@ -22,4 +22,5 @@ which was actually run as:
 ```bash
 $ flux-cloud run
 ```
+
 for the purposes of the demo, and runs a lammps job on two tiny nodes!
diff --git a/docs/getting_started/img/ui.png b/docs/getting_started/img/ui.png
diff --git a/examples/up-submit-down/README.md b/examples/up-submit-down/README.md
@@ -46,6 +46,16 @@ $ flux-cloud batch --cloud minikube
 $ flux-cloud batch --cloud google
 ```
 
+## UI
+
+If you want to just bring up the cluster and open the user interface to interact with:
+
+```bash
+$ flux-cloud up --cloud minikube
+$ flux-cloud ui --cloud minikube
+$ flux-cloud down --cloud minikube
+```
+
 
 ## Plot
 

diff --git a/examples/up-submit-down/data/k8s-size-4-n1-standard-1/.scripts/minicluster-size-2.yaml b/examples/up-submit-down/data/k8s-size-4-n1-standard-1/.scripts/minicluster-size-2.yaml
@@ -17,7 +17,7 @@ spec:
 
   # Optional credentials if running the flux restful api
   fluxRestful:
-    token: "b6223555-a19a-4035-b40f-68a6ce4dd5a5"
+    token: "6b8a7393-129b-4e2d-83a7-795a5a7c9d9b"
     username: "fluxuser"
 
   # TODO add pod resources, if needed

diff --git a/examples/up-submit-down/data/k8s-size-4-n1-standard-1/meta.json b/examples/up-submit-down/data/k8s-size-4-n1-standard-1/meta.json
@@ -1,24 +1,26 @@
 {
     "times": {
         "destroy-cluster": 324.709,
-        "create-cluster": 491.577,
+        "create-cluster": 86.521,
         "minicluster-submit-size-2": 183.626,
         "reaxc-hns-1-minicluster-size-2": 32.1847505569458,
         "reaxc-hns-2-minicluster-size-2": 33.41048860549927,
         "reaxc-hns-3-minicluster-size-2": 30.96457529067993,
         "reaxc-hns-4-minicluster-size-2": 30.777089595794678,
         "reaxc-hns-5-minicluster-size-2": 31.048890829086304,
-        "sleep-1-minicluster-size-2": 5.0783607959747314,
-        "sleep-2-minicluster-size-2": 5.040483474731445,
-        "sleep-3-minicluster-size-2": 5.04453706741333,
-        "sleep-4-minicluster-size-2": 5.048432111740112,
-        "sleep-5-minicluster-size-2": 5.058692455291748,
+        "sleep-1-minicluster-size-2": 5.028888463973999,
+        "sleep-2-minicluster-size-2": 5.045725584030151,
+        "sleep-3-minicluster-size-2": 5.072444677352905,
+        "sleep-4-minicluster-size-2": 5.034207582473755,
+        "sleep-5-minicluster-size-2": 5.025948762893677,
         "hello-world-1-minicluster-size-2": 0.07241106033325195,
         "hello-world-2-minicluster-size-2": 0.052734375,
         "hello-world-3-minicluster-size-2": 0.04248523712158203,
         "hello-world-4-minicluster-size-2": 0.045003652572631836,
         "hello-world-5-minicluster-size-2": 0.05110311508178711,
-        "minicluster-destroy-size-2": 0.277
+        "minicluster-destroy-size-2": 0.277,
+        "minicluster-create-persistent-size-2": 42.606,
+        "minicluster-persistent-destroy-size-2": 0.164
     },
     "size": 4,
     "machine": "n1-standard-1",
@@ -313,15 +315,15 @@
             "start_to_output_seconds": 92.92412114143372
         },
         "sleep-1-minicluster-size-2": {
-            "id": 6190792704000,
+            "id": 461004341248,
             "userid": 1234,
             "urgency": 16,
             "priority": 16,
-            "t_submit": 1674445129.2985795,
-            "t_depend": 1674445129.2985795,
-            "t_run": 1674445129.3183057,
-            "t_cleanup": 1674445134.3966665,
-            "t_inactive": 1674445134.400485,
+            "t_submit": 1674677415.8718548,
+            "t_depend": 1674677415.8718548,
+            "t_run": 1674677415.8845603,
+            "t_cleanup": 1674677420.9134488,
+            "t_inactive": 1674677420.9152129,
             "state": "INACTIVE",
             "name": "sleep",
             "ntasks": 1,
@@ -331,73 +333,73 @@
             "success": true,
             "exception_occurred": false,
             "result": "COMPLETED",
-            "expiration": 1675049929.0,
+            "expiration": 1675282215.0,
             "annotations": {
                 "sched": {
                     "queue": "default"
                 }
             },
             "waitstatus": 0,
             "returncode": 0,
-            "runtime": 5.0783607959747314,
+            "runtime": 5.028888463973999,
             "exception": {
                 "occurred": false,
                 "severity": "",
                 "type": "",
                 "note": ""
             },
             "duration": "",
-            "start_to_info_seconds": 62.536019802093506,
-            "start_to_output_seconds": 62.61232876777649
+            "start_to_info_seconds": 14.840466737747192,
+            "start_to_output_seconds": 17.383413314819336
         },
         "sleep-2-minicluster-size-2": {
-            "id": 7241700737024,
+            "id": 717628637184,
             "userid": 1234,
             "urgency": 16,
             "priority": 16,
-            "t_submit": 1674445191.9373004,
-            "t_depend": 1674445191.9373004,
-            "t_run": 1674445191.9571338,
-            "t_cleanup": 1674445196.9976172,
-            "t_inactive": 1674445197.001546,
+            "t_submit": 1674677431.16695,
+            "t_depend": 1674677431.16695,
+            "t_run": 1674677431.1903481,
+            "t_cleanup": 1674677436.2360737,
+            "t_inactive": 1674677436.2395134,
             "state": "INACTIVE",
             "name": "sleep",
             "ntasks": 1,
             "nnodes": 1,
-            "ranks": "1",
-            "nodelist": "lammps-job-1",
+            "ranks": "0",
+            "nodelist": "lammps-job-0",
             "success": true,
             "exception_occurred": false,
             "result": "COMPLETED",
-            "expiration": 1675049991.0,
+            "expiration": 1675282231.0,
             "annotations": {
                 "sched": {
                     "queue": "default"
                 }
             },
             "waitstatus": 0,
             "returncode": 0,
-            "runtime": 5.040483474731445,
+            "runtime": 5.045725584030151,
             "exception": {
                 "occurred": false,
                 "severity": "",
                 "type": "",
                 "note": ""
             },
             "duration": "",
-            "start_to_info_seconds": 13.608879566192627,
-            "start_to_output_seconds": 16.584359407424927
+            "start_to_info_seconds": 12.824117660522461,
+            "start_to_output_seconds": 15.347451210021973
         },
         "sleep-3-minicluster-size-2": {
-            "id": 7520689061888,
+            "id": 975108571136,
             "userid": 1234,
             "urgency": 16,
             "priority": 16,
-            "t_submit": 1674445208.5661342,
-            "t_depend": 1674445208.5661342,
-            "t_run": 1674445208.588134,
-            "t_cleanup": 1674445213.632671,
-            "t_inactive": 1674445213.63706,
+            "t_submit": 1674677446.5178363,
+            "t_depend": 1674677446.5178363,
+            "t_run": 1674677446.534995,
+            "t_cleanup": 1674677451.6074398,
+            "t_inactive": 1674677451.613382,
             "state": "INACTIVE",
             "name": "sleep",
             "ntasks": 1,
@@ -407,35 +409,35 @@
             "success": true,
             "exception_occurred": false,
             "result": "COMPLETED",
-            "expiration": 1675050008.0,
+            "expiration": 1675282246.0,
             "annotations": {
                 "sched": {
                     "queue": "default"
                 }
             },
             "waitstatus": 0,
             "returncode": 0,
-            "runtime": 5.04453706741333,
+            "runtime": 5.072444677352905,
             "exception": {
                 "occurred": false,
                 "severity": "",
                 "type": "",
                 "note": ""
             },
             "duration": "",
-            "start_to_info_seconds": 13.602198600769043,
-            "start_to_output_seconds": 16.66467046737671
+            "start_to_info_seconds": 12.840857744216919,
+            "start_to_output_seconds": 15.384143352508545
         },
         "sleep-4-minicluster-size-2": {
-            "id": 7800382029824,
+            "id": 1234333335552,
             "userid": 1234,
             "urgency": 16,
             "priority": 16,
-            "t_submit": 1674445225.2364438,
-            "t_depend": 1674445225.2364438,
-            "t_run": 1674445225.2567546,
-            "t_cleanup": 1674445230.3051867,
-            "t_inactive": 1674445230.309494,
+            "t_submit": 1674677461.9656863,
+            "t_depend": 1674677461.9656863,
+            "t_run": 1674677461.9789429,
+            "t_cleanup": 1674677467.0131505,
+            "t_inactive": 1674677467.0233643,
             "state": "INACTIVE",
             "name": "sleep",
             "ntasks": 1,
@@ -445,35 +447,35 @@
             "success": true,
             "exception_occurred": false,
             "result": "COMPLETED",
-            "expiration": 1675050025.0,
+            "expiration": 1675282261.0,
             "annotations": {
                 "sched": {
                     "queue": "default"
                 }
             },
             "waitstatus": 0,
             "returncode": 0,
-            "runtime": 5.048432111740112,
+            "runtime": 5.034207582473755,
             "exception": {
                 "occurred": false,
                 "severity": "",
                 "type": "",
                 "note": ""
             },
             "duration": "",
-            "start_to_info_seconds": 13.53053617477417,
-            "start_to_output_seconds": 16.569458484649658
+            "start_to_info_seconds": 12.951504468917847,
+            "start_to_output_seconds": 15.509077787399292
         },
         "sleep-5-minicluster-size-2": {
-            "id": 8079202582528,
+            "id": 1495168712704,
             "userid": 1234,
             "urgency": 16,
             "priority": 16,
-            "t_submit": 1674445241.855504,
-            "t_depend": 1674445241.855504,
-            "t_run": 1674445241.8752344,
-            "t_cleanup": 1674445246.9339268,
-            "t_inactive": 1674445246.9379504,
+            "t_submit": 1674677477.5129235,
+            "t_depend": 1674677477.5129235,
+            "t_run": 1674677477.5259533,
+            "t_cleanup": 1674677482.551902,
+            "t_inactive": 1674677482.555279,
             "state": "INACTIVE",
             "name": "sleep",
             "ntasks": 1,
@@ -483,24 +485,24 @@
             "success": true,
             "exception_occurred": false,
             "result": "COMPLETED",
-            "expiration": 1675050041.0,
+            "expiration": 1675282277.0,
             "annotations": {
                 "sched": {
                     "queue": "default"
                 }
             },
             "waitstatus": 0,
             "returncode": 0,
-            "runtime": 5.058692455291748,
+            "runtime": 5.025948762893677,
             "exception": {
                 "occurred": false,
                 "severity": "",
                 "type": "",
                 "note": ""
             },
             "duration": "",
-            "start_to_info_seconds": 13.458645582199097,
-            "start_to_output_seconds": 16.474400281906128
+            "start_to_info_seconds": 12.880193948745728,
+            "start_to_output_seconds": 15.410512447357178
         },
         "hello-world-1-minicluster-size-2": {
             "id": 8356177641472,

diff --git a/fluxcloud/client/__init__.py b/fluxcloud/client/__init__.py
@@ -130,6 +130,11 @@ def get_parser():
         description="Bring the cluster up, run experiments via applying CRDs, and bring it down.",
         formatter_class=argparse.RawTextHelpFormatter,
     )
+    ui = subparsers.add_parser(
+        "ui",
+        description="Once the cluster is up, create/open the user interface.",
+        formatter_class=argparse.RawTextHelpFormatter,
+    )
     batch = subparsers.add_parser(
         "batch",
         description="Bring the cluster up, run experiments via a Flux Restful API submit, and bring it down.",
@@ -168,7 +173,7 @@ def get_parser():
         description="List experiment ids available.",
         formatter_class=argparse.RawTextHelpFormatter,
     )
-    for command in run, up, down, apply, listing, batch, submit:
+    for command in run, up, down, apply, listing, batch, submit, ui:
         command.add_argument(
             "experiments",
             default="experiments.yaml",
@@ -183,7 +188,7 @@ def get_parser():
             choices=clouds.cloud_names,
         )
 
-    for command in apply, up, down, run, batch, submit:
+    for command in apply, up, down, run, batch, submit, ui:
         command.add_argument(
             "--force-cluster",
             dest="force_cluster",
@@ -292,6 +297,8 @@ def help(return_code=0):
         from .run import batch as main
     elif args.command == "config":
         from .config import main
+    elif args.command == "ui":
+        from .ui import main
     elif args.command == "up":
         from .up import main
     elif args.command == "down":
-Original file line number
+Diff line change
@@ Expand Up / @@ -22,4 +22,5 @@ which was actually run as: @@
     ```bash
     $ flux-cloud run
     ```
     for the purposes of the demo, and runs a lammps job on two tiny nodes!