Code

Memory prediction changes#12

Code

Merged

Lehmann-Fabian merged 76 commits intomasterCommonWorkflowScheduler/KubernetesScheduler:masterfrom memoryPredictionChangesCommonWorkflowScheduler/KubernetesScheduler:memoryPredictionChangesCopy head branch name to clipboardJul 19, 2024

+3,227-138

Commits

Commits on Mar 4, 2024

• Memory prediction and task scaling

  Show description for fb7ebac
  friederici

  authored
  fb7ebac

  Copy full SHA for fb7ebac

• ConstantPredictor uses the maximum value

  Show description for b4bd854
  Lehmann-Fabian

  committed
  b4bd854

  Copy full SHA for b4bd854

• Error instead of fallback to none

  Show description for 4fe6f5a
  Lehmann-Fabian

  committed
  4fe6f5a

  Copy full SHA for 4fe6f5a

Commits on Mar 5, 2024

• No need to enable tracing manually

  Show description for 9640125
  Lehmann-Fabian

  committed
  9640125

  Copy full SHA for 9640125

• NonePredictor and Statistics are now covered by Nextflow traces.

  Show description for 9c3d1dd
  Lehmann-Fabian

  committed
  9c3d1dd

  Copy full SHA for 9c3d1dd

• Remove remaining statistic references

  Show description for afa5b48
  Lehmann-Fabian

  committed
  afa5b48

  Copy full SHA for afa5b48

• added method to get exitCode of task

  Show description for 18c45ba
  Lehmann-Fabian

  committed
  18c45ba

  Copy full SHA for 18c45ba

• Make getExitCode public

  Show description for 9274f12
  Lehmann-Fabian

  committed
  9274f12

  Copy full SHA for 9274f12

Commits on Mar 7, 2024

• Added endpoint to receive task metrics

  Show description for 385df66
  Lehmann-Fabian

  committed
  385df66

  Copy full SHA for 385df66

• add TaskMetrics to Task

  Show description for c253c90
  Lehmann-Fabian

  committed
  c253c90

  Copy full SHA for c253c90

• Refactor MemoryPrediction

  Show description for a26ded3
  Lehmann-Fabian

  committed
  a26ded3

  Copy full SHA for a26ded3

• Add tests

  Show description for 1a77e98
  Lehmann-Fabian

  committed
  1a77e98

  Copy full SHA for 1a77e98

• No error for normal behaviour

  Show description for 7b8e261
  Lehmann-Fabian

  committed
  7b8e261

  Copy full SHA for 7b8e261

• fix task was finished in wrong place

  Show description for b598e1a
  Lehmann-Fabian

  committed
  b598e1a

  Copy full SHA for b598e1a

• Improve logging. Format bytes as GB, MB, ...

  Show description for ceae8d5
  Lehmann-Fabian

  committed
  ceae8d5

  Copy full SHA for ceae8d5

• Add validity check

  Show description for 08e2898
  Lehmann-Fabian

  committed
  08e2898

  Copy full SHA for 08e2898

• Store old memory values and set new directly

  Show description for 7285259
  Lehmann-Fabian

  committed
  7285259

  Copy full SHA for 7285259

• Fix tests

  Show description for 0c730f6
  Lehmann-Fabian

  committed
  0c730f6

  Copy full SHA for 0c730f6

Commits on Mar 11, 2024

• Remove copyTasks

  Show description for 8d02fd3
  Lehmann-Fabian

  committed
  8d02fd3

  Copy full SHA for 8d02fd3

• Store a task's request in a variable and not in the pod.

  Show description for 8fb6268
  Lehmann-Fabian

  committed
  8fb6268

  Copy full SHA for 8fb6268

• Patch pods if requirements changed

  Show description for 0aad2f6
  Lehmann-Fabian

  committed
  0aad2f6

  Copy full SHA for 0aad2f6

• Changed log level in patchTask

  Show description for c81041e
  Lehmann-Fabian

  committed
  c81041e

  Copy full SHA for c81041e

Commits on Mar 12, 2024

• Check if featureGate InPlaceVerticalScaling is active

  Show description for 798cf7a
  Lehmann-Fabian

  committed
  798cf7a

  Copy full SHA for 798cf7a

• Use fabric capabilities to patch tasks

  Show description for 5c12e95
  Lehmann-Fabian

  committed
  5c12e95

  Copy full SHA for 5c12e95

• Scale Task to full Mb

  Show description for 4c89408
  Lehmann-Fabian

  committed
  4c89408

  Copy full SHA for 4c89408

• Update tests

  Show description for 5a454d9
  Lehmann-Fabian

  committed
  5a454d9

  Copy full SHA for 5a454d9

Commits on Mar 13, 2024

• Use version to avoid prediction if possible

  Show description for 6ff81a3
  Lehmann-Fabian

  committed
  6ff81a3

  Copy full SHA for 6ff81a3

Commits on Mar 14, 2024

• Fix that always the memoryVersion was used

  Show description for 2ec2fec
  Lehmann-Fabian

  committed
  2ec2fec

  Copy full SHA for 2ec2fec

Commits on Mar 15, 2024

• Consider memoryscaler params

  Show description for 755ce88
  Lehmann-Fabian

  committed
  755ce88

  Copy full SHA for 755ce88

• Polynomial Predictor

  Show description for 4f27c6b
  Lehmann-Fabian

  committed
  4f27c6b

  Copy full SHA for 4f27c6b

• Pondering Predictor

  Show description for 67cf059
  Lehmann-Fabian

  committed
  67cf059

  Copy full SHA for 67cf059

• Fix problems with params parsing

  Show description for fc8a677
  Lehmann-Fabian

  committed
  fc8a677

  Copy full SHA for fc8a677

• Remove unnecessary logging.

  Show description for ca30338
  Lehmann-Fabian

  committed
  ca30338

  Copy full SHA for ca30338

• Optimize Imports

  Show description for 32a668b
  Lehmann-Fabian

  committed
  32a668b

  Copy full SHA for 32a668b

• Update versions

  Show description for b703bca
  Lehmann-Fabian

  committed
  b703bca

  Copy full SHA for b703bca

• Log human readable

  Show description for 92896e7
  Lehmann-Fabian

  committed
  92896e7

  Copy full SHA for 92896e7

Commits on Mar 20, 2024

• Check for null tasks

  Show description for 56a5094
  Lehmann-Fabian

  committed
  56a5094

  Copy full SHA for 56a5094

• Move code to create an array of observations into the OffsetApplier

  Show description for e3a7a11
  Lehmann-Fabian

  committed
  e3a7a11

  Copy full SHA for e3a7a11

• Check for SingularMatrix problems

  Show description for cc70571
  Lehmann-Fabian

  committed
  cc70571

  Copy full SHA for cc70571

• Add VarianceOffset

  Show description for d5dedfd
  Lehmann-Fabian

  committed
  d5dedfd

  Copy full SHA for d5dedfd

• Added StandardDeviation Offset

  Show description for daa51c9
  Lehmann-Fabian

  committed
  daa51c9

  Copy full SHA for daa51c9

• registered new Offsets and predictors

  Show description for 37e4fa6
  Lehmann-Fabian

  committed
  37e4fa6

  Copy full SHA for 37e4fa6

• Receive input_size

  Show description for eb64c83
  Lehmann-Fabian

  committed
  eb64c83

  Copy full SHA for eb64c83

Commits on Apr 10, 2024

• added constructor

  Show description for d4f8382
  Lehmann-Fabian

  committed
  d4f8382

  Copy full SHA for d4f8382

• Localize Formater

  Show description for d718fd3
  Lehmann-Fabian

  committed
  d718fd3

  Copy full SHA for d718fd3

• assert to if else

  Show description for 23cfc8e
  Lehmann-Fabian

  committed
  23cfc8e

  Copy full SHA for 23cfc8e

• change log level

  Show description for 764c8c5
  Lehmann-Fabian

  committed
  764c8c5

  Copy full SHA for 764c8c5

• allow for factor using StandardDeviationOffset

  Show description for 62db892
  Lehmann-Fabian

  committed
  62db892

  Copy full SHA for 62db892

• Refactor OffsetApplier

  Show description for 4919d24
  Lehmann-Fabian

  committed
  4919d24

  Copy full SHA for 4919d24

• Added MeanPredictor

  Show description for 2bfcc89
  Lehmann-Fabian

  committed
  2bfcc89

  Copy full SHA for 2bfcc89

• change log level

  Show description for 8b4367d
  Lehmann-Fabian

  committed
  8b4367d

  Copy full SHA for 8b4367d

• introduce linear regression with unequal loss

  Show description for 6b96b17
  Lehmann-Fabian

  committed
  6b96b17

  Copy full SHA for 6b96b17

• Added new ponder predictor

  Show description for 388a700
  Lehmann-Fabian

  committed
  388a700

  Copy full SHA for 388a700

Commits on Apr 25, 2024

• Set a label if memory was adapted

  Show description for 3973477
  Lehmann-Fabian

  committed
  3973477

  Copy full SHA for 3973477

Commits on May 3, 2024

• implemented leastfinishedfirst

  Show description for a077257
  Lehmann-Fabian

  committed
  a077257

  Copy full SHA for a077257

• Added tests for prioritizing strategies

  Show description for 8dfcec8
  Lehmann-Fabian

  committed
  8dfcec8

  Copy full SHA for 8dfcec8

• allow to update nodes

  Show description for bcff656
  Lehmann-Fabian

  committed
  bcff656

  Copy full SHA for bcff656

• TestProcess for prioritization tests

  Show description for 5e32833
  Lehmann-Fabian

  committed
  5e32833

  Copy full SHA for 5e32833

• Update dependencies

  Show description for 8ebdc2c
  Lehmann-Fabian

  committed
  8ebdc2c

  Copy full SHA for 8ebdc2c

• TestTask for prioritization tests

  Show description for 971266a
  Lehmann-Fabian

  committed
  971266a

  Copy full SHA for 971266a

Commits on May 11, 2024

• lff prioritize smaller tasks

  Show description for 70c46d7
  Lehmann-Fabian

  committed
  70c46d7

  Copy full SHA for 70c46d7

Commits on May 31, 2024

• Add new prioritization strategies

  Show description for 8ac0c1e
  Lehmann-Fabian

  committed
  8ac0c1e

  Copy full SHA for 8ac0c1e

• Check edge cases

  Show description for 34da84a
  Lehmann-Fabian

  committed
  34da84a

  Copy full SHA for 34da84a

• add getIndependetValue to Predictor interface

  Show description for ed09aa8
  Lehmann-Fabian

  committed
  ed09aa8

  Copy full SHA for ed09aa8

• remove getIndependetValue from LinearPredictor

  Show description for ebc3477
  Lehmann-Fabian

  committed
  ebc3477

  Copy full SHA for ebc3477

• Added lambda to UnequalLossFunction

  Show description for 6c167c2
  Lehmann-Fabian

  committed
  6c167c2

  Copy full SHA for 6c167c2

• also allow short form for linear regression

  Show description for fc919c2
  Lehmann-Fabian

  committed
  fc919c2

  Copy full SHA for fc919c2

• Add weighted PonderingPredictor

  Show description for cc5e634
  Lehmann-Fabian

  committed
  cc5e634

  Copy full SHA for cc5e634

• Restructure

  Show description for 9ca798f
  Lehmann-Fabian

  committed
  9ca798f

  Copy full SHA for 9ca798f

Commits on Jul 19, 2024

• Update dependencies

  Show description for e0c05cb
  Lehmann-Fabian

  committed
  e0c05cb

  Copy full SHA for e0c05cb

• Remove strategies from tests

  Show description for 125dc80
  Lehmann-Fabian

  committed
  125dc80

  Copy full SHA for 125dc80

• Unify naming

  Show description for a1a73eb
  Lehmann-Fabian

  committed
  a1a73eb

  Copy full SHA for a1a73eb

• Improve code quality

  Show description for 08b1680
  Lehmann-Fabian

  committed
  08b1680

  Copy full SHA for 08b1680

• Unify naming

  Show description for d76eba2
  Lehmann-Fabian

  committed
  d76eba2

  Copy full SHA for d76eba2

• Update readme and add scheduling and node assignment strategies + memory prediction and offset strategies

  Show description for 8c1f783
  Lehmann-Fabian

  committed
  8c1f783

  Copy full SHA for 8c1f783

• Merge branch 'master' into memoryPredictionChanges

  Show description for 71f7aab
  Lehmann-Fabian

  committed
  71f7aab

  Copy full SHA for 71f7aab