Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error creating a chart with fio-plot: Error: KeyError('iodepth') #149

Open
j4care-christianhahn opened this issue Sep 19, 2024 · 3 comments
Open

Error creating a chart with fio-plot: Error: KeyError('iodepth') #149

j4care-christianhahn opened this issue Sep 19, 2024 · 3 comments

Comments

@j4care-christianhahn
Copy link

Hi!

I'm trying to use fio 3.37 to do some iops test like this:
fio --name=test --rw=randread --bs=4k --size=1G --numjobs=1 --runtime=60 --output=fio_output.json --output-format=json

Afterwards I'm trying to use fio-plot to create a graph:
fio-plot --source "https://test.com" -T "TEST" -l -r randread -i .
Filename: /home/test/fio_output.json
Error: KeyError('iodepth')

First, open the file and check for errors at the top.
You can remove the error lines and the JSON will likely parse
but results may not be trustworthy.
If there are no error linkes at the top, please report this as a bug
and please include the JSON file if possible.

does anyone what could be wrong here ?
@louwrentius
Copy link
Owner

Hello, can you share your .json file here as attachment or by mail if you want to keep it private louwrentius gmail com.

@robinpecha
Copy link

Im also not able to run fio-plot with any fio configuration.
The command above do this result on my pc:

fio-plot --source "https://test.com/" -T "TEST" -l -r randread -i .

Filename: /srv/disk/fio_output.json
Error: KeyError('iodepth')

First, open the file and check for errors at the top.
You can remove the error lines and the JSON will likely parse
but results may not be trustworthy.
If there are no error linkes at the top, please report this as a bug
and please include the JSON file if possible.

fio --name=test --rw=randread --bs=4k --size=1G --numjobs=1 --runtime=60 --output=fio_output.json --output-format=json

{
  "fio version" : "fio-3.28",
  "timestamp" : 1727950008,
  "timestamp_ms" : 1727950008453,
  "time" : "Thu Oct  3 12:06:48 2024",
  "jobs" : [
    {
      "jobname" : "test",
      "groupid" : 0,
      "error" : 0,
      "eta" : 0,
      "elapsed" : 20,
      "job options" : {
        "name" : "test",
        "rw" : "randread",
        "bs" : "4k",
        "size" : "1G",
        "numjobs" : "1",
        "runtime" : "60"
      },
      "read" : {
        "io_bytes" : 1073741824,
        "io_kbytes" : 1048576,
        "bw_bytes" : 56114022,
        "bw" : 54798,
        "iops" : 13699.712569,
        "runtime" : 19135,
        "total_ios" : 262144,
        "short_ios" : 0,
        "drop_ios" : 0,
        "slat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "clat_ns" : {
          "min" : 29780,
          "max" : 4147980,
          "mean" : 72177.667210,
          "stddev" : 27659.471549,
          "N" : 262144,
          "percentile" : {
            "1.000000" : 63744,
            "5.000000" : 66048,
            "10.000000" : 67072,
            "20.000000" : 68096,
            "30.000000" : 68096,
            "40.000000" : 69120,
            "50.000000" : 70144,
            "60.000000" : 71168,
            "70.000000" : 72192,
            "80.000000" : 74240,
            "90.000000" : 77312,
            "95.000000" : 83456,
            "99.000000" : 98816,
            "99.500000" : 109056,
            "99.900000" : 244736,
            "99.950000" : 411648,
            "99.990000" : 937984
          }
        },
        "lat_ns" : {
          "min" : 29850,
          "max" : 4148520,
          "mean" : 72263.243042,
          "stddev" : 27671.608828,
          "N" : 262144
        },
        "bw_min" : 51856,
        "bw_max" : 56656,
        "bw_agg" : 100.000000,
        "bw_mean" : 54856.631579,
        "bw_dev" : 1156.204612,
        "bw_samples" : 38,
        "iops_min" : 12964,
        "iops_max" : 14164,
        "iops_mean" : 13714.157895,
        "iops_stddev" : 289.051153,
        "iops_samples" : 38
      },
      "write" : {
        "io_bytes" : 0,
        "io_kbytes" : 0,
        "bw_bytes" : 0,
        "bw" : 0,
        "iops" : 0.000000,
        "runtime" : 0,
        "total_ios" : 0,
        "short_ios" : 0,
        "drop_ios" : 0,
        "slat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "clat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "lat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "bw_min" : 0,
        "bw_max" : 0,
        "bw_agg" : 0.000000,
        "bw_mean" : 0.000000,
        "bw_dev" : 0.000000,
        "bw_samples" : 0,
        "iops_min" : 0,
        "iops_max" : 0,
        "iops_mean" : 0.000000,
        "iops_stddev" : 0.000000,
        "iops_samples" : 0
      },
      "trim" : {
        "io_bytes" : 0,
        "io_kbytes" : 0,
        "bw_bytes" : 0,
        "bw" : 0,
        "iops" : 0.000000,
        "runtime" : 0,
        "total_ios" : 0,
        "short_ios" : 0,
        "drop_ios" : 0,
        "slat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "clat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "lat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        },
        "bw_min" : 0,
        "bw_max" : 0,
        "bw_agg" : 0.000000,
        "bw_mean" : 0.000000,
        "bw_dev" : 0.000000,
        "bw_samples" : 0,
        "iops_min" : 0,
        "iops_max" : 0,
        "iops_mean" : 0.000000,
        "iops_stddev" : 0.000000,
        "iops_samples" : 0
      },
      "sync" : {
        "total_ios" : 0,
        "lat_ns" : {
          "min" : 0,
          "max" : 0,
          "mean" : 0.000000,
          "stddev" : 0.000000,
          "N" : 0
        }
      },
      "job_runtime" : 19134,
      "usr_cpu" : 2.195045,
      "sys_cpu" : 16.243336,
      "ctx" : 262180,
      "majf" : 0,
      "minf" : 11,
      "iodepth_level" : {
        "1" : 100.000000,
        "2" : 0.000000,
        "4" : 0.000000,
        "8" : 0.000000,
        "16" : 0.000000,
        "32" : 0.000000,
        ">=64" : 0.000000
      },
      "iodepth_submit" : {
        "0" : 0.000000,
        "4" : 100.000000,
        "8" : 0.000000,
        "16" : 0.000000,
        "32" : 0.000000,
        "64" : 0.000000,
        ">=64" : 0.000000
      },
      "iodepth_complete" : {
        "0" : 0.000000,
        "4" : 100.000000,
        "8" : 0.000000,
        "16" : 0.000000,
        "32" : 0.000000,
        "64" : 0.000000,
        ">=64" : 0.000000
      },
      "latency_ns" : {
        "2" : 0.000000,
        "4" : 0.000000,
        "10" : 0.000000,
        "20" : 0.000000,
        "50" : 0.000000,
        "100" : 0.000000,
        "250" : 0.000000,
        "500" : 0.000000,
        "750" : 0.000000,
        "1000" : 0.000000
      },
      "latency_us" : {
        "2" : 0.000000,
        "4" : 0.000000,
        "10" : 0.000000,
        "20" : 0.000000,
        "50" : 0.053406,
        "100" : 99.084854,
        "250" : 0.764465,
        "500" : 0.070572,
        "750" : 0.013733,
        "1000" : 0.010000
      },
      "latency_ms" : {
        "2" : 0.010000,
        "4" : 0.010000,
        "10" : 0.010000,
        "20" : 0.000000,
        "50" : 0.000000,
        "100" : 0.000000,
        "250" : 0.000000,
        "500" : 0.000000,
        "750" : 0.000000,
        "1000" : 0.000000,
        "2000" : 0.000000,
        ">=2000" : 0.000000
      },
      "latency_depth" : 1,
      "latency_target" : 0,
      "latency_percentile" : 100.000000,
      "latency_window" : 0
    }
  ],
  "disk_util" : [
    {
      "name" : "nvme0n1",
      "read_ios" : 261958,
      "write_ios" : 784,
      "read_merges" : 0,
      "write_merges" : 583,
      "read_ticks" : 15902,
      "write_ticks" : 187,
      "in_queue" : 16122,
      "util" : 84.268083
    }
  ]
}

@louwrentius
Copy link
Owner

@robinpecha Sorry for the late reply, maybe you already resolved your issue.

If not, your error is quite visible: the 'iodepth' key is missing in the JSON in the 'job options' dictionary because it was not provided as a parameter to the 'raw' fio-plot command.

You may want to try bench-fio (included) to run one or more benchmarks, which should assure correct JSON output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@louwrentius @robinpecha @j4care-christianhahn