generated from mlcommons/mlperf_inference_submissions
-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
d39c075
commit 056eef2
Showing
16 changed files
with
435 additions
and
442 deletions.
There are no files selected for viewing
2 changes: 1 addition & 1 deletion
2
...ty/measurements/default-reference-cpu-tvm-onnx_v1.19.2-default_config/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| | Model | Scenario | Accuracy | Throughput | Latency (in ms) | | ||
| |----------|------------|------------|--------------|-------------------| | ||
| | resnet50 | offline | 80 | 1.754 | - | | ||
| | resnet50 | offline | 80 | 1.722 | - | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 changes: 7 additions & 7 deletions
14
...fault-reference-cpu-tvm-onnx_v1.19.2-default_config/resnet50/offline/accuracy_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,10 @@ | ||
| python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_655a61cb --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_e53cfe20/test_results/default-reference-cpu-tvm-onnx-v1.19.2-default_config/resnet50/offline/accuracy" --backend tvm --scenario Offline --max-batchsize 1 --count 5 --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8e0f5b4d99784826930b71265e475d72.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_655a61cb --dataset-list /home/runner/MLC/repos/local/cache/extract-file_29278e33/val.txt | ||
| INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_655a61cb', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_29278e33/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_e53cfe20/test_results/default-reference-cpu-tvm-onnx-v1.19.2-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='tvm', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_655a61cb', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8e0f5b4d99784826930b71265e475d72.conf', audit_conf='audit.config', time=None, count=5, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
| python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_bb0ec862 --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_fe2b85b8/test_results/default-reference-cpu-tvm-onnx-v1.19.2-default_config/resnet50/offline/accuracy" --backend tvm --scenario Offline --max-batchsize 1 --count 5 --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/11bd3ab868b74c76997e018f16b767bc.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_bb0ec862 --dataset-list /home/runner/MLC/repos/local/cache/extract-file_06c50cce/val.txt | ||
| INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_bb0ec862', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_06c50cce/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=1, model='/home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_fe2b85b8/test_results/default-reference-cpu-tvm-onnx-v1.19.2-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='tvm', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_bb0ec862', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/11bd3ab868b74c76997e018f16b767bc.conf', audit_conf='audit.config', time=None, count=5, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
| INFO:imagenet:Loading 5 preprocessed images using 4 threads | ||
| INFO:imagenet:loaded 5 images, cache=0, already_preprocessed=True, took=0.0sec | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/model-tvm.so | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/model-tvm.so | ||
| INFO:main:starting TestScenario.Offline | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/model-tvm.so | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/model-tvm.so | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_69fd5eb5/model-tvm.so | ||
| TestScenario.Offline qps=1.74, mean=1.7642, time=2.879, acc=80.000%, queries=5, tiles=50.0:1.5012,80.0:1.7884,90.0:2.3199,95.0:2.5856,99.0:2.7982,99.9:2.8460 | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/model-tvm.so | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/model-tvm.so | ||
| TVM: loading model /home/runner/MLC/repos/local/cache/get-tvm-model_0838619d/model-tvm.so | ||
| TestScenario.Offline qps=1.72, mean=1.7538, time=2.908, acc=80.000%, queries=5, tiles=50.0:1.4796,80.0:1.7690,90.0:2.3221,95.0:2.5986,99.0:2.8198,99.9:2.8696 |
Oops, something went wrong.