Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[onert] Fix accuracy measurement #14523

Open
ragmani opened this issue Jan 3, 2025 · 0 comments
Open

[onert] Fix accuracy measurement #14523

ragmani opened this issue Jan 3, 2025 · 0 comments

Comments

@ragmani
Copy link
Contributor

ragmani commented Jan 3, 2025

What

I found that the accuracy measurement was not working, and I checked that obtaining output results during training does not works.

$ python3 runtime/onert/sample/minimal-python/experimental/src/train_with_dataset.py -m mobilenetv2 -i out/imagenet_a.test.input.100.bin -l out/imagenet_a.test.output.100.bin --data_length 100
Load data
Epoch 1/5
Batch 1: Loss=0.0012
Batch 2: Loss=0.0012
Batch 3: Loss=0.0011
Batch 4: Loss=0.0012
Batch 5: Loss=0.0011
Train Loss: 0.0012
Batch 1: Loss=0.0012
Batch 2: Loss=0.0013
Validation Loss: 0.0012
CategoricalAccuracy: 0.0000
$ ./Product/x86_64-linux.release/out/bin/onert_train mobilenetv2.circle --load_expected:raw test-models/imagenet_a/test.output.100.bin --load_input:raw test-models/imagenet_a/test.input.100.bin --loss 1 --loss_reduction_type 1 --optimizer 1 --learning_rate 0.01 --epoch 5 --batch_size 10 --num_of_trainable_ops -1 --validation_split 0.2 --metric 0
Model Filename mobilenetv2.circle
== training parameter ==
- learning_rate        = 0.01
- batch_size           = 10
- loss_info            = {loss = mean squared error, reduction = sum over batch size}
- optimizer            = sgd
- num_of_trainable_ops = -1
========================
Epoch 1/5 - time: 769.609ms/step - loss: [0] 0.0012 - categorical_accuracy: [0] 0.0000 - val_loss: [0] 0.0012 - val_categorical_accuracy: [0] 0.0000
Epoch 2/5 - time: 795.052ms/step - loss: [0] 0.0012 - categorical_accuracy: [0] 0.0000 - val_loss: [0] 0.0012 - val_categorical_accuracy: [0] 0.0000
Epoch 3/5 - time: 724.606ms/step - loss: [0] 0.0011 - categorical_accuracy: [0] 0.0000 - val_loss: [0] 0.0012 - val_categorical_accuracy: [0] 0.0000
Epoch 4/5 - time: 712.029ms/step - loss: [0] 0.0011 - categorical_accuracy: [0] 0.0000 - val_loss: [0] 0.0012 - val_categorical_accuracy: [0] 0.0000
Epoch 5/5 - time: 721.155ms/step - loss: [0] 0.0011 - categorical_accuracy: [0] 0.0000 - val_loss: [0] 0.0012 - val_categorical_accuracy: [0] 0.0000
===================================
MODEL_LOAD   takes 7.5380 ms
PREPARE      takes 268.6270 ms
EXECUTE      takes 30332.8390 ms
- Epoch 1      takes 6156.8730 ms
- Epoch 2      takes 6360.4160 ms
- Epoch 3      takes 5796.8470 ms
- Epoch 4      takes 5696.2360 ms
- Epoch 5      takes 5769.2410 ms
===================================

We need to fix the problem.

Originally posted by @ragmani in #14505 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant