PaddleOCR and NPU

[aropb]

Hi,

I have an Intel Core Ultra 9 Processor with NPU.
It was not possible to start recognition for the NPU. Everything works for the CPU.

This my code:

        string ocrPath = publishDirectory + "/wwwroot/ocr/";
        var options = new Sdcb.OpenVINO.DeviceOptions("NPU", new Dictionary<string, string>
        {
            { Sdcb.OpenVINO.PropertyKeys.HintPerformanceMode, "THROUGHPUT" },
            { Sdcb.OpenVINO.PropertyKeys.HintInferencePrecision, "f16" },
            { Sdcb.OpenVINO.PropertyKeys.HintEnableCpuPinning, "NO" },
            { Sdcb.OpenVINO.PropertyKeys.NumStreams, "1" },
            { Sdcb.OpenVINO.PropertyKeys.EnableProfiling, "NO" }
        });

        OcrEng = new PaddleOcrRecognizer(RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4), options, 192); // or dynamic shapes

I get an error when executing:

Exception "Sdcb.OpenVINO.OpenVINOException" in Sdcb.OpenVINO.dll: 'general error(-1) Exception from src\inference\src\cpp\core.cpp:109:
Exception from src\inference\src\dev\plugin.cpp:54:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:513:
get_shape was called on a descriptor::Tensor with dynamic shape

OpenVINO NPU documentations:
https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html

Limitations
Currently, only models with static shapes are supported on NPU.

Please tell me what to do?
Thanks.

[sdcb]

You can specify static shapes by this code:

OpenVINO.NET/projects/PaddleOCR/Sdcb.OpenVINO.PaddleOCR/PaddleOcrOptions.cs

Lines 27 to 28 in 35344c0

	DetectionStaticSize = new Size(1024, 1024);
	RecognitionStaticWidth = 512;

[aropb]

        string ocrPath = publishDirectory + "/wwwroot/ocr/";

        Sdcb.OpenVINO.DeviceOptions deviceOptions = new("NPU", new Dictionary<string, string>
        {
            { Sdcb.OpenVINO.PropertyKeys.HintPerformanceMode, "THROUGHPUT" },
            { Sdcb.OpenVINO.PropertyKeys.HintInferencePrecision, "f16" }, // i8 f16
            { Sdcb.OpenVINO.PropertyKeys.NumStreams, "1" },
            { Sdcb.OpenVINO.PropertyKeys.EnableProfiling, "NO" }
        });

        PaddleOcrOptions ocrOptions = new(deviceOptions)
        {
            DetectionStaticSize = new Size(480, 64),
            RecognitionStaticWidth = 480
        };

        OcrEng = new PaddleOcrAll(
            new FullOcrModel(
                DetectionModel.FromDirectory(ocrPath + "english", ModelVersion.V4), 
                RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4)
            ),
            ocrOptions);

Error:
Exception from src\inference\src\dev\plugin.cpp:54:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:513:
get_shape was called on a descriptor::Tensor with dynamic shape

What am I doing wrong?

Besides, I would like to use only recognition:
OcrEng = new PaddleOcrRecognizer(RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4), options);

Thanks.