[Release] OpenVINO for BI

What is the recommended method to handle a system restart. I'd like this server to start without requiring a login.
 
My stats so far on i5-8500 with GPU enabled:
API Requests: 1248 total requests
Average Response Time ():97.7milliseconds
Last Response Time: 76.9 milliseconds
Frames Per Second (FPS): 10.2
 
Last edited:
  • Love
Reactions: jrbeddow
I5 6500t igpu with a medium yolov11 model is average 270ms.
The times are on par with a gtx1660 (AI server was hosted on another pc) running blue onyx with the largest ONNX model.
However I find the accuracy significantly better. No false positives in over 3 days of running (>10000 requests)

Gpu benchmark:
L: 170.52 ms (5.9 FPS)
T: 7.1 FPS (req 4)
 
Last edited:
  • Like
Reactions: jrbeddow
Also started working on openvino integration, but differently: based on openvino/model_server container images, using the latest models from ultralytics (26). It will allow spreading the load over multiple servers/devices (Jetson Nano should in theory work as well), and will allow deploying to the OS of your liking. Will update here if I get it working properly.
Preliminary results show that inference with yolo26n is about 20% faster than yolo11n (on CPU, without any GPU).
 
Last edited:
You must log in or register to reply here.
Top Bottom