Blue Iris "FPS / key rate" error and no signal

TuckNTruck

Pulling my weight
Aug 16, 2022
88
139
NC
I have a Tyco / Illustra PTZ that I believe is a rebranded Hikvision DS-2DF8236I5X-AEL(W).

There's something going on with it and Blue Iris. Most of the time, no issues, but then the stream will slow down and I get the error "Check FPS and key rates".

Sometimes the error goes away and the stream "catches up" and works fine again until it happens again. Other times, it goes to a "No Signal" after being on the "Check FPS and key rates" error.

It's like there's a backlog of frames that are trying to get through but can't, and then they all come through at once before resuming normal operation. Which just reminded me, I have seen a message that says "frame bottleneck" as well, but it's very rare.

There have been times where it never recovers from "no signal" and I have to reset the switch, again it feels like to clear the backlog of frames that aren't getting through.

When all this happens, the camera's web interface is perfectly fine. I never have problems pinging the camera or viewing the live feed in the camera's interface. The problems seems to be with blue iris.

In the configuration, I have constant bitrate, H264, 30 FPS, I frame interval 30. I've tried 25 FPS along with bitrates as low as 6144 with no change in behavior.

In BI, I have the max rate set to 30.00 fps, no decoding, continuous recording BVR direct to disc. I don't use substreams at all with this camera.
 
Not using substreams could mean you are overloading the system and it cant keep up and that is why resetting or restarting the switch works because it is clearing out all of the lost packets sent that start piling up because it keeps trying to resend the packets.

Watching live view from camera interface doesnt say much unless you are watching it 24/7. Opening it when it shows no video in BI isn't proving anything other than the camera is working, which leads to lost packets being sent to BI.

Why not use substreams or drop FPS to 15?
 
+1^^^.
If no joy from the above, try UNchecking "Send RTSP keep-alives" and bump up "Receive Buffer" to 20MB (both settings under "camera settings" => "Video" tab).
 
  • Like
Reactions: Flintstone61
Not using substreams could mean you are overloading the system and it cant keep up and that is why resetting or restarting the switch works because it is clearing out all of the lost packets sent that start piling up because it keeps trying to resend the packets.

Watching live view from camera interface doesnt say much unless you are watching it 24/7. Opening it when it shows no video in BI isn't proving anything other than the camera is working, which leads to lost packets being sent to BI.

I monitor this camera most of the day while I'm working, so I like to see full quality. Nothing should be bandwidth limited and CPU usage on my BI PC is 10% and no storage issues either. When you say I could be overloading the system and it can't keep up, what system? The PC is hardly breaking a sweat, and everything in the network is gigabit or better, more details on that below.

Why not use substreams or drop FPS to 15?
I can manage with 25 FPS but the camera can do 30 FPS so I like to get everything I can out of it. All the components should be more than capable, so I don't understand what's up with this one camera and BI.

+1^^^.
If no joy from the above, try UNchecking "Send RTSP keep-alives" and bump up "Receive Buffer" to 20MB (both settings under "camera settings" => "Video" tab).
I'll try that on the RTSP keep-alive. First thing I did was bump up the receive buffer when I set up this camera. I've tried multiple settings between 20-50, no change.

what is the actual hardware your pushing data thru? like PC, Switch, Hard disk setup,
PC is a Dell Optiplex 5060, 32 GB ram, I5-8700, not running AI or any of that stuff, CPU usage stays around 10% and the PC is not used for anything other than BI.
Here's a ChatGPT generated network map. Everything is gigabit or better, quality CAT6 ethernet cables, everything in operation for years, but the PTZ in question is new.

The part I'm hung up on is that if I watch the PTZ's web interface, the livestream on that never slows down, glitches, drops out, etc. It's perfect. But the stream in BI has problems. There's usually 100-120 mbps of camera traffic flowing through everything, so nowhere near the gigabit everything is rated for. But again, BI is the only part of the setup that is having problems, and just with this one camera despite the feed being perfect in the camera's web interface.

Network.jpg
 
Who makes the switch that when you restart all is good?

So your cameras go thru the router to the BI computer? That alone can be the reason for the resend packets request and your problems. Removing the router from the equation solves lots of these issues. These routers are just not designed for the full-throttle never ease up demands of cameras. Unlike Netflix, these things never buffer and that causes issues with lost packets.

Under Task Manager, what does the performance of the NIC card look like? It appears you are only one ethernet port on the BI computer? If so you have all the cameras blasting in on mainstream and viewing out on the same NIC and you could be overloading it.

For example, On my isolated camera NIC, my cameras are streaming non-stop 350Mbps. This is full-on, never stopping to take a breath. Even if someone has a gigabit router, a 3rd of non-buffering 24/7 data will impact its speed.

If I didn't have dual NIC, then I could potentially be having 350Mbps incoming and watching multi-camera on mainstream at 350Mbps, so I am now pushing 700Mbps thru a gigabit ethernet port and other devices. And most GB routers and devices cannot push that through. For example, the EdgeRouter X is claimed to be 1Gbps, but you see tests all over where people are only getting in the 700Mbps range.

Are you not using substreams at all? If not, do you know that by using substreams, when you solo the camera it goes to mainstream?

And since it is just the PTZ, it could be any of the devices it is going thru.

Also, keep in mind just because the camera is rated at 30FPS doesn't mean it can actually do it with other things in operation.

Of course, some cameras are better than others, but many have just barely enough processor power and can overload easily.
 
I'm thinkin the software (Blue Iris) is only as fast as your hardware.
what hardware( i.e. WD purple(s) are all these cam streams being written to?
are all your cams running high frame rates? at 30 FPS ?
i5 is an 8500, 6 core 6 thread processor. You should be good with either processor.
i7 is an 8700 6 core 12 thread processor. ( which I'm running as well)
I would do a trial of 15-18 FPS and see if the problem continues or goes away.
then you might get more data to work with. like is it a limitation of the cam? or the Wireless lan? or ?
 
Who makes the switch that when you restart all is good?

So your cameras go thru the router to the BI computer? That alone can be the reason for the resend packets request and your problems. Removing the router from the equation solves lots of these issues. These routers are just not designed for the full-throttle never ease up demands of cameras. Unlike Netflix, these things never buffer and that causes issues with lost packets.

Under Task Manager, what does the performance of the NIC card look like? It appears you are only one ethernet port on the BI computer? If so you have all the cameras blasting in on mainstream and viewing out on the same NIC and you could be overloading it.

For example, On my isolated camera NIC, my cameras are streaming non-stop 350Mbps. This is full-on, never stopping to take a breath. Even if someone has a gigabit router, a 3rd of non-buffering 24/7 data will impact its speed.

If I didn't have dual NIC, then I could potentially be having 350Mbps incoming and watching multi-camera on mainstream at 350Mbps, so I am now pushing 700Mbps thru a gigabit ethernet port and other devices. And most GB routers and devices cannot push that through. For example, the EdgeRouter X is claimed to be 1Gbps, but you see tests all over where people are only getting in the 700Mbps range.

Are you not using substreams at all? If not, do you know that by using substreams, when you solo the camera it goes to mainstream?

And since it is just the PTZ, it could be any of the devices it is going thru.

Also, keep in mind just because the camera is rated at 30FPS doesn't mean it can actually do it with other things in operation.

Of course, some cameras are better than others, but many have just barely enough processor power and can overload easily.
All my switches are Mokerlink - yes, cheap amazon brand, but they've been flawless and I have several of them. I have the one on the boat dock plugged into a cloud power switch and that's how I cycle the power, so it actually cycles the power on that switch but that switch going down also cuts the data link to the tree switch, POE++ injector, and PTZ, so not sure what the actual "fix" is, but it works every time it goes to "no signal".

I'm confused about the question about the cameras going thru the router to the BI computer. Don't they have to? The router sits next to the BI PC and is connected to it via ethernet LAN1. Then I run ethernet from LAN2 to the outdoor switch. After that it goes dock switch, the tree switch, and finally the POE++ injector all downstream from the router. Those are all daisy chained together, starting at the router it goes via ethernet to outdoor switch, then 60ghz bridge to dock switch, then fiber from dock switch to tree switch, then tree switch to POE++ injector via ethernet patch cable, then from injector up to the PTZ.

If we're narrowing it down to a network issue, seems like the POE++ would be the first possible culprit, correct? The POE++ injector is connected to the tree switch, and the single camera on that switch works fine, so maybe the cheap POE++ can't handle the throughput even though it's only handling the data from a single PTZ?

Task manager on the BI PC shows 115 mbps receive and 2-3 mbps send. When the PTZ goes to no signal, it drops approx 12 mbps, which is the continuous bitrate I have it set to right now. BTW, I've adjusted the FPS, but bitrate stays the same - it's still pushing 12 mpbs whether it's 30, 20, or 10 FPS. Makes sense to me, is that the way it should be? If I'm trying to figure out a "too much data" problem, I need to drop bitrate, right? I've gone all the way down to 6 mbps CBR and no change.

I do use substreams on all but a couple cameras.

Anyways, I did have trouble when I first installed this camera as it would power cycle when the IR was on and pan/tilt were active - it was pulling too much power or my injector just wasn't cooperating. It was a very nice Cody unit rated for 90W so it should have been no problem. I replaced it with a cheap POE++ from amazon and that resolved the power cycle issue, but maybe it introduced this data bottleneck? I'll put the Cody injector back in tomorrow to see if it resolves the throughput issue. If it does, then I'll just dump the POE completely and power it with an AC adaptor.

I'm thinkin the software (Blue Iris) is only as fast as your hardware.
what hardware( i.e. WD purple(s) are all these cam streams being written to?
are all your cams running high frame rates? at 30 FPS ?
i5 is an 8500, 6 core 6 thread processor. You should be good with either processor.
i7 is an 8700 6 core 12 thread processor. ( which I'm running as well)
I would do a trial of 15-18 FPS and see if the problem continues or goes away.
then you might get more data to work with. like is it a limitation of the cam? or the Wireless lan? or ?
Yeah, it's all going to a 4gb WD purple on the BI PC. All cams are set to at least 20 fps, most 25 or 30. Sorry, you're right - it's an I7-8700. I'm building an I5 gaming set up for my kid, so I5 was on the brain lol.

While I've been typing all this, I've dropped FPS down to 10 and bitrate to 6000 kbps and still having the issue. In fact, I've disabled ALL cameras except the problem PTZ, and I still get the same problem, ultimately ending in "no signal" error for a few seconds, then it's back. So literally no change even after disabling every other camera on the network.

So, I guess now I'm down to checking POE++ injector. If that doesn't fix it, what's next? The ethernet cable from injector to PTZ? Maybe a bad connector?

After I made the fiber run from dock switch to tree switch, I connected my laptop to tree switch to make sure the fiber connection was good, and I played youtube videos at 4K, ran speed tests, etc and got 900+ mbps with no drops or buffering etc, so I don't think it's an issue with the fiber link.

Oh well, it's late, going to bed lol. I'll try the Cody injector tomorrow and go from there. Open to more suggestions!
 
  • Like
Reactions: Flintstone61
Ok, I was going to mention this first, but your other descriptions were leading to another possible cause.

The other thing I have noticed is BI can be "overly sensitive" to issues that are happening before any other device notices.

I had a camera doing the same thing in BI and it turned out the POE port on the switch was starting to fail. Swapped out switches and all has been fine since.

So I think swapping out that power supply is a good 1st step.

In regards to the router, most of us here do not have our cameras going thru the router. I can unplug my router and my BI computer will still have every camera streaming to it.

This is accomplished via a Dual NIC system - install another ethernet card in the computer and have all the cameras going to one ethernet port on one IP subnet and the Internet/LAN connected to the other ethernet port. This keeps the cameras isolated off of your home LAN so that these never buffering cameras are not clogging up the home LAN, keeps the cameras off the internet to eliminate being hacked/used as DDoS attacks, and removes a known problematic device out of the network topology.

If yours is close enough - any way you can rewire to avoid the router?



1778067919229.png
 
+1^^^ again :cool:...or at least this if doable:

move-BI-to-switch.jpg
 
+1^^^ again :cool:...or at least this if doable:

View attachment 243138
I'm set up like TonyR's pic. The workload is going thru the switch via the green wires ( in this pic). The switch does the acknowledge/send/receive data packet work, and the router knows all the Ip cams are there, but isn't sending packets thru the router. ( we hope ) :)
I was able to run the setup the wrong way as well, until my 5 port switch order from Amazon showed up...with the router doing all the work. It worked but router was warm, and we must have 9 phones minimum on wifi, and wifi didn't seem as quick. Running an Asus Rt-AX55.
 
Last edited:
  • Like
Reactions: TonyR and jrbeddow
I'm set up like TonyR's pic. The workload is going thru the switch via the green wires ( in this pic). The switch does the acknowledge/send/receive data packet work, and the router knows all the Ip cams are there, but isn't sending packets thru the router. ( we hope ) :)
I was able to run the setup the wrong way as well, until my 5 port switch order from Amazon showed up...with the router doing all the work. It worked but router was warm, and we must have 9 phones minimum on wifi, and wifi didn't seem as quick. Running an Asus Rt-AX55.

Yep on the we hope - and a good test would be to unplug the router and see if the computer still sees the cameras. We have seen a few instances here were somehow it was still going thru the router and the person lost their cameras when unplugging the router - go figure LOL.
 
Yep on the we hope - and a good test would be to unplug the router and see if the computer still sees the cameras. We have seen a few instances here were somehow it was still going thru the router and the person lost their cameras when unplugging the router - go figure LOL.
Maybe cameras were still set for DHCP, no static IP's assigned? :idk:
 
  • Like
Reactions: Flintstone61
+1^^^ again :cool:...or at least this if doable:

View attachment 243138
Well, apparently I was smarter a few years ago than I am today, because I checked and this is exactly how I have it set up. I verified that I can take the router out of the loop and my BI PC still sees the cameras.

I tried the Cody POE++ injector instead of the cheap amazon one, and there's no change. Still get the same errors and drop outs. To make troubleshooting a little easier, I took my laptop with me and plugged it into the switch mounted on the tree that feeds the injector. The laptop showed internet disconnects "no internet access" sometimes when the BI errors would show up so the problem is before the POE++ injector. I could load a 4K youtube video on my laptop and that would trigger the errors on the camera. The switch is getting overloaded somehow?

So, what are the options here? Bad switch, bad individual lan port on the switch, bad fiber link from dock switch to tree switch? I didn't have time to plug the laptop into the dock switch to see if I got similar results downstream, but I guess that's next to check.
 
  • Love
Reactions: Flintstone61
Interesting.

Anything intermittently blocking the wireless bridge like overgrown trees blowing in the wind and blocking line of site temporarily?
 
  • Love
Reactions: Flintstone61
Interesting.

Anything intermittently blocking the wireless bridge like overgrown trees blowing in the wind and blocking line of site temporarily?
I think I'm going to look at this next. Looking at the bridge, I usually have a signal strength of 80 and MCS of 8. Currently steady at 30 and 1, but we've got extremely heavy rain at the moment. I do have the dock receiver in a different position than usual because I'm having work done on the dock, so maybe the link isn't what it usually is. Just kinda weird that if that's the problem, why aren't other cameras being affected? There's like 7 cameras that rely on that link, and a couple of them push just as many frames, higher resolution, and higher bitrates than the PTZ.

That also brings me back to disabling all the cameras except for the PTZ - still got the errors even with the only data being that of the PTZ. I need to just get another switch with SFP and replace the one at the tree and see what happens. Worst case, the switch is good and I have another switch to put somewhere and add more cameras, right? And that might help narrow it down to a bad fiber link?
 
Wow disabling all the cams but the PTZ and it still happens could be a wonky firmware or protocol in BI.

I would probably first delete the camera in BI and add from scratch as that is the simplest one. If that doesn't work then do a factory reset on the camera.

But yea I do like the get another switch and then add more cameras!
 
  • Like
Reactions: Techie007L
FYI: Mokerlink, and Cody are bottom of the barrel sketchy equipment. Both are prone to multiple failures.
Stick with name brand equipment if you want reliability. Netgear, Tplink, Cisco, etc.
Why would you put up cameras and then go cheap for the switches, which are the backbone of your system.
 
  • Exclamation
Reactions: Flintstone61
Wow disabling all the cams but the PTZ and it still happens could be a wonky firmware or protocol in BI.

I would probably first delete the camera in BI and add from scratch as that is the simplest one. If that doesn't work then do a factory reset on the camera.

But yea I do like the get another switch and then add more cameras!
Well, I don't understand it, but deleting the camera from BI and adding it back seems to have resolved the FPS and bottleneck errors. Almost 3 hours and zero "no signal" errors and no timeouts or even a hint of problems, and I've been watching it the entire time.

I had previously had the camera set up using a hikvision profile, and I decided to try the generic/onvif profile this time. It initially didn't work with the default settings under that profile at all, so after a little searching I found the suggestion to use "videostreamid=1" in the video profile setting, and voila, it works. When I set it up, the p2p bridge was back up to 80 signal strength, but it's been raining hard for the last 30 minutes and the link has been down to 10-20% and everything is still streaming.

This makes zero sense considering the issues with the laptop hooked up to the switch and seeing it affected as well, but I've gone from several "no signal" errors every few minutes to none in the last 3 hours. I'll keep watching it and see if I get any no signal errors overnight and then go back with a laptop again tomorrow to see if I can make it act up again.
FYI: Mokerlink, and Cody are bottom of the barrel sketchy equipment. Both are prone to multiple failures.
Stick with name brand equipment if you want reliability. Netgear, Tplink, Cisco, etc.
Why would you put up cameras and then go cheap for the switches, which are the backbone of your system.
I've been running Mokerlink switches for years without a single failure other than losing a few lan ports to lightning/electrical spikes during storms (I used to have 150' CAT6 running exposed over the ground connecting two switches and I think it was really just a giant antenna lol, hence the switch to fiber now). I've never lost a camera on a mokerlink switch - the switches have always protected the equipment from zaps. If this issue comes back, maybe it will be the first problem I've had with Mokerlink. Oh, and most of them I have are installed in outdoor boxes or sheds running in extreme heat, sun, wind, rain, humidity, you name it.

The Cody/Cudy brand, I don't have any experience with until this POE++ injector that's been a complete failure. I was under the impression that it was quality stuff, but it seemed to be stuck at 30W output and never stepped up to higher output so the camera would power cycle anytime the IR was on and PTZ motion was commanded. Maybe I just got a bad unit, maybe the brand is junk, I dunno. But I doubt I'd give them another shot at this point. What kinda failures do you see happening with Mokerlink switches?
 
  • Like
Reactions: Flintstone61
Yeah some cameras are finicky to add from new. It makes you think your crazy sometimes. I have 3 identical amcrest 1179's.
When I add them, using Find/Inspect, 1 of them doesn't end up with same results in the substream mainstream menu's.
I have to copy them from the other camera.