IR sensitivity differences between 5442-S3 and T54PRO-ZE ... puzzling results

[alekk]

In some of the discussions about trying to get good night-time motion capture with the 54PRO's, I commented that "Is it possible that the "3D NR" on the 54PRO that (as you can imagine with something that "averages" over multiple frames) just does dismally with moving objects in low light?" and "is it possible that the S3 IR light is better ... and/or maybe the S3 sensor is more sensitive to IR than the 54PRO?"

So I ran some simple tests to assess IR sensitivity of the 5442 versus the 54PRO ... short answer is if anything, the 54PRO appears to be more sensitive (in this DAYTIME scene) in IR as the image is brighter ... albeit some/most/all of this could be processing. HOWEVER, at night (which obviously is the use case for IR), I found the opposite - be sure to see the 2nd post!

So what I thought would be an easy test ended up providing puzzling results!!!

The first animated GIF (original PNG's attached) is using the default settings, but setting shutter to 1/30000s (intentionally under-exposed), iris 100 (both are F/1.2 at full wide and the zoom is fairly wide with the 5442@430 and 54PRO@550 to match horizontal FOV), gain 0, and all NR 0. So the "exposure triangle" should (!) be the same.

So we can answer the IR sensitivity question (again, this is DAYTIME - I'll test at night) by comparing the exposure when I force B&W ... and the 54PRO is certainly brighter ... but again, some/most/all of this could be in the processing. Regardless, it seems pretty certain the 54PRO sensor is at least just as sensitive to IR as the 5442 ... which I would be extremely surprised if it was not ... but never hurts to make sure and rule that out.

The second animated GIF is the same params in Color. Remember that iris=100, gain=0, and shutter is FORCED AT 1/30000s ... so the camera shouldn't (!) change the settings and "correctly" meter/expose. Once again, the 54PRO is brighter (probably due to processing) plus it's a LOT more colorful and less noisy.

Finally, the third animated GIF is the same params in Color, but instead of "forcing" the shutter to 1/30000, I set it to 0-1msec (1/1000s is more than enough light here) ... so the camera can properly meter ... which they do. Unfortunately, I don't know if any way to "see" what settings are being actually applied - i.e. we are assuming(!) that iris(100) & gain(0) are LOCKED and don't change, what shutter speed did each camera "decide" to use.


I think the IR sensitivity is basically the SAME between the 5442 and 54PRO camera's ... or at least "same" relative to the color differences ... since we don't know how much the difference is due to processing. In fairness, I need to do a nighttime test under low light to see if for some reason that would be different ... but shouldn't be ... which turned out to be WRONG.

So in summary (and after looking at the nighttime posts), I'm really puzzled what the heck is going on! ;-)

Again, this ideally should be a fairly simple test - we are just looking at exposure - not sharpness, noise, motion, etc. It's frustrating that my results were not consistent between day & night (photons should be photons!) and I can only guess that there some funky processing going on that is not allowing me to truly assess lens/sensor capability.

Note these tests do NOT tell us anything about possible differences between the IR LIGHT ... and how the camera (smartly) adjusts exposure from that.
Nor do they tell us anything about possible "3D NR multi-frame averaging" that the 54PRO may be overly aggressive with compared to the 5442 which would result in a cleaner STATIC image, but would cause more motion blur.








View attachment 232050

 

[alekk]

So here's my night-time comparison of the IR sensitivity of the 5442 versus the 54PRO.

After seeing the 54PRO being brighter during daylight (albeit probably mostly due to processing), I figured that night time would be the same ... after all photons are photons!

For my nighttime test, I again used iris 100, gain 0 and all NR turned off. But instead of 1/30000s, I went with 1/3s (!) for the shutter speed - had to go super-duper slow since ZERO gain. I'm using an external IR illuminator - i.e. I was NOT using the onboard IR lights ... so both cameras (less than a foot vertically apart) were seeing the SAME light from an IR lights that was at least 5 feet away.

The first animated gif shows the 5442 versus the 54PRO having identical settings. Why the heck is the 5442 image noticeably brighter?!?

Note this is at odd with @tigerwillow1 results (see next post) so I don't have an explanation for this except that something "funky" is going on with the processing ... albeit my image is much darker than his.


For completeness, I provide a second animated GIF that shows the 54PRO with HLC NONE (as shown in the first GIF) versus HLC=1 ... which I usually use as it provides more dynamic range.

 

[tigerwillow1]

I'm repeating a bit of an earlier post here. I ran a 3.6mm PRO next to a 3.6 5442 S2. My summary conclusion for IR sensitivity is that the PRO has a little bit of an edge overall. For instance, some darker areas are brighter in the PRO, but some small details are clearer in the 5442. I did find the PRO's IR lighting to be better distributed as compared to the 5442's center hotspot. For motion blur, I saw no difference between the two. The one thing where the PRO clearly excelled for me is in color mode with low light, which includes using its warm white light.

So is it worth upgrading to the PRO? IMO it depends on the usage profile. Since I use external IR emitters, the PRO's better IR distribution adds no value. Ditto for the warm white light, because I won't use it. For someone using the built-in IR or wanting to stay in color mode overnight, it would be an opposite conclusion. There's another factor that's the "elephant" for me: AI animal detection. As a first glance, it looks much better to me than the S3's animal detection, in the sense of significantly less false triggers from shadows, blowing grass, and blowing tree branches. No final conclusion on that however until I run side-by-side with the S3.

These are sample pictures both using the PRO's built in IR. PRO on the left, 5442 S2 on the right. Overall the PRO looks better, but when an animal walked through fairly close in, there wasn't any difference in its moving image. Further away, maybe there's a difference but none of the critters obliged me during the dual camera setup.

 

[alekk]

That's a good test @tigerwillow1 ... especially since you used the SAME illuminator to compare 'em. Would have been slightly better to have used an external illuminator so you don't have the light source right on the optical axis of the 54PRO ... but I'm looking at 'em to compare exposure .. and since you just did the PRO's IR light, you've removed any differences from switching to the 5442 IR light ... plus you also provided some helpful comments on.

Did you run both camera's at identical "LOCKED" exposure settings (fixed shutter speed and gain - no iris on those - right?) since I was surprised that your exposures matched up which is different than what I saw in my night-time testing ... please see post #2 which has been updated with those results.


As noted above, I am VERY puzzled by my results.

Again, this ideally should be a fairly simple test - we are just looking at exposure - not sharpness, noise, motion, etc. It's frustrating that my results were not consistent between day & night (photons should be photons!) and I can only guess that there some funky processing going on that is not allowing me to truly assess lens/sensor capability.

This seems to further support the theory that the "challenges" people are seeing in low-light IR is due to that processing getting a bit wonky ...

 

[tigerwillow1]

Here are the 54PRO and 5442 S2 3.6mm comparison pictures with only external IR lighting. 54PRO on the left. I did not have the exposure locked down. I have to admit this static picture looks better on the pro.

 

[tigerwillow1]

Here are the 54PRO and 5442 S2 3.6mm comparison pictures with only external IR lighting. 54PRO on the left. I did not have the exposure locked down. I have to admit this static picture looks better on the pro.

View attachment 232125 View attachment 232126

 

[bigredfish]

Here are the 54PRO and 5442 S2 3.6mm comparison pictures with only external IR lighting. 54PRO on the left. I did not have the exposure locked down. I have to admit this static picture looks better on the pro.

View attachment 232125 View attachment 232126

The static picture on the 54PRO DOES look better on every cam we've tested this past week.

Problem is that's meaningless as the VIDEO with MOVING targets is clearly NOT better

 

[tigerwillow1]

Here are the 54PRO and 5442 S2 3.6mm comparison pictures with only external IR lighting. 54PRO on the left. I did not have the exposure locked down. I have to admit this static picture looks better on the pro.

 

[alekk]

The static picture on the 54PRO DOES look better on every cam we've tested this past week.

Problem is that's meaningless as the VIDEO with MOVING targets is clearly NOT better

It's possible (but unlikely) that IF the 54PRO is LESS sensitive to IR, that in order to expose the picture properly, it has to run at slower shutter speed and/or higher gain ... which would both contribute to motion blur.

So that's why I was trying to assess using locked down exposure settings (along with ZERO gain ... since we don't know if those scales are the same) if there was any difference.


@tigerwillow1 : I like your pictures because it includes a foreground element lit up ... with the background that is essentially ambient, but also has a bright light. Plus you did an excellent job aligning the camera images. Also, you have waayyy more hands-on experience that me (quite the FLEET of cameras!), so your opinions are good to hear.

So if by chance you have both both camera's up again and are so inclined, I'd be curious to see a side-by-side comparison with locked down settings - ideally using an external illuminator that is (semi) equidistant from the camera's - i.e. so they both are seeing the same light.

BTW, you may find it challenging to get a non-BLACK image at night with gain=ZERO .... ;-)

 

[tigerwillow1]

side-by-side comparison with locked down settings

I had one motion comparison with the shutters both set at 1/250, but nothing else locked down (a cat walking through). My subjective judgement was that there was no difference between the 2 cameras in terms of motion blur. But if one was running more gain or other different exposure setting than the other, I wouldn't know. I won't have these cameras side-by-side any more. I have to get the PRO permanently installed before the winter weather hits. What I'm actually more interested in is setting up a 6mm IL camera next to a 6mm S3. I've got the IL on order but don't know when I'll actually get my hands on it.

 

[CamCrazy]

Here are the 54PRO and 5442 S2 3.6mm comparison pictures with only external IR lighting. 54PRO on the left. I did not have the exposure locked down. I have to admit this static picture looks better on the pro.

View attachment 232131 View attachment 232132

The Pro has some over exposure/blown hightlights going on there. The big takeaway for me is the massive overprocessing with the 54Pro, looks like combination of sharpening, gain and NR, couldn't say which. It results in less detail and a far from natural look to some of the shrubs/trees/grass.

 

[alekk]

For grins, I moved the external IR illuminator to the edge of the roof ... so it's closer to the driveway and therefore more lit-up ... and I was able to increase the shutter speed to 1/10s. As before, other params are iris 100, gain 0, and NR 0 ... as I'm trying to just compare the IR sensitivity of the 54PRO-ZE versus the 5442-S3 with (ideally!) locked settings.

I'm surprised to see this much difference ... but don't know what to make of it.

The 2nd animated GIF shows the 54PRO with HLC OFF versus HLC ONE.

 

[alekk]

Here's the exact same setup after turning on BOTH onboard IR lights (with max manual settings) which adds a little bit of light to the scene.

 

[CamCrazy]

For grins, I moved the external IR illuminator to the edge of the roof ... so it's closer to the driveway and therefore more lit-up ... and I was able to increase the shutter speed to 1/10s. As before, other params are iris 100, gain 0, and NR 0 ... as I'm trying to just compare the IR sensitivity of the 54PRO-ZE versus the 5442-S3 with (ideally!) locked settings.

I'm surprised to see this much difference ... but don't know what to make of it.

The 2nd animated GIF shows the 54PRO with HLC OFF versus HLC ONE.


View attachment 232331

View attachment 232332

Probably don't need to tell you this but logic goes out of the window with these Pro's at the moment. Not really sure you can compare the settings like for like. If the motion quality was as good as these stills Andy would need to order as much stock as he could!

Oddly enough I am in the process of replacing the connector on one of my 49225 PTZ's due to corrosion (silicone grease I know! ) and happened to power it up on the bench today. The IR illuminators on those things are next level, you can see the power of them in dim daylight, no substitute for good IR or white LED as we know.

 

[tigerwillow1]

Probably don't need to tell you this but logic goes out of the window with these Pro's at the moment. Not really sure you can compare the settings like for like. If the motion quality was as good as these stills Andy would need to order as much stock as he could!

I was starting to think the same thing myself. For other than shutter speed, I don't think the numbers on the sliders have any defined value other than the position of the slider. I'm thinking the only way to make a good comparison is to ignore the numbers and tune each camera for its optimal performance, a lengthy process, then compare the images. Then after going through all of that, the optimal settings will change when a camera is in a different location or lighting situation, and could flip which one is the "winner". Unless the differences are obvious, deciding which camera is better might not be easy, instead taking at least months for user reports to show a consensus. The improvement was obvious when going from the 2MP starlights to the 5442s. That doesn't seem to be the case this time.

 

[CamCrazy]

I was starting to think the same thing myself. For other than shutter speed, I don't think the numbers on the sliders have any defined value other than the position of the slider. I'm thinking the only way to make a good comparison is to ignore the numbers and tune each camera for its optimal performance, a lengthy process, then compare the images. Then after going through all of that, the optimal settings will change when a camera is in a different location or lighting situation, and could flip which one is the "winner". Unless the differences are obvious, deciding which camera is better might not be easy, instead taking at least months for user reports to show a consensus. The improvement was obvious when going from the 2MP starlights to the 5442s. That doesn't seem to be the case this time.

100% this, the sliders and numbers, particularly shutter speed are of no relevance on these at this point. Your quote re the 5442's hit the nail right on the head, at this stage the real upgrade from a 5442 is....a 5442 S3! if you don't already own one. The 2MP starlights were such an easy buy at the time, similar the 5442 which were crazy good in low light and on a 4MP sensor! But the 54Pro will make you question why you 'upgraded', in some instances even from a 2MP starlight frankly.

Myself and others have many hours into these Pro's, the most bizarre camera I have ever played around with, virtually zero logic to them, common sense need not apply!

 

[alekk]

I agree with all that you guys wrote above ... and yea, comparing the T54PRO-ZE and 5442-S3 using the SAME(!) settings (to see if the sensor/lens improved) may not be easily doable.

Ideally (again!) shutter speed should be straightforward - in the examples above 1/10s (locked - not 0-10msec) should be the same. And the aperture of the lens at close to full-wide-zoom should be comparable given the game F/1.2 specs.

However, I'm not convinced that gain=0 is the same - the darker images of the 54PRO suggests maybe a lower "base" ISO?

I'm very certain that the gain scale is different as the 54PRO (latest firmware) goes to "Infinity and Beyond" (I appreciate they don't "limit" it) ... but it's gets pretty crazy above 70 compared to the 5442.

Ideally (yet again!) setting NR levels to ZERO and also toggling it OFF should make those match up ... but who knows.

Ditto the differences in the image processing pipeline.


So I thought this would be somewhat straightforward comparison ... but it turned out not to be - oh well.