Sharing setups with CodeProject AI on Coral hardware (Spring 2024)

[Ryushin]

Just upgraded my M.2 single Edge TPU to M.2 Dual Edge TPU. No even need for driver re-installation. It works right out of the ESD carrier Other hardware is Asrock DeskMini A300 with Ryzen 3 3200, 32Gb RAM. Real powerhouse!

Where you able to get both TPU's working Code Project. The dual code won't use both of my TPUs. I even moved code project to my Linux server in docker and still the same result. So for now, I'm still using a single TPU even though the OS sees both TPUs.

 

[koops]

Where you able to get both TPU's working Code Project. The dual code won't use both of my TPUs. I even moved code project to my Linux server in docker and still the same result. So for now, I'm still using a single TPU even though the OS sees both TPUs.

I just upgraded an added in a dual tpu chip into my server in addition to its original single tpu. I was occasionally getting timeouts when the single tpu was busy.

Multi-tpu is now working as far as I can tell. Both code project and blue iris show multi-tpu in use.

I now have the original single tpu M2 M key and dual tpu e key via a cheap aliexpress pci-e wireless adaptor card (cant paste a link to this).
The dual tpu only shows as a single device in windows (confirmed this by having the dual tpu as the only tpu and checking available devices).

The only modification I had to do was edit the options.py file. There were a heap of red messages within the CPAI log window (No multi-TPU interpreters" error).

The work around was to edit C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\options.py) and changed the 60 seconds to 1 day.

Exact line

self.MAX_IDLE_SECS_BEFORE_RECYCLE = 86400.0 # To be added to non-multi code

source :

Anyone know of any logging option to be able to tell which tpu is being sent the detection job?

 

[AlwaysSomething]

I just upgraded an added in a dual tpu chip into my server in addition to its original single tpu. I was occasionally getting timeouts when the single tpu was busy.

Multi-tpu is now working as far as I can tell. Both code project and blue iris show multi-tpu in use.

I now have the original single tpu M2 M key and dual tpu e key via a cheap aliexpress pci-e wireless adaptor card (cant paste a link to this).
The dual tpu only shows as a single device in windows (confirmed this by having the dual tpu as the only tpu and checking available devices).

The only modification I had to do was edit the options.py file. There were a heap of red messages within the CPAI log window (No multi-TPU interpreters" error).

The work around was to edit C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\options.py) and changed the 60 seconds to 1 day.

Exact line

self.MAX_IDLE_SECS_BEFORE_RECYCLE = 86400.0 # To be added to non-multi code

source :

Anyone know of any logging option to be able to tell which tpu is being sent the detection job?

AFAIK, the only way to get both TPUs working (for the DUAL TPU card) is to use one of these adapters:

PCIe adapter

m.2 B+M adapter

I don't remember the exact reason but if I remember correctly it had to do with the number of PCIe lanes for the WIFI slot which is handled by the motherboard.

As far as logging to see which TPU is being used, I think you'd have to modify the python code yourself. I forget the exact file but I think it's objectdetection_coral_multitpu.py

 

[koops]

Thanks!

Yeah i did see those. Just ordered one now. I'll have a play with the logging and see if that info tpu instance information is available.

edit

As far as logging to see which TPU is being used, I think you'd have to modify the python code yourself. I forget the exact file but I think it's objectdetection_coral_multitpu.py

You were right. Its only seeing 2 tpu's not 3.

Had a look around and and modified near line #519 of tpu_runner.py (in the logs it shows as objectdetection_coral_adapter.py but its not) to put in :

show_tpu_list = edgetpu.list_edge_tpus()
logging.critical(f"TPU list {tpu_count}")
logging.critical(f"TPU list {show_tpu_list}")

Which then shows detected tpu's on start up.

16:28:42:Started Object Detection (Coral) module
16:28:44 objectdetection_coral_adapter.py: CRITICAL:root:TPU list 2
16:28:44 objectdetection_coral_adapter.py: CRITICAL:root:TPU list [{'type': 'pci', 'path': '\\\\?\\ApexDevice0'}, {'type': 'pci', 'path': '\\\\?\\ApexDevice1'}]
16:28:44 objectdetection_coral_adapter.py: TPU detected
16:28:44 objectdetection_coral_adapter.py: Attempting multi-TPU initialisation
16:28:44 objectdetection_coral_adapter.py: Supporting multiple Edge TPUs

 

[Ryushin]

I just upgraded an added in a dual tpu chip into my server in addition to its original single tpu. I was occasionally getting timeouts when the single tpu was busy.

Multi-tpu is now working as far as I can tell. Both code project and blue iris show multi-tpu in use.

I now have the original single tpu M2 M key and dual tpu e key via a cheap aliexpress pci-e wireless adaptor card (cant paste a link to this).
The dual tpu only shows as a single device in windows (confirmed this by having the dual tpu as the only tpu and checking available devices).

The only modification I had to do was edit the options.py file. There were a heap of red messages within the CPAI log window (No multi-TPU interpreters" error).

The work around was to edit C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\options.py) and changed the 60 seconds to 1 day.

Exact line

self.MAX_IDLE_SECS_BEFORE_RECYCLE = 86400.0 # To be added to non-multi code

source :

Anyone know of any logging option to be able to tell which tpu is being sent the detection job?

Thanks!

Yeah i did see those. Just ordered one now. I'll have a play with the logging and see if that info tpu instance information is available.

edit

You were right. Its only seeing 2 tpu's not 3.

Had a look around and and modified near line #519 of tpu_runner.py (in the logs it shows as objectdetection_coral_adapter.py but its not) to put in :

show_tpu_list = edgetpu.list_edge_tpus()
logging.critical(f"TPU list {tpu_count}")
logging.critical(f"TPU list {show_tpu_list}")

Which then shows detected tpu's on start up.

16:28:42:Started Object Detection (Coral) module
16:28:44 objectdetection_coral_adapter.py: CRITICAL:root:TPU list 2
16:28:44 objectdetection_coral_adapter.py: CRITICAL:root:TPU list [{'type': 'pci', 'path': '\\\\?\\ApexDevice0'}, {'type': 'pci', 'path': '\\\\?\\ApexDevice1'}]
16:28:44 objectdetection_coral_adapter.py: TPU detected
16:28:44 objectdetection_coral_adapter.py: Attempting multi-TPU initialisation
16:28:44 objectdetection_coral_adapter.py: Supporting multiple Edge TPUs

I have not looked at this for a few months. I had opened a Git issue about it and nothing has been done since posting it:

Coral Dual TPU Second TPU Does Not Work · Issue #17 · codeproject/CodeProject.AI-ObjectDetectionCoral

I've tested this with Windows and Docker (on Linux) installer. I'm currently running the Docker version which exposes CPAI to the network so the Blue Iris server can access it. I run docker contain...

github.com

There are a bunch of open issues with Multi-TPU. So you are seeing both of your TPUs from the single PCIe card? What version of CPAI are you running?

 

[koops]

So you are seeing both of your TPUs from the single PCIe card? What version of CPAI are you running?

No im seeing 1 x single tpu (m2) 1 x dual tpu (m2 e key on a pcie card) for a total of 2 available tpu's when i should have 3.

So mutli tpu is working for me for a day or so now but not all 3 (or so I believe).

For reference im using CPAI 2.9.5 & Object detection (coral) 2.4.0.

 

[AlwaysSomething]

Just to clarify, koops was talking about 2 separate issues.

1. Getting the Dual TPU card (M.2 Accelerator with Dual Edge TPU | Coral) to get both TPUs recognized by the OS. That is where you need one of the two adapters I linked to. That has nothing to do with CPAI. FYI - I have one of each of those cards and they both worked well for me. It did take about 10 days to get since it's coming from China.

2. Getting the CPAI software to work with Multiple TPU's. This is the problem everyone has with the Assets (models files) and has goes back to the version after 2.5.1. (that is the version I kept on my BI PC even thought it's only single TPU). I opened an issue on the Code Project site but that's gone now. I opened it later on IPCamTalk as well but that didn't help so I gave up. However I've seen a few people post about issues about it on Git so will see if that helps. FYI - The developer for the Coral module is very helpful but he doesn't control the released code or the assets so we can't blame him.

The Coral code isn't bad and I can figure it out. You can run it standalone which is what the developer does (executes from CLI).

Me venting now...
I tried looking at the server code with each release and it just keeps getting more convoluted in my opinion. Instead of focusing on simplicity and reliability it seems to be going the route of fancy and elaborate (which if you read the release notes it even says it may cause some problems). There also hasn't been any development in 4-5 months. I get this is FOSS so can't complain and just venting...