Qwen 4B and 9B should be faster and better reasoning than Gemma 4 E4B. Other good options for that much RAM are Gemma 4 12b and 31b. Gemma 4 E4B would be better for native audio, but OP is using Whisper for STT so prob doesn't matter
The 12B and 31B models will be useless on a Pi 5; maybe the 12B can be persuaded to run, but it may not manage more than one token per second. It only manages 17GB/s memory bandwidth (I have seen a suggestion that the 4GB model manages more).
But yes — some sort of small reasoning-oriented model (Ornith?) seems a better candidate than Qwen 35B.
(Don't get me wrong, I think the 35B model is ace… just seems like at least an unusual choice here)
Qwen3.5-4B would be a good (better?) candidate. It uses a gated, deltanet hybrid, so your KV cache stays nearly flat as context grows, important for RAM-constrained environments like the Pi.
You get a bluetooth device that plugs into the odb port, or a patch cord that you connect to your raspberry pi. There are fancier ones with more features, that can do more than just report signals like watch for certain signals or changes.
I'm currently integrating one for simple diagnostic readouts to integrated small touchscreen. The main goal being a retro looking display that mainly dumps ODB data/monitors, but also reads maps data to show a "waypointer" a la crazy taxi. (This is proving difficult)
Respectfully, I don't think this is....real?
-I mean, getting the absolute basic bits like diag codes converted from the odb 2 data to some program w/ a graphic interface is about as far as people get, if they can get passed the power and power on/off conundrums.
-If you get that far, How in the lord's name did you manage to seemingly....MITM the vehicles cloud service (climate/locks/remote start) using raspian or whatever? IE; how are you able to establish a TLS connection (yeah, like that lingo?) from the raspberry to the manufacturer API? I think the manufacturer would be interested, as you have functionaley made another key to the car.
-A more important question....what purpose does this serve? IE; the cloud connection/controller is in the car. I can't imagine it's easier to connect to the rasp shell (is it on 24/7?) to connect to the car app to turn on the ac.
- I assume you have one of the "big boy" ones with more WAM. THat said.....are you implying by saying "the pie can actually run qwen" that you are running it locally? if so.....erm...I need reciepts. Cuz idk what that even means nowadays.
Maybe i'm just jaded because my "crazy, crazy taxi style waymarker + ODB2" monitor isn't progressing, but this is...fishy
I have no idea on OP, but as far as a waypointer, have been in the area of building bespoke automotive telemetry solutions for a while now (including reversing some existing tools), I would wager that something like the Freematics One+ could do it easily if you combine the IMU with the GPS data (I've managed to get good data for something similar this way). The Freematics dongle does suck in comparison to even the now gone STN2120 as it artificially bottlenecks PID data retrieval on the CAN by the boneheaded design of the STM32 coprocessor firmware, but it probably would work well for your use case.
EDIT: I've actually got a couple cars with the One+ B, doing that sort analysis/data capture, so speaking from actual experience here.
Fair warning, I have found local models and frontier models to be very bad at the specifics when it comes to cars.
Small differences like month and year model can impact oil capacity, oil weight and things like that, the details that matter quite a bit.
I found frontier models couldn't get things like what engine was in a 1994 Nissan Skyline, one of the more infamous and talked about cars on internet forums for decades, with dedicated fan databases that would have been scraped.
Questions like "what air filter do I need for my 1994 Suzuki Swift?" are hit and miss.
Yeah referencing is the way to go, as even finetuning probably captures style more than concrete facts. I know with large context windows we don't really RAG anymore, but for owner's manual lookup with a smaller model it seems ideal.
Something every LLM user ends up learning is that they're far better used as search and summarization tools than as knowledge databases in themselves.
Just a few hours ago I gave ChatGPT my window sticker and the installation manual for a new suspension setup. I asked for new hardware that would typically be replaced during this install, like torque-to-yield bolts and fasteners. I also asked for new oil filters. I got a comprehensive grid of the exact part numbers needed in a nice dense table. sol 5.6 high is my daily driver.
It's unfortunate that a 16GB raspberry pi is so incredibly expensive now. Fascinating how we've scope creeped over the years from the original $30 or $35 raspberry pi SBC to this.
You can buy a pi 4 1GB for $49 CAD which is pretty close to the original price, especially accounting for USD:CAD conversion and inflation in the last 12 years. That's a much more normal price for something with all the I/O pins of a raspberry pi. I would never in my life even think of trying to run a LLM of that size on one (spending $300+ on the board just to do so), it's really just a weird use...
If the agent could read info like the manual and had a memory system logging vehicle operation data, this seems like it would get pretty useful over time
What's the point in having it use an offline LLM if it still requires an internet connection to send all your chat messages through the "Groupmind" paid service?
Seems like the real purpose of this project was to plug Groupmind and whatever else the author made.
Maybe it's different in the People's Republic of Europe or wherever, but either that's a translation error or I don't understand, but I would be interested to see how quickly the EPA gets mad when Claude tells you to recover your R134a to the atmosphere.
dofm | 9 hours ago
I guess it is only 3B active which helps a lot but is Gemma 4 E4B not more practical?
0xbadcafebee | 9 hours ago
dofm | 8 hours ago
The 12B and 31B models will be useless on a Pi 5; maybe the 12B can be persuaded to run, but it may not manage more than one token per second. It only manages 17GB/s memory bandwidth (I have seen a suggestion that the 4GB model manages more).
But yes — some sort of small reasoning-oriented model (Ornith?) seems a better candidate than Qwen 35B.
(Don't get me wrong, I think the 35B model is ace… just seems like at least an unusual choice here)
aamargulies | 7 hours ago
yjftsjthsd-h | 9 hours ago
Alive-in-2025 | 9 hours ago
https://www.amazon.com/s?k=obd+bluetooth+adapter
Reubachi | 9 hours ago
Respectfully, I don't think this is....real?
-I mean, getting the absolute basic bits like diag codes converted from the odb 2 data to some program w/ a graphic interface is about as far as people get, if they can get passed the power and power on/off conundrums.
-If you get that far, How in the lord's name did you manage to seemingly....MITM the vehicles cloud service (climate/locks/remote start) using raspian or whatever? IE; how are you able to establish a TLS connection (yeah, like that lingo?) from the raspberry to the manufacturer API? I think the manufacturer would be interested, as you have functionaley made another key to the car.
-A more important question....what purpose does this serve? IE; the cloud connection/controller is in the car. I can't imagine it's easier to connect to the rasp shell (is it on 24/7?) to connect to the car app to turn on the ac.
- I assume you have one of the "big boy" ones with more WAM. THat said.....are you implying by saying "the pie can actually run qwen" that you are running it locally? if so.....erm...I need reciepts. Cuz idk what that even means nowadays.
Maybe i'm just jaded because my "crazy, crazy taxi style waymarker + ODB2" monitor isn't progressing, but this is...fishy
VTimofeenko | 8 hours ago
"""
Unverified against the real car — it will confirm or refute itself on the next drive
"""
Looks like a ton of this is aspirational.
For your project -- do you really need the car's GPS positioning? Could go with a dedicated GPS antenna, like the freematics dongle
rpcope1 | an hour ago
EDIT: I've actually got a couple cars with the One+ B, doing that sort analysis/data capture, so speaking from actual experience here.
tessierashpool | 9 hours ago
the Mermaid diagram doesn't even render.
almogo | 8 hours ago
thoughtbefore | 7 hours ago
hypfer | 8 hours ago
Why does it need to be hooked up to the car for you to ask it which type of engine oil the manual recommends?
What is the point?
ehnto | 8 hours ago
Small differences like month and year model can impact oil capacity, oil weight and things like that, the details that matter quite a bit.
I found frontier models couldn't get things like what engine was in a 1994 Nissan Skyline, one of the more infamous and talked about cars on internet forums for decades, with dedicated fan databases that would have been scraped.
Questions like "what air filter do I need for my 1994 Suzuki Swift?" are hit and miss.
javier123454321 | 8 hours ago
efskap | 4 hours ago
Something every LLM user ends up learning is that they're far better used as search and summarization tools than as knowledge databases in themselves.
whalesalad | 4 hours ago
ssl-3 | 2 hours ago
Mine, too.
Did you fact-check the output?
I've found the bot to be particularly bad at stuff like producing accurate lists of parts to replace during automotive projects.
alexandra_au | 4 hours ago
Robdel12 | 7 hours ago
I wouldn't fully trust LLMs with this :p BUT this project is still really freaking sweet.
apinstein | 5 hours ago
ellis0n | 4 hours ago
walrus01 | 3 hours ago
blululu | 3 hours ago
https://www.adafruit.com/product/1914?src=raspberrypi
walrus01 | 2 hours ago
dowonseo | 3 hours ago
nacs | 3 hours ago
Seems like the real purpose of this project was to plug Groupmind and whatever else the author made.
tzs | an hour ago
[1] https://github.com/ThinkOffApp/CarWatch#local-vs-online-the-...
rpcope1 | an hour ago
Maybe it's different in the People's Republic of Europe or wherever, but either that's a translation error or I don't understand, but I would be interested to see how quickly the EPA gets mad when Claude tells you to recover your R134a to the atmosphere.
meatmanek | an hour ago
bitwize | 10 minutes ago