The telecom regulator/inspection agency in Norway found several not so temporary fiber cable fixes. The telcos had laid fiber exposed on the ground and over creeks to cell towers after storms had disrupted the cables. They had made that as a temororary fix right after storms, but had never protected the cables properly afterwards.
Yes that’s the dream.. the CAT6 was already there though, and no proper conduit to pull fibre.. so I will use it till it dies and then replace it with fibre. I can saturate 10Gbps across the trunk though!
I was thinking more for longevity and to avoid possible electrical issues, but if it was already there and difficult to replace, it certainly makes sense to use it until it dies. Maybe look into surge protection on each end though (something like this) just for peace of mind.
The router-on-a-stick is a compelling approach, though I ran into an obstacle when thinking about it for my setup: The router needs to terminate PPPoE and in order to keep 1500 MTU for Internet connections a MTU of 1508 is used for the ONT--Router ethernet link. Putting this in a VLAN would mean a weird setup with mixed MTUs for different VLANs, sounds like a headache. Haven't got around to experimenting with this.
The MTU is indeed only a "maximum", it doesn't change anything for frames that are already on the network. Setting the MTU to 1508 on the switches wouldn't affect behaviour of nodes in any way.
The scary thing is that interface MTUs are often used by operating systems to determine higher-level behavior such as the TCP MSS. But as long as it stays set to 1500 that's not an issue.
So the only problem is the router itself which needs to be very carefully configured to understand the different MTUs on different VLANs, but maybe that's actually easy (just set MTU on the vlan interface?).
Having said all that, I would rather not spend a lot of time debugging MTU issues right now, I have enough bad memories of IPv6 PMTUD being consistently broken.
Several fiber ONT’s support unwapping the pppoe as an option as part of “bridge mode”, so you can let the ONT handle the pppoe side and your equipment just does a vlan tag (if needed, such as quantum requires).
Ha, I'm just about to embark on a very similar adventure. We have 8Gb/s FTTP, so I'm running fibre from the distribution point to the office upstairs and the rec room in the basement. No VLANs yet, because I don't want to replace my old mesh network in this tranche of spending lots of money on cables (as my partner dismissively puts it).
support 10Gbps networking for speed and future proofing
As much as I love tinkering with new stuff, future proofing is one of the big fallacies in home network setup. You end up overpaying for overprovisioning, while it would be cheaper to simply build what you need and extend later with then cheaper hardware.
10Gbps networking hardware is still pretty expensive, whereas 2.5Gbps hardware has become really affordable. I am currently setting up a small home network with 2.5Gbps, where two switches are connected each with a 10Gbps fiber line. Overkill? Maybe, but I like playing with new stuff. :)
One note: I think if I ever build a house in my life, I'd just run fiber everywhere. This is because I learnt that transceivers, compared to RJ45 ethernet ports, use much less energy during operation and induce significantly less latency. For desktop computers, it doesn't make a difference if I add a SFP+ network card or a high-bandwidth RJ45 ethernet card. There are WiFi-APs with SFP+ ports, though one needs to keep power supply in mind, as you can't use PoE. And for everything else, you can either place a switch/adapter or just use WiFi.
Interesting VLAN numbering, similar to mine (30 iot, 40 guest, 50 dmz), though I picked untagged for LAN, 20 for servers, 10 for lab (WAN is tagged on 600 on my ISP)
Really liked your diagrams, were those from draw.io, or something else?
The pattern was always the same: changing a port’s VLAN assignment while connected through that port (or a dependent path), then losing the ability to reach the switch management interface to undo the change.
kel | 17 hours ago
XKCD 2347 "All Modern Digital Infrastructure" but the tiny block is the Cat6 running in a trench through your garden. Great post
[OP] mtsolitary | 13 hours ago
Thanks! Yes it is a little scary that it all depends on that one cable..
enpo | 11 hours ago
The telecom regulator/inspection agency in Norway found several not so temporary fiber cable fixes. The telcos had laid fiber exposed on the ground and over creeks to cell towers after storms had disrupted the cables. They had made that as a temororary fix right after storms, but had never protected the cables properly afterwards.
trousers | 4 hours ago
I’d recommend running fiber and not cat6 through the garden.
[OP] mtsolitary | 4 hours ago
Yes that’s the dream.. the CAT6 was already there though, and no proper conduit to pull fibre.. so I will use it till it dies and then replace it with fibre. I can saturate 10Gbps across the trunk though!
trousers | 4 hours ago
Nice!
I was thinking more for longevity and to avoid possible electrical issues, but if it was already there and difficult to replace, it certainly makes sense to use it until it dies. Maybe look into surge protection on each end though (something like this) just for peace of mind.
sammko | a day ago
The router-on-a-stick is a compelling approach, though I ran into an obstacle when thinking about it for my setup: The router needs to terminate PPPoE and in order to keep 1500 MTU for Internet connections a MTU of 1508 is used for the ONT--Router ethernet link. Putting this in a VLAN would mean a weird setup with mixed MTUs for different VLANs, sounds like a headache. Haven't got around to experimenting with this.
[OP] mtsolitary | a day ago
Yeah not sure how to handle PPPoE! Interesting challenge. I imagine you’ll probably need enterprise switches to make that work?
sammko | 21 hours ago
The MTU is indeed only a "maximum", it doesn't change anything for frames that are already on the network. Setting the MTU to 1508 on the switches wouldn't affect behaviour of nodes in any way.
The scary thing is that interface MTUs are often used by operating systems to determine higher-level behavior such as the TCP MSS. But as long as it stays set to 1500 that's not an issue.
So the only problem is the router itself which needs to be very carefully configured to understand the different MTUs on different VLANs, but maybe that's actually easy (just set MTU on the vlan interface?).
Having said all that, I would rather not spend a lot of time debugging MTU issues right now, I have enough bad memories of IPv6 PMTUD being consistently broken.
trousers | 4 hours ago
Several fiber ONT’s support unwapping the pppoe as an option as part of “bridge mode”, so you can let the ONT handle the pppoe side and your equipment just does a vlan tag (if needed, such as quantum requires).
shapr | 21 hours ago
I just installed a MikroTik CRS304 today, and was about to begin on VLANs tomorrow morning, so this is prescient!
jfb | 3 hours ago
Ha, I'm just about to embark on a very similar adventure. We have 8Gb/s FTTP, so I'm running fibre from the distribution point to the office upstairs and the rec room in the basement. No VLANs yet, because I don't want to replace my old mesh network in this tranche of spending lots of money on cables (as my partner dismissively puts it).
FRIGN | 2 hours ago
As much as I love tinkering with new stuff, future proofing is one of the big fallacies in home network setup. You end up overpaying for overprovisioning, while it would be cheaper to simply build what you need and extend later with then cheaper hardware.
10Gbps networking hardware is still pretty expensive, whereas 2.5Gbps hardware has become really affordable. I am currently setting up a small home network with 2.5Gbps, where two switches are connected each with a 10Gbps fiber line. Overkill? Maybe, but I like playing with new stuff. :)
One note: I think if I ever build a house in my life, I'd just run fiber everywhere. This is because I learnt that transceivers, compared to RJ45 ethernet ports, use much less energy during operation and induce significantly less latency. For desktop computers, it doesn't make a difference if I add a SFP+ network card or a high-bandwidth RJ45 ethernet card. There are WiFi-APs with SFP+ ports, though one needs to keep power supply in mind, as you can't use PoE. And for everything else, you can either place a switch/adapter or just use WiFi.
deivid | 23 hours ago
Interesting VLAN numbering, similar to mine (30 iot, 40 guest, 50 dmz), though I picked untagged for LAN, 20 for servers, 10 for lab (WAN is tagged on 600 on my ISP)
thesnarky1 | 22 hours ago
Really liked your diagrams, were those from draw.io, or something else?
I know this feeling well!
[OP] mtsolitary | 15 hours ago
Hey thanks, yes draw.io just with transparent background and the same font I use on the blog. And two versions for most diagrams for light and dark :)