I'm pretty dissatisfied with all the mesh VPNs options, and am closer to "no mesh; only IPv6 and workload mTLS" than adopting a new mesh, but I'm keen to learn more.
Common mesh VPN and SD-WAN tools (Tailscale, NetBird, ZeroTier, Nebula, etc.) admit only nodes that run their own agent, and ranet3 runs beside them on the same host.
From this I infer your project is some sort of alternative to the various mesh VPNs, which is helpful to orient my understanding. I'm not that familiar with much of this, so my questions probably make no sense.
Babel measures the round trip of every link and routes through a relay whenever the direct link is slower.
In a typical mesh VPN, when might it be faster to route via another node, than to go direct?
A policy on source and destination prefix sends traffic through chosen waypoints and out at a chosen exit. Segment routing is implemented in userspace and works even on macOS, whose XNU kernel has no support for it.
In a typical mesh VPN, when might I want a policy to route via a chosen waypoint?
For the "chosen exit" case, is this similar to Tailscale exit nodes?
More generally, what are the limitations of this, compared to typical mesh VPNs?
I suppose it doesn't do NAT punching, but instead unreachable nodes (such as your MacBook) are meant to reach out to your reachable nodes, and this architecture will find the best path? (Best being policy or lowest RTT).
Is it end-to-end encrypted? i.e. If traffic from node A to node B is routed via a waypoint W, does W see the original packet, or the encrypted packet?
Where does addressing fit here? A typical mesh VPN gives nodes their own address, on a "flat" (?) network.
IIUC, ranet is "bring your own addressing"? Your page mentions you run two ASs on this, so "bring your own addressing" is a feature, I suppose?
If I don't have an AS (but I do have an ISP or VPS that gives me a /56) what might a typical way be to allocate an address or prefix to my laptop? Static routing?
If I travel internationally and want to avoid round trips to my home when accessing my mesh nodes, how might I do that? Is this the "faster to route via another node, than to go direct" case?
I'm pretty dissatisfied with all the mesh VPNs options
Me too! Especially so if you want to play with overlay network intensively
From this I infer your project is some sort of alternative to the various mesh VPNs
Yes. Not only this, I want ranet3 to interact nicely with other RFC compliant VPN (say any other ipsec apps on your phone or server) and routing daemons (say BIRD ‘s babel implementation)
Btw this already woks ;)
In a typical mesh VPN, when might it be faster to route via another node, than to go direct?
Maybe during provider outages if you are behind residential ISPs? or busy hours? I'm not sure exactly when multiple hops will be faster.
This behavior is actually a "nice byproduct" of wanting to have ranet3 on laptops (which moves between networks). Babel is more often used to track lossy links, I would want routable addresses configured on laptops to be withdrew quickly when its not reachable
In a typical mesh VPN, when might I want a policy to route via a chosen waypoint?
This is probably fairly useless to normal users but quite standard in ISP deployments. For example some transit provider have lower cost and operators might want to send most of the traffic to that link instead of sending traffic to more expensive providers.
But you can interpret this as traffic can take arbitrary routes you asked for, or the route with lowest metric value
For the "chosen exit" case, is this similar to Tailscale exit nodes?
Exactly
I suppose it doesn't do NAT punching
This is planned and will be implemented very soon
Is it end-to-end encrypted? i.e. If traffic from node A to node B is routed via a waypoint W, does W see the original packet, or the encrypted packet?
No. Transit node traffic are decrypted and re-encrypted (I mean it's just a "routed mesh" right now). Also I think making it "real e2ee" will result in some performance penalties (in the sense of ESP in ESP) and losing some information of the packet (say clamping TCP MSS)
One argument I would make is that, (similar to Tailscale and other existing solutions) you should probably trust the nodes you add to the mesh...
Where does addressing fit here? A typical mesh VPN gives nodes their own address, on a "flat" (?) network.
You can use arbitrary addresses, if you prefer tailscale like behavior you can just carve out the entire CGNAT range or subset of it, or go IPv6 only
IIUC, ranet is "bring your own addressing"? Your page mentions you run two ASs on this, so "bring your own addressing" is a feature, I suppose?
Yes :)
If I don't have an AS (but I do have an ISP or VPS that gives me a /56) what might a typical way be to allocate an address or prefix to my laptop? Static routing?
This is exactly the target use case. You can configure the VPS to allow traffic egress for the /56, and arbitrary addresses from that prefix can be added to devices connected to your VPS (in some sense, yes, static routing, since you do need to manually list address on nodes that want to use the addresses in the prefix)
If I travel internationally and want to avoid round trips to my home when accessing my mesh nodes, how might I do that? Is this the "faster to route via another node, than to go direct" case?
Whichever route have the lowest metrics will be used, be it direct or going thru multiple hops
btw this is already running on all my machines (including a macbook), and i've configured all of them to have routable public ip addresses ;)
especially for the mac, my ISP is SFR (over coaxial connection) with only dynamic ipv4, i can use this to get stable v4/v6 addresses very easily for all my devices behind the router
tuxes | 22 hours ago
I'm pretty dissatisfied with all the mesh VPNs options, and am closer to "no mesh; only IPv6 and workload mTLS" than adopting a new mesh, but I'm keen to learn more.
From this I infer your project is some sort of alternative to the various mesh VPNs, which is helpful to orient my understanding. I'm not that familiar with much of this, so my questions probably make no sense.
In a typical mesh VPN, when might it be faster to route via another node, than to go direct?
In a typical mesh VPN, when might I want a policy to route via a chosen waypoint?
For the "chosen exit" case, is this similar to Tailscale exit nodes?
More generally, what are the limitations of this, compared to typical mesh VPNs?
I suppose it doesn't do NAT punching, but instead unreachable nodes (such as your MacBook) are meant to reach out to your reachable nodes, and this architecture will find the best path? (Best being policy or lowest RTT).
Is it end-to-end encrypted? i.e. If traffic from node A to node B is routed via a waypoint W, does W see the original packet, or the encrypted packet?
Where does addressing fit here? A typical mesh VPN gives nodes their own address, on a "flat" (?) network.
IIUC, ranet is "bring your own addressing"? Your page mentions you run two ASs on this, so "bring your own addressing" is a feature, I suppose?
If I don't have an AS (but I do have an ISP or VPS that gives me a /56) what might a typical way be to allocate an address or prefix to my laptop? Static routing?
If I travel internationally and want to avoid round trips to my home when accessing my mesh nodes, how might I do that? Is this the "faster to route via another node, than to go direct" case?
[OP] ysun | 13 hours ago
Me too! Especially so if you want to play with overlay network intensively
Yes. Not only this, I want ranet3 to interact nicely with other RFC compliant VPN (say any other ipsec apps on your phone or server) and routing daemons (say BIRD ‘s babel implementation)
Btw this already woks ;)
Maybe during provider outages if you are behind residential ISPs? or busy hours? I'm not sure exactly when multiple hops will be faster.
This behavior is actually a "nice byproduct" of wanting to have ranet3 on laptops (which moves between networks). Babel is more often used to track lossy links, I would want routable addresses configured on laptops to be withdrew quickly when its not reachable
This is probably fairly useless to normal users but quite standard in ISP deployments. For example some transit provider have lower cost and operators might want to send most of the traffic to that link instead of sending traffic to more expensive providers.
But you can interpret this as traffic can take arbitrary routes you asked for, or the route with lowest metric value
Exactly
This is planned and will be implemented very soon
No. Transit node traffic are decrypted and re-encrypted (I mean it's just a "routed mesh" right now). Also I think making it "real e2ee" will result in some performance penalties (in the sense of ESP in ESP) and losing some information of the packet (say clamping TCP MSS)
One argument I would make is that, (similar to Tailscale and other existing solutions) you should probably trust the nodes you add to the mesh...
You can use arbitrary addresses, if you prefer tailscale like behavior you can just carve out the entire CGNAT range or subset of it, or go IPv6 only
Yes :)
This is exactly the target use case. You can configure the VPS to allow traffic egress for the /56, and arbitrary addresses from that prefix can be added to devices connected to your VPS (in some sense, yes, static routing, since you do need to manually list address on nodes that want to use the addresses in the prefix)
Whichever route have the lowest metrics will be used, be it direct or going thru multiple hops
[OP] ysun | a day ago
btw this is already running on all my machines (including a macbook), and i've configured all of them to have routable public ip addresses ;)
especially for the mac, my ISP is SFR (over coaxial connection) with only dynamic ipv4, i can use this to get stable v4/v6 addresses very easily for all my devices behind the router
ios/android app soon^{TM}