This made me spend a few minutes on another try at setting up an IPv6 tunnel (since Swedish ISP:s seem to want to ignore IPv6 forever) for my Unifi network - and it seems I was successful! Living in the future at last…
edit: but apparently the tunnel is considered suspicious by various geo locking schemes, bah.
I began this journey in December 2020, so it was over the last 14 months. Trying to think back about it, I’d say
10 hours to find how to do it (i.e. pick the RIR, pick a sponsoring org etc)
3 hours to deal administratively with Grifon/RIPE
30 hours to find/deal with hosting providers, installing a new VM (I had two VMs I haven’t anymore because of the unsatisfying quality of service)
8 hours learning more than the basic traffic engineering I knew
15 hours doing sys/netadmin, writing ansible stuff, moving from my initial simple network tunneling setup to this bgp mesh tunneling setup
5 hours writing my igpd before I aborted
30 hours writing this blog post haha, this took so loooong
+/- 25% for each, since I’m not sure how long it took me
Playing with BGP was probably between 1 and 5% of the overall time. BGP is quite simple and works very reliably, which is both a curse and a blessing :)
This is a great post, thanks for sharing it! I started my own AS a few years ago, but the “kick start” was receiving a v4 block from ARIN after being on their waiting list for 2 or 3 years… Once I had a block, I was able to announce to my upstream provider, but rather than a VM, it was from a dedicated server at a cheap colo DC. It has now grown, and I host a friends’ business internet, and have also joined the local internet exchange. It was an awesome learning experience so far. Cheers!
is this using provider independent (PI) IPv6 space
For my two /48s from RIPE territory, one is PI the other is PA. Not sure if something exist in ARIN land, but it’s like a PA.
would you be able to get your residential ISP to route this to your home?
I really don’t think so. My ISP and the ISP which own the last mile, both have IPv6 prefixes, so me not having IPv6 access is not because of a lack of prefix.
My understanding of how it works in IPv4, is my ISP “gives” the IP addresses to the last mile owner so they can do the put them on the last mile equipment.
So it would mean I would need to give my ISP the IPv6 who would then give them to the last mile owner.
Very few (if any?) eyeball networks (consumer ISPs) will send you full routes. Most people’s routers can barely route with a default, let alone handle the 3GB of RAM for each full table from ISPs.
If you bumped up to commercial you may be able to find something? It’s going to set you back so much more than anything else because of the nature of peering.
Sprint/T-Mobile and Comcast can get you circuits with BGP over them, but they obviously are not home style circuits (or even the basic commercial level of their normal service). Also, you’re still on an eyeball network, so you’re not getting much advantage from using BGP except for very basic fail over (losing all wireguard links from the article).
The third aspect of this is that they are already advertising from near home. Most ISPs will only allow you to specify continent level preference. Their home internet probably doesn’t have a shorter path to anything than the VPC in Toronto except their own eyeball network. It would probably never be preferred for ingress. Then egress may all dump to the home ISP and you’d have to deal with asymmetric disputes (incoming ISP is fine with third ISP that home ISP is having a dispute with) and outages (eyeball network ISP ‘s router is oversubscribed and dropping outgoing packets only). You’re just going to depref all routes from that ISP anyhow. It would be fine as a third tier of peering usable as a fallback. Especially if you path prepend a bunch for incoming and depref locally for outgoing.
As mentioned in my infrastructure blog post, I have multiple networks (VLAN) at home. Because I didn’t want to do some unholy things, I needed to have a /64 per network, meaning multiple /64s for my home.
I don’t understand this part. Why can’t he split the network into multiple segments? What use case does anyone have for multiple /64 in their home? That’s 18446744073709551616 addresses per subnet.
Because that’s the only functioning way we’ve been able to come up with for devices to be able to be able to automatically configure themselves with a predictable persistent address without any conflicts.
The issue is that people seem to have a hard time comprehending just how big of a number 2^128 is. With that address space we could for example assign 2^32 /64’s to each IPv4 address (of which there are 2^32). We can give the entire IPv4 address space to each IPv4 address.
Additionally, RIPE strongly discourages assigning prefixes longer than /56, and in general recommends assigning end-customers a /48 or /56, and that assigning a /48 to all customers is the most practical address plan.
Thanks for the explanation. Indeed these numbers are just too large to properly imagine them…
Additionally, RIPE strongly discourages assigning prefixes longer than /56, and in general recommends assigning end-customers a /48 or /56, and that assigning a /48 to all customers is the most practical address plan.
Well, at least in Germany consumer ISPs seem to hand out /64 by default, though. I suppose one can ask to get a /48 or /58, though.
Just checked it, from Telekom I get a /56 without any interaction. As far as I know as a consumer you can ask for a /48 and as a commercial customer you just get a /48. A few years ago there was a news about Telekom asking for a bigger prefix then a default ISP get, because they wanted to follow the RIPE guidelines. As the biggest ISP in Germany they could argue this. As far as I know most ISP in Germany does this similar.
I’m not sure how it is handled for mobile access. As far as I know you get default slaac in a provider managed /64 and can request prefixes per dhcpv6. I can’t check this, because I don’t have mobile Internet.
Thanks for the comment, I guess I should update my post to be more precise about what the problem is. As kyrias explained, it’s not the number of addresses, but to be able to use SLAAC.
I love the article, but one minor point I didn’t quite follow — If your preferred solution was native IPv6, which is presumably unencrypted, why was lack of tunnel encryption a deal-breaker?
The tunnel endpoint would have likely been “far” network wise. So if I visit e.g. a no-tls website, the non encrypted packets would take a longer path (so the security risk increases) than it would have with native IPv6.
That said, my homemade routing may make the path longer than using a tunnel from professionals like HE.
This made me spend a few minutes on another try at setting up an IPv6 tunnel (since Swedish ISP:s seem to want to ignore IPv6 forever) for my Unifi network - and it seems I was successful! Living in the future at last…
edit: but apparently the tunnel is considered suspicious by various geo locking schemes, bah.
it reads so simple. I’d love to know how long this whole endeavor took you.
I began this journey in December 2020, so it was over the last 14 months. Trying to think back about it, I’d say
+/- 25% for each, since I’m not sure how long it took me
Playing with BGP was probably between 1 and 5% of the overall time. BGP is quite simple and works very reliably, which is both a curse and a blessing :)
This is a great post, thanks for sharing it! I started my own AS a few years ago, but the “kick start” was receiving a v4 block from ARIN after being on their waiting list for 2 or 3 years… Once I had a block, I was able to announce to my upstream provider, but rather than a VM, it was from a dedicated server at a cheap colo DC. It has now grown, and I host a friends’ business internet, and have also joined the local internet exchange. It was an awesome learning experience so far. Cheers!
Appreciate the detailed break down. Thank you! :)
So is this using provider independent (PI) IPv6 space, and would you be able to get your residential ISP to route this to your home?
Yes
that would be very cool, but the post is using tunnels (wireguard) to bring the network from their datacenter based VM’s to their home
blake is right but I’ll try to give more details:
For my two /48s from RIPE territory, one is PI the other is PA. Not sure if something exist in ARIN land, but it’s like a PA.
I really don’t think so. My ISP and the ISP which own the last mile, both have IPv6 prefixes, so me not having IPv6 access is not because of a lack of prefix.
My understanding of how it works in IPv4, is my ISP “gives” the IP addresses to the last mile owner so they can do the put them on the last mile equipment. So it would mean I would need to give my ISP the IPv6 who would then give them to the last mile owner.
Very few (if any?) eyeball networks (consumer ISPs) will send you full routes. Most people’s routers can barely route with a default, let alone handle the 3GB of RAM for each full table from ISPs.
If you bumped up to commercial you may be able to find something? It’s going to set you back so much more than anything else because of the nature of peering.
Sprint/T-Mobile and Comcast can get you circuits with BGP over them, but they obviously are not home style circuits (or even the basic commercial level of their normal service). Also, you’re still on an eyeball network, so you’re not getting much advantage from using BGP except for very basic fail over (losing all wireguard links from the article).
The third aspect of this is that they are already advertising from near home. Most ISPs will only allow you to specify continent level preference. Their home internet probably doesn’t have a shorter path to anything than the VPC in Toronto except their own eyeball network. It would probably never be preferred for ingress. Then egress may all dump to the home ISP and you’d have to deal with asymmetric disputes (incoming ISP is fine with third ISP that home ISP is having a dispute with) and outages (eyeball network ISP ‘s router is oversubscribed and dropping outgoing packets only). You’re just going to depref all routes from that ISP anyhow. It would be fine as a third tier of peering usable as a fallback. Especially if you path prepend a bunch for incoming and depref locally for outgoing.
I don’t understand this part. Why can’t he split the network into multiple segments? What use case does anyone have for multiple /64 in their home? That’s 18446744073709551616 addresses per subnet.
Each network need to be /64 for SLAAC to work.
But that’s… a giant amount of addresses. Why is this? Not allowing smaller sizes looks as if we’re repeating the IPv4 mistakes?
Because that’s the only functioning way we’ve been able to come up with for devices to be able to be able to automatically configure themselves with a predictable persistent address without any conflicts.
The issue is that people seem to have a hard time comprehending just how big of a number 2^128 is. With that address space we could for example assign 2^32 /64’s to each IPv4 address (of which there are 2^32). We can give the entire IPv4 address space to each IPv4 address.
Additionally, RIPE strongly discourages assigning prefixes longer than /56, and in general recommends assigning end-customers a /48 or /56, and that assigning a /48 to all customers is the most practical address plan.
Thanks for the explanation. Indeed these numbers are just too large to properly imagine them…
Well, at least in Germany consumer ISPs seem to hand out /64 by default, though. I suppose one can ask to get a /48 or /58, though.
Vodafone (previously Unitymedia) gives a /56 by default for IPv6-only cable, so it’s not uncommon.
When have consumer ISPs ever been known to follow guidelines. ;)
Are you sure it’s the ISP specifically only giving a /64 and not the DHCP-PD client only taking a /64 out of the available /56?
That might actually be it. Sorry for the noise.
Just checked it, from Telekom I get a /56 without any interaction. As far as I know as a consumer you can ask for a /48 and as a commercial customer you just get a /48. A few years ago there was a news about Telekom asking for a bigger prefix then a default ISP get, because they wanted to follow the RIPE guidelines. As the biggest ISP in Germany they could argue this. As far as I know most ISP in Germany does this similar.
I’m not sure how it is handled for mobile access. As far as I know you get default slaac in a provider managed /64 and can request prefixes per dhcpv6. I can’t check this, because I don’t have mobile Internet.
Thanks for the comment, I guess I should update my post to be more precise about what the problem is. As kyrias explained, it’s not the number of addresses, but to be able to use SLAAC.
IoT is old, new is IoA (internet of atoms).
I used to work for a now defunct IoT startup. Your comment wounds/intrigues me.
I love the article, but one minor point I didn’t quite follow — If your preferred solution was native IPv6, which is presumably unencrypted, why was lack of tunnel encryption a deal-breaker?
Good point!
The tunnel endpoint would have likely been “far” network wise. So if I visit e.g. a no-tls website, the non encrypted packets would take a longer path (so the security risk increases) than it would have with native IPv6.
That said, my homemade routing may make the path longer than using a tunnel from professionals like HE.
It’s probably not a good argument haha