Tyblog | Building my ideal router for $50

9

I’m glad to hear the build is running for you, that’s great!

I forgot to mention that the AUR PKGBUILD for ipt-netflow doesn’t currently list aarch64 as a supported architecture. If you edit the arch list to include aarch64 manually, the build should compile, install, and run just fine on the espressobin.

It’s been many months since I’ve started using netflow monitoring on the firewall at this point without any problems whatsoever, so I think aarch64 is a supported architecture for the module - I’ve just commented on the AUR package to request that aarch64 be added, so hopefully the maintainer does so in order to avoid making people add the architecture manually.

10

Thank you for your answer. After including ‘aarch64’ in the arch list, building the package went smoothly.
I followed your tuto to create the .conf files and loaded the ipt_NETFLOW module.
I got a warning: ‘ipt_NETFLOW: loading out-of-tree module taints kernel’, but no error.
I could not verify the content of the capture, because Arch Linux does not have a Logstash package. There is a Elasticsearch package, but I have still to look if it is the full stack (with Logstash) and find how to use it.
Anyway, I could verify with netcat that some data is sent to the port 2055 of my server:
netcat -u -l -p 2055 > netflow.capture

11

Update to my former post.
There is an Arch Linux Logstash package for x86_64 architecture. I didn’t found it at first, because I was searching for my home server, an ODROID-HC1, whose arch=armv7h.
So, after installing the ELK stack (Elasticsearch, Logstash, Kibana) on a x86_64 machine, and starting the services, I could resume the ‘Netflow Monitoring’ section of your tuto, and have those nice Kibana reports. I was really impressed to have all this working. Hopefully, you gave sufficient clues to know where to go.
Many thanks to you for having written this tuto.

12

About building ipt_NETFLOW DKMS module on Arch Linux:

For information:

  • there was an error building ipt_NETFLOW DKMS module with kernel linux-espressobin 4.17.0 … 4.17.4
  • good news: building the module is anew ok with kernel linux-espressobin 4.17.5 and (today) 4.17.6
13

Wow! its an amazing guidance and a beneficial blog .It is quite amazing and the way you explained it step by step it was fabulous.I have been looking for a walkthrough like it for a very long time but couldnt find a proper one and at last I finally found it and also it is amazing.You are a great blogger.williamjacket

14

That’s an exquisite piece of work! Can I know what modem do you use with this build?

15

Hi @EdenSajid, pretty much any modem will work with this setup, as long as the espressobin can get a DHCP address. At the moment I’m using this Netgear modem, which as been running for about a year.

16

@tylerjl Thank you so much for the tutorial. It’s been great playing with the espressobin! My current setup is practically identical to yours minus all the NETFLOW stuff. I’m noticing a huge reduction in speedtest when switching from my current pfsense router to the espressobin. I typically get 200down/11up with pfsense. With espressobin I’m getting 45down/15up.
I’ve gone over all the tcdevices and tcclasses and don’t see why it’s being limited. Any thoughts?

Thanks

17

Hmm, I haven’t observed any significant slowdowns moving from a traditional router to a homebrew espressobin. I will say that the espressobin drivers and upstream support is still pretty active/in development - what distribution/kernel are you running?

18

Linux alarm 4.18.14-1-ARCH #1 SMP Sat Oct 13 18:35:28 MDT 2018 aarch64 GNU/Linux

I followed your tutorial almost to the letter. I used http://wiki.espressobin.net/tiki-index.php?page=Boot+from+removable+storage+-+ArchLinux to setup the sdcard boot device.

19

Any of your updates break the router so far or have you been stable? How often do you update?

20

I update the router about once a month, just to ensure all the relevant packages are kept current with upstream. So far the only breakages have been in kernel incompatibilities with the ipt-netflow module, but I think that’s only happened once so far - any Arch updates to shorewall, dnsmasq, etc. have been stable.

21

First let me thank you for that great blog post!

Over the weekend I tried to follow you guide but I came across a curious, at least for me, problem. Ip forwarding seems only to work if I start the network service after the shorewall service. On the clients when I try to traceroute any webaddress, im stuck at the router, after stopping and starting systemd-networkd.service the route completes. Is this normal behaviour and I just did something wrong, or did you take care that the network service starts after the shorewall?

22

Hmm, I haven’t modified either the systemd-networkd service or the shorewall service. I believe that indicating the wan and br0 interfaces should be configured for IPv4 is sufficient - here’s my relevant files:

[root@router ~]$ cat /etc/systemd/network/br0.network
[Match]
Name=br0

[Network]
Address=192.168.1.1/24
IPForward=ipv4
IPMasquerade=yes
ConfigureWithoutCarrier=yes
[root@router ~]$ cat /etc/systemd/network/wan.network
[Match]
Name=wan

[Network]
IPv6AcceptRA=no
DHCP=ipv4
BindCarrier=eth0
IPForward=ipv4
[root@charon ~]#

As far as I know, this should set the interfaces into a forwarding state.

23

Any particular reason you went for classfull qdisc rather than e.g. cake?

The piece-of-cake recipe from openwrt works very well for me, but, otoh, I don’t really know what it’s doing.

24

I read up on traffic shaping strategies for a long time before finally just settling on using eqhmcow’s strategy primarily because I felt like I understood the class choices and it seemed to fit my use case.

I suspect that latest-and-greatest traffic shaping algorithms (used to be codel, now cake) may achieve the same results, honestly. My understanding is that the tradeoff is router load, but after my time with the espressobin, I certainly think that it could handle the additional processing power.

My hope is that I can provide a revisit of my router setup after a while and spend some time benchmarking cake versus other options to verify whether it works, but if anyone wants to experiment, my understanding is that the module(s) are readily available from the Arch repositories/AUR.

25

Thanks for a nice post!

I did something similar - built my router using an Espressobin. I used Ubuntu rather than arch.
Works very nicely. Today I’ve had fiber based Internet service installed - 1Gb/s downsteam (replacing the meager 50-60Mb/s I had previously). Suddenly I hit a performance wall. The board goes to high CPU when I speedtest it, and hits a ceiling at ~125Mb/s.
Looking at what’s going on I see pppoe using very high CPU. Tried to upgrade rp-pppoe, but to no avail.
Any thoughts on that one?

EDIT: Typically after giving up on solving it and posting, you hit the solution. So I did. I’ve been using userland pppoe - and sure enough, it hits userland limits. Once I moved to kernel pppoe, Performance skyrocketed to what I believe is the ISP limit - at this moment, well over 500Mb/s.

27

It seems I am also having speed issues as well with lan to wan traffic. Lan to Lan I am getting gigabit, from the Espressobin OS console outbound I am getting gigabit, but from a machine plugged into the lan to a machine hooked directly into the wan port I am only getting about a 3rd of a gig. Have you tried a bandwidth test on your router with something capable of gigabit speeds? I am getting these speeds from stock installs of Arch and Armbian, with no mods and no FW…

I saw doron’s comment, I do not believe the issue is pppoe on mine, at least I am not seeing that in the logs. Any ideas?

28

Sorry - didnt reply to doron correctly, reposing
If you don’t mind me asking, what did you do to upgrade it? I am running into the exact same issue, as in hitting a ceiling…

29

@RickS - what I did is change my pppoe configuration template. It had a “pty” line calling /usr/sbin/pppoe with some flags, which translates to running pppoe as a userland process. (When I was on VDSL, it worked just fine; when I connected to FTTH, CPU went to the roof and performance was blocked.)

Instead, I –

  1. Made sure my kernel is compiled with PPP and PPPOE (as modules, in my case, but that’s unimportant)
  2. Remove the pty lines from my pppoe configuration
  3. Add “plugin rp-pppoe.so” to that configuration

This makes pppoe use the kernel module. Performance upped 5-7 times once I did that.

Hope this helps!