Thanks for a nice post!
I did something similar - built my router using an Espressobin. I used Ubuntu rather than arch.
Works very nicely. Today I’ve had fiber based Internet service installed - 1Gb/s downsteam (replacing the meager 50-60Mb/s I had previously). Suddenly I hit a performance wall. The board goes to high CPU when I speedtest it, and hits a ceiling at ~125Mb/s.
Looking at what’s going on I see pppoe using very high CPU. Tried to upgrade rp-pppoe, but to no avail.
Any thoughts on that one?
EDIT: Typically after giving up on solving it and posting, you hit the solution. So I did. I’ve been using userland pppoe - and sure enough, it hits userland limits. Once I moved to kernel pppoe, Performance skyrocketed to what I believe is the ISP limit - at this moment, well over 500Mb/s.