If you’re not familiar with TRex, it’s a pretty cool open-source project put out by Cisco. Its goal is to be a realistic and highly scalable traffic generator that can be used in a variety of situations. I’ve been using it a lot lately to test traffic ingests on network sensors, but recently needed to test throughput rates going through a firewall.
For this test, I’m running pfSense 2.6.0 on a 16-core server with 128 GB of RAM. The server has both 1 Gbps Ethernet interfaces as well as 10 Gbps SFP interfaces. Right now, I’m just trying to get the stupid thing working so will be testing using the 1 Gbps Ethernet interfaces.
From a physical cabling standpoint, below is a quick diagram of what I’ve got set up.
I won’t go into the configuration of the pfSense. Because I was interested more in basic routing throughput instead of complex firewall rules, I simply added an allow-all rule to each interface. I deliberately didn’t use the LAN or WAN ports on the pfSense just to make sure that things like NAT didn’t get in the way of this initial test.
I configured TRex with the below configuration. The interfaces listed there correspond to eno1 and eno2 on the TRex box.
### Config file generated by dpdk_setup_ports.py ### - version: 2 interfaces: ['67:00.0', '67:00.1'] port_info: - ip: 220.127.116.11 default_gw: 18.104.22.168 - ip: 22.214.171.124 default_gw: 126.96.36.199 platform: master_thread_id: 0 latency_thread_id: 31 dual_if: - socket: 0 threads: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
With that in place, I gave it a test and immediately found that I was dropping everything. I couldn’t figure out why.
I was pretty sure that everything was set up correctly, so I decided to run a quick packet capture on the pfSense and see what it was seeing.
I failed to remember that just because I set up TRex to have the correct IP address on its interfaces, the packets that it generates actually “come” and “go” to other subnets. A quick addition of two new gateways on pfSense (one for each interface), along with corresponding static routes for both interfaces on the TRex box, and we’re in business.