Sorry for the late reply! was OOO... See below my comments.
1) Theoretically, I would like to steer the packet depending on information encoded in the packet, not the header. In any case, this information can be shrunk to 32 bits so as to be placed in the header (e.g., in the destination port field). Doing a simple hash, however, is not enough, as the mapping field_in_the_header -> queue could change over time.
[GB] Steering and RSS according to the payload for ConnectX-3 are not supported sadly.
2) Yes I have read much about DPDK and it seems I could leverage it for higher performance.
About other things you have said.
- As far as I have understood, flow director uses information on the header, but then the steering is done in hardware either according to a hard-wired hash function or using a 1:1 perfect filter. Therefore, I am not sure I can do what I want since I cannot afford informing the sender about the "header--> core" mapping (to implement 1:1), if this changes often over time, or if this map is huge. This is why I actually would like to change the hash function, to make it a "non-hash" function, but a custom steering one. I did not expect this to be doable in hardware, but I was wondering whether it could be done even before the packet is read, in software, by the dpdk library. Looking around, I guess this is not possible.
[GB] This is something i don't feel i can answer, probably need a DPDK expert
- Similarly, changing the RSS table would not be enough, as I would not have control over the hashing function.
- For now, I am committed to using DPDK to bypass the kernel, so using ethtool would not be useful for my purpose, as far as I have understood
[GB] Indeed, i was suggesting one or the other. Actually, in our model, ethtool still works when using DPDK but in practice you can't configure steering to the DPDK rx queues with ethtool.
- I would love to switch to ConnectX-4, but I cannot afford changing hardware right now.
My curiosity, though, still stands: if I were to give up on using DPDK, where should I put my hand in the mlx4 driver/linux kernel to implement a custom steering (either in RSS or RPS)?
[GB] First, our driver is open source and you are more than welcome to have a look at the code , i can try to help you with some pointers if needed. You can not change the HW to do this for you and i don't think the payload will be available for the generic steering you want so it will comes down to software layer.
In general, it would be great if you can contact me directly because i am am very interested in your use case and if we can find some workaround with our newer products (most likely not ConnectX-3).