[RFC PATCH 30/30] Documentation/networking: Document net policy
From: kan . liang
Date: Mon Jul 18 2016 - 10:23:51 EST
From: Kan Liang <kan.liang@xxxxxxxxx>
Signed-off-by: Kan Liang <kan.liang@xxxxxxxxx>
Documentation/networking/netpolicy.txt | 158 +++++++++++++++++++++++++++++++++
1 file changed, 158 insertions(+)
create mode 100644 Documentation/networking/netpolicy.txt
diff --git a/Documentation/networking/netpolicy.txt b/Documentation/networking/netpolicy.txt
new file mode 100644
@@ -0,0 +1,158 @@
+What is Linux Net Policy?
+It is a big challenge to get good network performance. First, the network
+performance is not good with default system settings. Second, it is too
+difficult to do automatic tuning for all possible workloads, since workloads
+have different requirements. Some workloads may want high throughput. Some may
+need low latency. Last but not least, there are lots of manual configurations.
+Fine grained configuration is too difficult for users.
+"NET policy" intends to simplify the network configuration and get a
+good network performance according to the hints(policy) which is applied by
+user. It provides some typical "policies" for user which can be set
+per-socket, per-task or per-device. The kernel will automatically figures out
+how to merge different requests to get good network performance.
+"Net policy" is designed for multiqueue network devices. This document
+describes the concepts and APIs of "net policy" support.
+NET POLICY CONCEPTS
+Scope of Net Policies
+ Device net policy: this policy applies to the whole device. Once the
+ device net policy is set, it automatically configures the system
+ according to the applied policy. The configuration usually includes irq
+ affinity, irq balance disable, interrupt moderation, and so on. But the
+ device net policy does not change the packet direction.
+ Task net policy: this is a per-task policy. When it is applied to specific
+ task, all packets transmissions of the task will be redirect to the
+ assigned queues accordingly. If a task does not define a task policy,
+ it "falls back" to the system default way to direct the packets. The
+ per-task policy must be compatible with device net policy.
+ Socket net policy: this is a per-socket policy. When it is applied to
+ specific socket, all packets transmissions of the socket will be redirect
+ to the assigned queues accordingly. If a socket does not define a socket
+ policy, it "falls back" to the system default way to direct the packets.
+ The per-socket policy must be compatible with both device net policy and
+ per-task policy.
+Components of Net Policies
+ Net policy object: it is a combination of cpu and queue. The queue irq has
+ to set affinity with the cpu. It can be shared between sockets and tasks.
+ A reference counter is used to track the sharing number.
+ Net policy object list: each device policy has an object list. Once the
+ device policy is determined, the net policy object will be inserted into
+ the net policy object list. The net policy object list does not change
+ unless the cpu/queue number is changed, the netpolicy is disabled or
+ the device policy is changed.
+ The network performance for objects could be different because of the
+ queue/cpu topology and dev location. The objects which can bring high
+ performance are in the front of the list.
+ RCU hash table: a RCU hash table to maintain the relationship between
+ the task/socket and the assigned object. The task/socket can get the
+ assigned object by searching the table.
+ If it is the first time, there is no assigned object in the table. It will
+ go through the object list to find the available object based on position
+ and reference number.
+ If the net policy object list changes, all the assigned object will become
+NET POLICY APIs
+Interfaces between net policy and device driver
+ int (*ndo_netpolicy_init)(struct net_device *dev,
+ struct netpolicy_info *info);
+ The device driver who has NET policy support must implement this interface.
+ In this interface, the device driver do necessory initialization, and fill
+ the info for net policy module. The information could inlcude supported
+ policy, MIX policy support, queue pair support and so on.
+ int (*ndo_get_irq_info)(struct net_device *dev,
+ struct netpolicy_dev_info *info);
+ This interface is used to get more accurate device irq information.
+ int (*ndo_set_net_policy)(struct net_device *dev,
+ enum netpolicy_name name);
+ This interface is used to set device net policy by name
+Interfaces between net policy and kernel
+ int netpolicy_register(struct netpolicy_reg *reg);
+ void netpolicy_unregister(struct netpolicy_reg *reg);
+ This interface is used to register per task/socket net policy.
+ If it's the first time to register, an record will be created and inserted
+ into RCU hash table. The record includes ptr, policy and object
+ information. There is only one user for each record. The record cannot be
+ int netpolicy_pick_queue(struct netpolicy_reg *reg, bool is_rx);
+ This interface is used to find the proper queue(object) for packet
+ receiving and transmitting. The proper queue is picked from object list
+ according to policy, reference, location and so on.
+ int netpolicy_set_rules(struct netpolicy_reg *reg,
+ u32 queue_index,
+ struct netpolicy_flow_spec *flow);
+ This interface is used to add device specific rules. Once the rule is
+ applied, the packet from specific IP and port will be redirect to the
+ given queue. This interface usually be used in receive side.
+NET POLICY INTERFACE
+Device net policy setting
+ Concatenating(cat) the "policy" file can show the available device
+ policies, if there is no device policy applied. Otherwise, the device
+ policy name will be print out. If it is MIX policy, the policy for each
+ queue will also be print out.
+ User can set device net policy by writing policy name.
+Task policy setting
+ Concatenating(cat) the "net_policy" file can show the applied per task
+ User can set per task net policy by writing policy name.
+ prctl(PR_SET_NETPOLICY, POLICY_NAME, NULL, NULL, NULL)
+ "prctl" is an alternative way to set/get per task policy.
+Socket policy setting
+ The socket net policy can be set by option SO_NETPOLICY of setsockopt
+AVAILABLE NET POLICIES
+ The available net policies are defined as below:
+ - CPU: intends to get higher throughput and lower CPU%. This policy can be
+ applied as either device net policy or task/socket net policy.
+ - BULK: intends to get highest throughput. This policy can be applied as
+ either device net policy or task/socket net policy.
+ - LATENCY: intends to get lowest latency. This policy can be applied as
+ either device net policy or task/socket net policy.
+ - MIX: combination of other policies, which allows each queue has
+ different policy. This policy can only be set as device net policy.