RE: [PATCH v9 net-next 1/2] hv_sock: introduce Hyper-V Sockets

[Dexuan Cui]

> From: David Miller [mailto:davem@xxxxxxxxxxxxx]
> Sent: Monday, May 9, 2016 1:45
> To: Dexuan Cui <decui@xxxxxxxxxxxxx>
> Cc: gregkh@xxxxxxxxxxxxxxxxxxx; netdev@xxxxxxxxxxxxxxx; linux-
> kernel@xxxxxxxxxxxxxxx; devel@xxxxxxxxxxxxxxxxxxxxxx; olaf@xxxxxxxxx;
> apw@xxxxxxxxxxxxx; jasowang@xxxxxxxxxx; cavery@xxxxxxxxxx; KY
> Srinivasan <kys@xxxxxxxxxxxxx>; Haiyang Zhang <haiyangz@xxxxxxxxxxxxx>;
> joe@xxxxxxxxxxx; vkuznets@xxxxxxxxxx
> Subject: Re: [PATCH v9 net-next 1/2] hv_sock: introduce Hyper-V Sockets
>
> From: Dexuan Cui <decui@xxxxxxxxxxxxx>
> Date: Sun, 8 May 2016 06:11:04 +0000
>
> > Thanks for pointing this out!
> > I understand, so I think I should add a module parameter, e.g.,
> > "hv_sock.max_socket_number" with a default value, say, 1024?
>
> No, you should get rid of the huge multi-page buffers.

Hi David,
Ok, how do you like the below proof-of-concept patch snippet?

I use 1 page for the recv buf and another page for send buf. They should be
allocated by kmalloc(sizeof(struct hvsock_send/recv_buf), GFP_KERNEL).

And, by default, I use 2 pages for VMBUS send/recv ringbuffers respectively.
(Note: 2 is the minimal ringbuffer size because actually 1 page of the two is used
as the shared read/write index etc, rather than data)
A module parameter will be added to allow the user to use a big ringbuffer
size, if the user cares too much about the performance.

Another parameter will be added to limit how many hvsock sockets can be
created at most. The default value can be 1024, meaning at most
1024 * (2+2+1+1) * 4KB = 24MB memory is used.

-#define VMBUS_RINGBUFFER_SIZE_HVSOCK_RECV (5 * PAGE_SIZE)
-#define VMBUS_RINGBUFFER_SIZE_HVSOCK_SEND (5 * PAGE_SIZE)
+#define VMBUS_RINGBUFFER_SIZE_HVSOCK_RECV (2 * PAGE_SIZE)
+#define VMBUS_RINGBUFFER_SIZE_HVSOCK_SEND (2 * PAGE_SIZE)

-#define HVSOCK_RCV_BUF_SZ      VMBUS_RINGBUFFER_SIZE_HVSOCK_RECV
+#define HVSOCK_RCV_BUF_SZ      PAGE_SIZE
 #define HVSOCK_SND_BUF_SZ      PAGE_SIZE

+struct hvsock_send_buf {
+       struct vmpipe_proto_header hdr;
+       u8 buf[HVSOCK_SND_BUF_SZ];
+};
+
+struct hvsock_recv_buf {
+       struct vmpipe_proto_header hdr;
+       u8 buf[HVSOCK_RCV_BUF_SZ];
+
+       unsigned int data_len;
+       unsigned int data_offset;
+};
+
@@ -35,21 +48,8 @@ struct hvsock_sock {

        struct vmbus_channel *channel;

-       struct {
-               struct vmpipe_proto_header hdr;
-               u8 buf[HVSOCK_SND_BUF_SZ];
-       } send;
-
-       struct {
-               struct vmpipe_proto_header hdr;
-               u8 buf[HVSOCK_RCV_BUF_SZ];
-
-               unsigned int data_len;
-               unsigned int data_offset;
-       } recv;
+       struct hvsock_send_buf *send_buf;
+       struct hvsock_recv_buf *recv_buf;
 };

Thanks,
-- Dexuan