Re: [linus:master] [x86/bugs] a3af84b0fa: stress-ng.seccomp.ops_per_sec 95.7% regression

From: Chengfeng Lin

Date: Wed Nov 04 2026 - 10:09:28 EST

Hi Dave, Pawan,

I collected CPU-clock profiles on the parent 96cce16e26dd,
a3af84b0fa00, and the parent again. Both kernels were built from
unmodified sources. The machine was an i7-12700KF with microcode 0x3e;
no protection helpers were used for these runs.

Most of the additional ZPOPMIN samples are in the kernel, particularly
on the server. The increase is spread across syscall and networking
code rather than one dominant function.

After each fresh boot, I ran the full MMTests recipe once without
profiling, then twice with CPU-clock sampling at 1999 Hz.

The table gives CPU-clock sample counts, averaged over the two profiles
per boot. Each profile sums the five native command-loop windows for
that command. These are separate diagnostic runs, not the unprofiled
throughput measurements.

ZPOPMIN Parent A Child Parent B
server, kernel 581.0 673.0 559.5
client, kernel 381.5 428.0 376.5
server, user 408.5 442.0 407.0
client, user 103.5 136.0 97.5
total 1474.5 1679.0 1440.5
GET, total 286.5 291.0 293.0

Using the mean of the two parent runs as the baseline, about 69% of
the additional samples are in kernel code. The attached CSV includes
all sampled symbol differences for ZPOPMIN and GET, with both parent
runs and unknown symbols retained. The five largest increases in
ZPOPMIN kernel samples, combining client and server, are below.
Increase = child - mean(parent A, parent B).

Parent A Child Parent B Increase
__tcp_transmit_skb() 29.0 44.5 32.5 13.75
do_syscall_64() 30.5 44.5 34.0 12.25
_raw_spin_unlock_irq() 15.5 26.5 19.0 9.25
skb_release_data() 17.5 24.5 16.0 7.75
release_sock() 1.5 8.5 1.5 7.00

For Pawan's microcode comparison, I kept the parent kernel 96cce16e26dd
and workload unchanged. Each microcode revision used control / pre-test
protection trigger / control, with nine unprofiled runs per boot. The
trigger was deliberate, not unexplained variation across boots.
ZPOPMIN throughput fell by 12.14% with 0x37 and 12.03% with 0x3e.
Control return drift was below 0.19%, and the maximum within-boot CV
was 1.40%. GET was essentially unchanged; the full 20-command test time
increased by 0.24%-0.27%.

Thanks,
Chengfeng
command,role,space,symbol,dso,parent_first,child,parent_return,baseline,delta_samples,return_delta_samples
GET,server,user,[unknown],libc.so.6,13.0,21.0,15.5,14.25,6.75,2.5
GET,server,user,__vdso_clock_gettime,[vdso],20.0,26.5,19.5,19.75,6.75,-0.5
GET,server,user,__vdso_gettimeofday,[vdso],14.0,17.5,12.0,13.0,4.5,-2.0
GET,server,user,je_free,redis-server,6.5,10.0,8.0,7.25,2.75,1.5
GET,client,user,readHandler,redis-benchmark,2.5,7.0,7.5,5.0,2.0,5.0
GET,client,kernel,populate_seccomp_data,[kernel.kallsyms],0,1.5,0,0.0,1.5,0
GET,server,kernel,_raw_spin_unlock_irqrestore,[kernel.kallsyms],0.5,2.5,1.5,1.0,1.5,1.0
GET,server,user,addReplyBulk,redis-server,0,2.0,1.0,0.5,1.5,1.0
GET,server,user,decrRefCount,redis-server,3.0,5.0,4.0,3.5,1.5,1.0
GET,server,user,processCommand,redis-server,3.5,4.0,2.0,2.75,1.25,-1.5
GET,client,kernel,do_epoll_wait,[kernel.kallsyms],0,1.0,0,0.0,1.0,0
GET,client,user,redisReaderGetReply,redis-benchmark,8.0,9.5,9.0,8.5,1.0,1.0
GET,server,user,addReplyLongLongWithPrefix,redis-server,1.0,1.5,0,0.5,1.0,-1.0
GET,server,user,authRequired,redis-server,0,1.0,0,0.0,1.0,0
GET,server,kernel,tcp_queue_rcv,[kernel.kallsyms],0,1.0,0,0.0,1.0,0
GET,client,user,aeProcessEvents.part.0,redis-benchmark,0,1.0,0.5,0.25,0.75,0.5
GET,client,kernel,fdget,[kernel.kallsyms],0,1.0,0.5,0.25,0.75,0.5
GET,server,user,checkType,redis-server,0.5,1.0,0,0.25,0.75,-0.5
GET,server,user,createStringObject,redis-server,0.5,1.0,0,0.25,0.75,-0.5
GET,server,user,hasActiveChildProcess,redis-server,0.5,1.0,0,0.25,0.75,-0.5
GET,server,user,memcmp@plt,redis-server,1.0,1.5,0.5,0.75,0.75,-0.5
GET,client,kernel,__local_bh_enable_ip,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,__mod_memcg_state,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,__sk_mem_raise_allocated,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,__sys_sendto,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,user,__vdso_clock_gettime,[vdso],0,0.5,0,0.0,0.5,0
GET,client,kernel,__x64_sys_epoll_wait,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,_copy_from_user,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,_copy_to_iter,[kernel.kallsyms],1.0,1.0,0,0.5,0.5,-1.0
GET,client,kernel,do_softirq.part.0,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,ep_remove_file.isra.0,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,free_tail_page_prepare,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,ipv4_dst_check,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,is_vmalloc_addr,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,user,memcpy@plt,redis-benchmark,0,0.5,0,0.0,0.5,0
GET,client,kernel,prandom_u32_state,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,rb_insert_color,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,remove_wait_queue,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,skb_release_data,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,tcp_ack,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,tcp_ack_update_rtt,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,tcp_poll,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,tcp_queue_rcv,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,tcp_rate_check_app_limited,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,user,ACLCheckCommandPerm,redis-server,0.5,1.0,0.5,0.5,0.5,0.0
GET,server,user,SentReplyOnKeyMiss,redis-server,0,0.5,0,0.0,0.5,0
GET,server,kernel,__dev_queue_xmit,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,__inet_lookup_established,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,__ip_queue_xmit,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,__rcu_read_unlock,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,__skb_datagram_iter,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,_copy_from_iter,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,_find_next_bit,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,aa_sk_perm,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,user,activeExpireCycle,redis-server,0,0.5,0,0.0,0.5,0
GET,server,user,addReply.part.0,redis-server,5.0,4.0,2.0,3.5,0.5,-3.0
GET,server,user,beforeSleep,redis-server,0,0.5,0,0.0,0.5,0
GET,server,kernel,begin_current_label_crit_section,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,user,commandProcessed,redis-server,1.0,1.0,0,0.5,0.5,-1.0
GET,server,user,connSocketRead,redis-server,0,0.5,0,0.0,0.5,0
GET,server,kernel,finish_task_switch.isra.0,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,user,lookupKeyReadOrReply,redis-server,1.5,2.0,1.5,1.5,0.5,0.0
GET,server,kernel,schedule_hrtimeout_range,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,user,string2ll,redis-server,0.5,1.0,0.5,0.5,0.5,0.0
GET,server,kernel,tcp_chrono_stop,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,tcp_rate_check_app_limited,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,tcp_skb_entail,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,server,kernel,x64_sys_call,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
GET,client,kernel,_copy_from_iter,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,kernel,clear_highpages_kasan_tagged,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,user,counts_index_for,redis-benchmark,0.5,1.0,1.0,0.75,0.25,0.5
GET,client,kernel,do_epoll_ctl_file,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,kernel,ep_remove,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,kernel,finish_task_switch.isra.0,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,user,malloc,libc.so.6,1.0,1.0,0.5,0.75,0.25,-0.5
GET,client,kernel,memset,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
GET,client,kernel,skb_defer_free_flush,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,client,kernel,tcp_event_new_data_sent,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,server,kernel,__check_object_size,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
GET,server,user,_addReplyToBuffer,redis-server,6.0,4.5,2.5,4.25,0.25,-3.5
GET,server,user,aeProcessEvents.part.0,redis-server,0,0.5,0.5,0.25,0.25,0.5
GET,server,user,createEmbeddedStringObject,redis-server,2.5,2.5,2.0,2.25,0.25,-0.5
GET,server,user,dictSdsCaseHash,redis-server,0.5,0.5,0,0.25,0.25,-0.5
GET,server,user,freeClientOriginalArgv,redis-server,0,0.5,0.5,0.25,0.25,0.5
GET,server,user,getExpire,redis-server,0.5,0.5,0,0.25,0.25,-0.5
GET,server,user,getMonotonicUs_posix,redis-server,2.0,1.5,0.5,1.25,0.25,-1.5
GET,server,user,processInputBuffer,redis-server,0.5,1.5,2.0,1.25,0.25,1.5
GET,server,user,readQueryFromClient,redis-server,0.5,0.5,0,0.25,0.25,-0.5
GET,server,kernel,skb_page_frag_refill,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
GET,server,kernel,skb_release_data,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
GET,server,user,writeCommandsDeniedByDiskError,redis-server,0,0.5,0.5,0.25,0.25,0.5
GET,server,user,writeToClient,redis-server,0.5,0.5,0,0.25,0.25,-0.5
GET,client,kernel,ep_send_events,[kernel.kallsyms],0.5,0.5,0.5,0.5,0.0,0.0
GET,client,user,hdr_record_value,redis-benchmark,0,0.5,1.0,0.5,0.0,1.0
GET,client,user,string2ll,redis-benchmark,1.0,0.5,0,0.5,0.0,-1.0
GET,server,user,clock_gettime@plt,redis-server,0,0.5,1.0,0.5,0.0,1.0
GET,server,user,dictFetchValue,redis-server,0,0.5,1.0,0.5,0.0,1.0
GET,server,user,getCommand,redis-server,0.5,0.5,0.5,0.5,0.0,0.0
GET,server,user,je_malloc,redis-server,5.0,4.5,4.0,4.5,0.0,-1.0
GET,server,user,lookupKeyReadWithFlags,redis-server,0.5,0.5,0.5,0.5,0.0,0.0
GET,server,user,replicationFeedMonitors,redis-server,0,0.5,1.0,0.5,0.0,1.0
GET,server,user,resetClient,redis-server,0.5,1.0,1.5,1.0,0.0,1.0
GET,server,user,stringObjectLen,redis-server,2.5,2.0,1.5,2.0,0.0,-1.0
GET,client,user,[unknown],libc.so.6,0.5,1.0,2.0,1.25,-0.25,1.5
GET,client,kernel,__account_obj_stock,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,__check_object_size,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,__rcu_read_unlock,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,__seccomp_filter,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,__sk_mem_schedule,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,__skb_datagram_iter,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,__tcp_transmit_skb,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,__zone_watermark_ok,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,_raw_spin_lock,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,_raw_spin_trylock,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,_raw_spin_unlock_irq,[kernel.kallsyms],0,0.5,1.5,0.75,-0.25,1.5
GET,client,kernel,dst_release,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,enqueue_to_backlog,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,ep_item_poll.isra.0,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,user,epoll_ctl,libc.so.6,0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,fput,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,user,freeReplyObject,redis-benchmark,0,0.5,1.5,0.75,-0.25,1.5
GET,client,user,hi_sdsclear,redis-benchmark,0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,hook_socket_sendmsg,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,ip_send_check,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,kmem_cache_alloc_noprof,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,ktime_get_ts64,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,kvfree_call_rcu,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,mem_cgroup_sk_charge,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,user,moveToNextTask,redis-benchmark,0.5,0,0,0.25,-0.25,-0.5
GET,client,user,random@plt,redis-benchmark,0.5,0,0,0.25,-0.25,-0.5
GET,client,user,redisReaderFeed,redis-benchmark,0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,schedule,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,skb_clone,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,kernel,sock_poll,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,tcp_do_parse_auth_options,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,tcp_rcv_space_adjust,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,tcp_recvmsg_locked,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,try_charge_memcg,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,client,kernel,x64_sys_call,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,__secure_computing,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,_copy_to_iter,[kernel.kallsyms],2.0,2.0,2.5,2.25,-0.25,0.5
GET,server,kernel,_raw_spin_lock_bh,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,bpf_skops_write_hdr_opt.isra.0,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,dst_release,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,ep_item_poll.isra.0,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,fdget_pos,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,user,gettimeofday@plt,redis-server,0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,inet_recvmsg,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,ip_finish_output,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,user,je_tcache_event_hard,redis-server,0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,lock_sock_nested,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,mod_timer,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,user,moduleCallCommandFilters,redis-server,0,0,0.5,0.25,-0.25,0.5
GET,server,user,moduleNotifyKeyspaceEvent,redis-server,0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,page_counter_try_charge,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,user,read@plt,redis-server,0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,select_estimate_accuracy,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,skb_release_head_state,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,user,slowlogPushEntryIfNeeded,redis-server,0.5,0.5,1.0,0.75,-0.25,0.5
GET,server,kernel,sock_recvmsg,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,sock_rfree,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,server,kernel,tcp_poll,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
GET,server,kernel,tcp_write_xmit,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
GET,client,user,__libc_calloc,libc.so.6,4.0,3.0,3.0,3.5,-0.5,-1.0
GET,client,kernel,__secure_computing,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
GET,client,user,createStringObject,redis-benchmark,1.0,1.0,2.0,1.5,-0.5,1.0
GET,client,user,random,libc.so.6,0.5,0,0.5,0.5,-0.5,0.0
GET,client,user,random_r,libc.so.6,1.0,0.5,1.0,1.0,-0.5,0.0
GET,server,user,LRU_CLOCK,redis-server,0.5,0,0.5,0.5,-0.5,0.0
GET,server,kernel,hook_socket_sendmsg,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
GET,server,user,memcpy@plt,redis-server,1.5,1.0,1.5,1.5,-0.5,0.0
GET,server,user,siphash_nocase,redis-server,2.5,2.5,3.5,3.0,-0.5,1.0
GET,server,kernel,sock_def_readable,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
GET,server,kernel,sock_write_iter,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
GET,server,user,strcasecmp@plt,redis-server,0,0,1.0,0.5,-0.5,1.0
GET,client,user,normalize_index,redis-benchmark,1.0,0,0.5,0.75,-0.75,-0.5
GET,client,user,redisGetReply,redis-benchmark,1.5,0.5,1.0,1.25,-0.75,-0.5
GET,client,user,writeHandler,redis-benchmark,8.0,8.0,9.5,8.75,-0.75,1.5
GET,server,user,ACLCheckAllPerm,redis-server,1.5,0.5,1.0,1.25,-0.75,-0.5
GET,server,user,clock_gettime,libc.so.6,0.5,0,1.0,0.75,-0.75,0.5
GET,server,user,dictSdsKeyCaseCompare,redis-server,0,0,1.5,0.75,-0.75,1.5
GET,server,user,expireIfNeeded,redis-server,2.0,0.5,0.5,1.25,-0.75,-1.5
GET,server,user,getGenericCommand,redis-server,1.0,0,0.5,0.75,-0.75,-0.5
GET,server,user,processMultibulkBuffer,redis-server,7.0,6.5,7.5,7.25,-0.75,0.5
GET,server,user,string2ll.part.0,redis-server,2.0,1.0,1.5,1.75,-0.75,-0.5
GET,client,kernel,do_syscall_64,[kernel.kallsyms],2.0,1.0,2.0,2.0,-1.0,0.0
GET,client,user,hdr_record_values,redis-benchmark,2.5,0.5,0.5,1.5,-1.0,-2.0
GET,client,kernel,mutex_lock,[kernel.kallsyms],1.5,0,0.5,1.0,-1.0,-1.0
GET,client,user,redisGetReplyFromReader,redis-benchmark,2.5,0.5,0.5,1.5,-1.0,-2.0
GET,server,user,prepareClientToWrite,redis-server,1.0,0,1.0,1.0,-1.0,0.0
GET,server,user,zmalloc_used_memory,redis-server,1.5,0,0.5,1.0,-1.0,-1.0
GET,server,user,addReply,redis-server,1.5,0,1.0,1.25,-1.25,-0.5
GET,server,user,siphash,redis-server,3.0,3.0,5.5,4.25,-1.25,2.5
GET,client,kernel,_raw_spin_unlock_irqrestore,[kernel.kallsyms],1.0,1.0,4.0,2.5,-1.5,3.0
GET,client,user,cfree,libc.so.6,4.5,2.5,4.5,4.5,-2.0,0.0
GET,server,user,zmalloc,redis-server,14.5,10.0,9.5,12.0,-2.0,-5.0
GET,server,user,lookupKey,redis-server,5.5,5.0,9.0,7.25,-2.25,3.5
GET,server,user,call,redis-server,8.5,7.0,11.5,10.0,-3.0,3.0
GET,server,user,zfree,redis-server,10.0,9.0,15.0,12.5,-3.5,5.0
GET,server,user,dictFind,redis-server,35.5,29.5,31.0,33.25,-3.75,-4.5
GET,server,user,dictSdsKeyCompare,redis-server,10.0,5.5,10.5,10.25,-4.75,0.5
GET,server,user,je_malloc_usable_size,redis-server,12.5,6.5,16.5,14.5,-8.0,4.0
ZPOPMIN,client,user,[unknown],libc.so.6,16.5,29.5,18.5,17.5,12.0,2.0
ZPOPMIN,server,kernel,skb_release_data,[kernel.kallsyms],15.0,24.0,14.5,14.75,9.25,-0.5
ZPOPMIN,client,kernel,__tcp_transmit_skb,[kernel.kallsyms],8.0,18.5,11.5,9.75,8.75,3.5
ZPOPMIN,server,user,[unknown],libc.so.6,110.0,127.5,130.0,120.0,7.5,20.0
ZPOPMIN,client,kernel,_raw_spin_unlock_irq,[kernel.kallsyms],9.5,17.5,11.0,10.25,7.25,1.5
ZPOPMIN,server,kernel,do_syscall_64,[kernel.kallsyms],11.5,19.0,12.0,11.75,7.25,0.5
ZPOPMIN,server,kernel,tcp_sendmsg_locked,[kernel.kallsyms],20.0,25.0,16.5,18.25,6.75,-3.5
ZPOPMIN,client,user,__libc_calloc,libc.so.6,6.5,13.5,8.5,7.5,6.0,2.0
ZPOPMIN,client,kernel,do_epoll_wait,[kernel.kallsyms],4.5,12.5,8.5,6.5,6.0,4.0
ZPOPMIN,server,user,connSocketWrite,redis-server,0,6.0,0,0.0,6.0,0
ZPOPMIN,server,kernel,do_softirq.part.0,[kernel.kallsyms],1.5,7.5,2.0,1.75,5.75,0.5
ZPOPMIN,client,user,readHandler,redis-benchmark,9.0,12.0,4.5,6.75,5.25,-4.5
ZPOPMIN,server,kernel,ksys_write,[kernel.kallsyms],1.5,6.0,0,0.75,5.25,-1.5
ZPOPMIN,client,kernel,do_syscall_64,[kernel.kallsyms],19.0,25.5,22.0,20.5,5.0,3.0
ZPOPMIN,server,kernel,__tcp_transmit_skb,[kernel.kallsyms],21.0,26.0,21.0,21.0,5.0,0.0
ZPOPMIN,client,kernel,__dev_queue_xmit,[kernel.kallsyms],3.0,9.5,6.5,4.75,4.75,3.5
ZPOPMIN,server,kernel,vfs_write,[kernel.kallsyms],9.0,12.0,5.5,7.25,4.75,-3.5
ZPOPMIN,client,kernel,finish_task_switch.isra.0,[kernel.kallsyms],10.5,15.0,10.5,10.5,4.5,0.0
ZPOPMIN,server,kernel,net_rx_action,[kernel.kallsyms],7.5,11.0,5.5,6.5,4.5,-2.0
ZPOPMIN,server,kernel,release_sock,[kernel.kallsyms],0.5,5.5,1.5,1.0,4.5,1.0
ZPOPMIN,client,kernel,__sys_recvfrom,[kernel.kallsyms],5.0,8.5,3.5,4.25,4.25,-1.5
ZPOPMIN,client,user,redisNetRead,redis-benchmark,0.5,4.5,0,0.25,4.25,-0.5
ZPOPMIN,server,kernel,tcp_push,[kernel.kallsyms],0.5,5.5,2.0,1.25,4.25,1.5
ZPOPMIN,server,kernel,__rcu_read_unlock,[kernel.kallsyms],3.5,8.0,4.5,4.0,4.0,1.0
ZPOPMIN,server,user,__vdso_gettimeofday,[vdso],8.5,13.5,10.5,9.5,4.0,2.0
ZPOPMIN,server,user,je_malloc_usable_size,redis-server,30.0,33.0,28.0,29.0,4.0,-2.0
ZPOPMIN,client,kernel,skb_attempt_defer_free,[kernel.kallsyms],7.0,10.0,5.5,6.25,3.75,-1.5
ZPOPMIN,server,kernel,handle_softirqs,[kernel.kallsyms],3.0,5.5,0.5,1.75,3.75,-2.5
ZPOPMIN,server,user,write,libc.so.6,0,3.5,0,0.0,3.5,0
ZPOPMIN,server,kernel,x64_sys_call,[kernel.kallsyms],0.5,4.0,0.5,0.5,3.5,0.0
ZPOPMIN,client,kernel,__check_object_size,[kernel.kallsyms],9.0,13.5,11.5,10.25,3.25,2.5
ZPOPMIN,client,kernel,_raw_spin_lock_bh,[kernel.kallsyms],4.5,8.5,6.0,5.25,3.25,1.5
ZPOPMIN,client,user,aeProcessEvents.part.0,redis-benchmark,8.0,11.0,7.5,7.75,3.25,-0.5
ZPOPMIN,client,kernel,fdget,[kernel.kallsyms],4.5,7.5,4.0,4.25,3.25,-0.5
ZPOPMIN,server,kernel,__ip_finish_output,[kernel.kallsyms],2.0,4.5,0.5,1.25,3.25,-1.5
ZPOPMIN,server,user,siphash_nocase,redis-server,2.5,5.5,2.0,2.25,3.25,-0.5
ZPOPMIN,client,user,cfree,libc.so.6,7.5,9.5,5.5,6.5,3.0,-2.0
ZPOPMIN,client,kernel,x64_sys_call,[kernel.kallsyms],1.5,4.0,0.5,1.0,3.0,-1.0
ZPOPMIN,server,kernel,__local_bh_enable_ip,[kernel.kallsyms],2.5,5.0,1.5,2.0,3.0,-1.0
ZPOPMIN,server,kernel,__tcp_push_pending_frames,[kernel.kallsyms],1.0,4.0,1.0,1.0,3.0,0.0
ZPOPMIN,client,kernel,tcp_cleanup_rbuf,[kernel.kallsyms],0,3.0,0.5,0.25,2.75,0.5
ZPOPMIN,server,kernel,__check_object_size,[kernel.kallsyms],2.0,4.5,1.5,1.75,2.75,-0.5
ZPOPMIN,server,kernel,ip_output,[kernel.kallsyms],2.5,5.5,3.0,2.75,2.75,0.5
ZPOPMIN,server,kernel,skb_clone,[kernel.kallsyms],3.5,7.0,5.0,4.25,2.75,1.5
ZPOPMIN,server,kernel,tcp_write_xmit.cold,[kernel.kallsyms],1.0,3.5,0.5,0.75,2.75,-0.5
ZPOPMIN,server,user,writeToClient,redis-server,3.5,6.0,3.0,3.25,2.75,-0.5
ZPOPMIN,client,user,hi_sdscatlen,redis-benchmark,0,3.0,1.0,0.5,2.5,1.0
ZPOPMIN,client,kernel,release_sock,[kernel.kallsyms],1.0,3.0,0,0.5,2.5,-1.0
ZPOPMIN,server,kernel,__mod_timer,[kernel.kallsyms],3.0,4.5,1.0,2.0,2.5,-2.0
ZPOPMIN,server,kernel,kfree_skbmem,[kernel.kallsyms],11.5,12.5,8.5,10.0,2.5,-3.0
ZPOPMIN,client,kernel,__tcp_select_window,[kernel.kallsyms],0,2.5,0.5,0.25,2.25,0.5
ZPOPMIN,client,kernel,__x64_sys_recvfrom,[kernel.kallsyms],0.5,2.5,0,0.25,2.25,-0.5
ZPOPMIN,client,kernel,refill_stock,[kernel.kallsyms],0.5,2.5,0,0.25,2.25,-0.5
ZPOPMIN,client,kernel,tcp_recvmsg,[kernel.kallsyms],1.0,3.0,0.5,0.75,2.25,-0.5
ZPOPMIN,server,kernel,__release_sock,[kernel.kallsyms],8.0,9.0,5.5,6.75,2.25,-2.5
ZPOPMIN,server,kernel,bpf_skops_write_hdr_opt.isra.0,[kernel.kallsyms],2.5,6.0,5.0,3.75,2.25,2.5
ZPOPMIN,server,kernel,skb_defer_free_flush,[kernel.kallsyms],23.0,25.5,23.5,23.25,2.25,0.5
ZPOPMIN,server,kernel,sock_write_iter,[kernel.kallsyms],7.0,8.5,5.5,6.25,2.25,-1.5
ZPOPMIN,client,kernel,ip_local_out,[kernel.kallsyms],0,2.0,0,0.0,2.0,0
ZPOPMIN,client,kernel,ip_queue_xmit,[kernel.kallsyms],0,2.0,0,0.0,2.0,0
ZPOPMIN,client,kernel,ktime_get_ts64,[kernel.kallsyms],3.5,6.5,5.5,4.5,2.0,2.0
ZPOPMIN,client,kernel,sock_poll,[kernel.kallsyms],6.5,7.5,4.5,5.5,2.0,-2.0
ZPOPMIN,server,kernel,__dev_queue_xmit,[kernel.kallsyms],8.5,11.0,9.5,9.0,2.0,1.0
ZPOPMIN,server,kernel,__ip_queue_xmit,[kernel.kallsyms],6.5,5.5,0.5,3.5,2.0,-6.0
ZPOPMIN,server,kernel,__napi_poll,[kernel.kallsyms],0,2.0,0,0.0,2.0,0
ZPOPMIN,server,user,_addReplyProtoToList.part.0,redis-server,3.5,5.5,3.5,3.5,2.0,0.0
ZPOPMIN,server,kernel,_raw_spin_unlock_irq,[kernel.kallsyms],6.0,9.0,8.0,7.0,2.0,2.0
ZPOPMIN,server,kernel,ip_local_deliver_finish,[kernel.kallsyms],0,2.5,1.0,0.5,2.0,1.0
ZPOPMIN,server,kernel,tcp_v4_rcv,[kernel.kallsyms],8.0,10.0,8.0,8.0,2.0,0.0
ZPOPMIN,client,kernel,__hrtimer_setup,[kernel.kallsyms],0,2.0,0.5,0.25,1.75,0.5
ZPOPMIN,client,kernel,__inet_lookup_established,[kernel.kallsyms],7.0,10.5,10.5,8.75,1.75,3.5
ZPOPMIN,client,kernel,__rcu_read_unlock,[kernel.kallsyms],2.0,3.5,1.5,1.75,1.75,-0.5
ZPOPMIN,client,user,createStringObject,redis-benchmark,1.0,3.0,1.5,1.25,1.75,0.5
ZPOPMIN,client,user,epoll_wait,libc.so.6,1.0,2.5,0.5,0.75,1.75,-0.5
ZPOPMIN,client,kernel,schedule_hrtimeout_range_clock,[kernel.kallsyms],1.0,3.5,2.5,1.75,1.75,1.5
ZPOPMIN,server,kernel,_find_first_bit,[kernel.kallsyms],0,2.0,0.5,0.25,1.75,0.5
ZPOPMIN,server,kernel,aa_sk_perm,[kernel.kallsyms],1.0,3.0,1.5,1.25,1.75,0.5
ZPOPMIN,server,user,genericZpopCommand,redis-server,3.0,3.5,0.5,1.75,1.75,-2.5
ZPOPMIN,server,kernel,ip_rcv_core,[kernel.kallsyms],1.5,3.0,1.0,1.25,1.75,-0.5
ZPOPMIN,server,user,je_arena_dalloc_bin_junked_locked,redis-server,2.0,4.0,2.5,2.25,1.75,0.5
ZPOPMIN,client,kernel,__schedule,[kernel.kallsyms],2.0,2.5,0,1.0,1.5,-2.0
ZPOPMIN,client,user,aeMain,redis-benchmark,0,1.5,0,0.0,1.5,0
ZPOPMIN,client,user,createArrayObject,redis-benchmark,0,1.5,0,0.0,1.5,0
ZPOPMIN,client,kernel,ip_skb_dst_mtu,[kernel.kallsyms],1.0,2.5,1.0,1.0,1.5,0.0
ZPOPMIN,client,user,redisGetReply,redis-benchmark,1.5,3.0,1.5,1.5,1.5,0.0
ZPOPMIN,server,kernel,__x64_sys_write,[kernel.kallsyms],0,1.5,0,0.0,1.5,0
ZPOPMIN,server,kernel,_raw_spin_unlock,[kernel.kallsyms],1.0,2.0,0,0.5,1.5,-1.0
ZPOPMIN,server,user,dictSdsKeyCaseCompare,redis-server,0,2.0,1.0,0.5,1.5,1.0
ZPOPMIN,server,kernel,inet_sendmsg,[kernel.kallsyms],0.5,2.0,0.5,0.5,1.5,0.0
ZPOPMIN,server,kernel,ip_local_out,[kernel.kallsyms],1.0,2.0,0,0.5,1.5,-1.0
ZPOPMIN,server,kernel,ip_protocol_deliver_rcu,[kernel.kallsyms],0.5,2.0,0.5,0.5,1.5,0.0
ZPOPMIN,server,kernel,ip_queue_xmit,[kernel.kallsyms],0,1.5,0,0.0,1.5,0
ZPOPMIN,server,kernel,kfree,[kernel.kallsyms],4.5,4.5,1.5,3.0,1.5,-3.0
ZPOPMIN,server,kernel,sock_def_readable,[kernel.kallsyms],2.0,3.0,1.0,1.5,1.5,-1.0
ZPOPMIN,server,kernel,tcp_v4_do_rcv,[kernel.kallsyms],1.0,3.5,3.0,2.0,1.5,2.0
ZPOPMIN,server,user,zsetRemoveFromSkiplist,redis-server,0,1.5,0,0.0,1.5,0
ZPOPMIN,client,kernel,fput,[kernel.kallsyms],2.0,2.5,0.5,1.25,1.25,-1.5
ZPOPMIN,client,kernel,select_estimate_accuracy,[kernel.kallsyms],1.0,2.5,1.5,1.25,1.25,0.5
ZPOPMIN,client,kernel,tcp_measure_rcv_mss,[kernel.kallsyms],1.0,3.0,2.5,1.75,1.25,1.5
ZPOPMIN,server,kernel,__rcu_read_lock,[kernel.kallsyms],5.0,7.0,6.5,5.75,1.25,1.5
ZPOPMIN,server,kernel,_raw_spin_lock_irqsave,[kernel.kallsyms],1.5,3.0,2.0,1.75,1.25,0.5
ZPOPMIN,server,kernel,_raw_spin_unlock_irqrestore,[kernel.kallsyms],68.5,66.0,61.0,64.75,1.25,-7.5
ZPOPMIN,server,user,clock_gettime@plt,redis-server,0,1.5,0.5,0.25,1.25,0.5
ZPOPMIN,server,user,dictSdsHash,redis-server,0,1.5,0.5,0.25,1.25,0.5
ZPOPMIN,server,kernel,fdget_pos,[kernel.kallsyms],1.5,5.0,6.0,3.75,1.25,4.5
ZPOPMIN,server,user,getMonotonicUs_posix,redis-server,1.5,3.0,2.0,1.75,1.25,0.5
ZPOPMIN,server,kernel,ip_finish_output,[kernel.kallsyms],0.5,2.0,1.0,0.75,1.25,0.5
ZPOPMIN,server,kernel,ip_finish_output2,[kernel.kallsyms],8.5,10.0,9.0,8.75,1.25,0.5
ZPOPMIN,server,kernel,ip_rcv,[kernel.kallsyms],1.5,2.0,0,0.75,1.25,-1.5
ZPOPMIN,server,kernel,kmem_cache_alloc_node_noprof,[kernel.kallsyms],7.0,9.5,9.5,8.25,1.25,2.5
ZPOPMIN,server,kernel,process_backlog,[kernel.kallsyms],1.0,2.0,0.5,0.75,1.25,-0.5
ZPOPMIN,server,kernel,skb_free_head,[kernel.kallsyms],0.5,1.5,0,0.25,1.25,-0.5
ZPOPMIN,server,kernel,skb_page_frag_refill,[kernel.kallsyms],5.5,7.0,6.0,5.75,1.25,0.5
ZPOPMIN,server,kernel,tcp_event_new_data_sent,[kernel.kallsyms],6.5,8.0,7.0,6.75,1.25,0.5
ZPOPMIN,server,kernel,tcp_write_xmit,[kernel.kallsyms],6.0,6.0,3.5,4.75,1.25,-2.5
ZPOPMIN,server,kernel,validate_xmit_skb,[kernel.kallsyms],2.0,2.5,0.5,1.25,1.25,-1.5
ZPOPMIN,server,user,zslDeleteNode,redis-server,2.0,3.0,1.5,1.75,1.25,-0.5
ZPOPMIN,client,kernel,__netif_receive_skb_core.constprop.0,[kernel.kallsyms],1.0,2.5,2.0,1.5,1.0,1.0
ZPOPMIN,client,kernel,_raw_spin_lock_irqsave,[kernel.kallsyms],0,1.0,0,0.0,1.0,0
ZPOPMIN,client,kernel,do_softirq.part.0,[kernel.kallsyms],1.5,2.0,0.5,1.0,1.0,-1.0
ZPOPMIN,client,kernel,handle_softirqs,[kernel.kallsyms],2.0,2.0,0,1.0,1.0,-2.0
ZPOPMIN,client,kernel,kmem_cache_alloc_node_noprof,[kernel.kallsyms],5.0,5.0,3.0,4.0,1.0,-2.0
ZPOPMIN,client,kernel,net_rx_action,[kernel.kallsyms],2.0,4.0,4.0,3.0,1.0,2.0
ZPOPMIN,client,user,redisGetReplyFromReader,redis-benchmark,1.0,1.5,0,0.5,1.0,-1.0
ZPOPMIN,client,kernel,tcp_rcv_established,[kernel.kallsyms],8.0,8.5,7.0,7.5,1.0,-1.0
ZPOPMIN,server,kernel,__netif_receive_skb,[kernel.kallsyms],0,1.0,0,0.0,1.0,0
ZPOPMIN,server,user,__sprintf_chk,libc.so.6,0,1.0,0,0.0,1.0,0
ZPOPMIN,server,user,dictGenericDelete,redis-server,7.0,9.5,10.0,8.5,1.0,3.0
ZPOPMIN,server,kernel,ip_rcv_finish_core,[kernel.kallsyms],4.0,4.0,2.0,3.0,1.0,-2.0
ZPOPMIN,server,user,je_free,redis-server,9.0,13.5,16.0,12.5,1.0,7.0
ZPOPMIN,server,user,processMultibulkBuffer,redis-server,6.5,8.0,7.5,7.0,1.0,1.0
ZPOPMIN,server,user,sdscmp,redis-server,1.5,3.0,2.5,2.0,1.0,1.0
ZPOPMIN,server,kernel,skb_csum_hwoffload_help,[kernel.kallsyms],0,1.0,0,0.0,1.0,0
ZPOPMIN,server,kernel,tcp_do_parse_auth_options,[kernel.kallsyms],0,1.5,1.0,0.5,1.0,1.0
ZPOPMIN,server,kernel,tcp_sendmsg,[kernel.kallsyms],1.0,1.5,0,0.5,1.0,-1.0
ZPOPMIN,server,kernel,tcp_update_pacing_rate,[kernel.kallsyms],0.5,2.0,1.5,1.0,1.0,1.0
ZPOPMIN,server,user,touchWatchedKey,redis-server,0,1.0,0,0.0,1.0,0
ZPOPMIN,server,user,zpopminCommand,redis-server,0.5,1.5,0.5,0.5,1.0,0.0
ZPOPMIN,server,user,ztryrealloc_usable,redis-server,3.0,4.0,3.0,3.0,1.0,0.0
ZPOPMIN,client,kernel,_raw_spin_lock_irq,[kernel.kallsyms],0,1.5,1.5,0.75,0.75,1.5
ZPOPMIN,client,user,clock_gettime,libc.so.6,0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,client,kernel,do_epoll_ctl_file,[kernel.kallsyms],0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,client,kernel,ep_done_scan,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,client,user,hi_sdsMakeRoomFor,redis-benchmark,0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,client,kernel,ip_finish_output,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,client,kernel,raw_local_deliver,[kernel.kallsyms],1.0,1.5,0.5,0.75,0.75,-0.5
ZPOPMIN,client,kernel,schedule,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,client,kernel,sock_def_readable,[kernel.kallsyms],0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,server,user,ACLCheckAllPerm,redis-server,0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,server,user,__libc_alloca_cutoff,libc.so.6,0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,server,kernel,_raw_spin_lock,[kernel.kallsyms],4.0,5.0,4.5,4.25,0.75,0.5
ZPOPMIN,server,kernel,_raw_spin_unlock_bh,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,server,user,addReplyDeferredLen,redis-server,0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,server,kernel,apparmor_file_permission,[kernel.kallsyms],1.0,1.5,0.5,0.75,0.75,-0.5
ZPOPMIN,server,kernel,apparmor_socket_sendmsg,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,server,kernel,check_stack_object,[kernel.kallsyms],0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,server,user,decrRefCount,redis-server,4.0,3.5,1.5,2.75,0.75,-2.5
ZPOPMIN,server,user,dictRehash,redis-server,0,1.0,0.5,0.25,0.75,0.5
ZPOPMIN,server,kernel,fput,[kernel.kallsyms],1.5,3.0,3.0,2.25,0.75,1.5
ZPOPMIN,server,user,getExpire,redis-server,0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,server,user,memcpy@plt,redis-server,2.0,2.5,1.5,1.75,0.75,-0.5
ZPOPMIN,server,user,siphash,redis-server,9.5,9.0,7.0,8.25,0.75,-2.5
ZPOPMIN,server,kernel,tcp_chrono_stop,[kernel.kallsyms],1.0,1.5,0.5,0.75,0.75,-0.5
ZPOPMIN,server,kernel,tcp_data_ready,[kernel.kallsyms],0.5,1.0,0,0.25,0.75,-0.5
ZPOPMIN,server,kernel,tcp_options_write,[kernel.kallsyms],1.5,1.5,0,0.75,0.75,-1.5
ZPOPMIN,server,kernel,tcp_update_skb_after_send,[kernel.kallsyms],1.5,3.0,3.0,2.25,0.75,1.5
ZPOPMIN,server,kernel,tcp_v4_fill_cb,[kernel.kallsyms],0.5,1.5,1.0,0.75,0.75,0.5
ZPOPMIN,server,kernel,validate_xmit_xfrm,[kernel.kallsyms],1.0,1.5,0.5,0.75,0.75,-0.5
ZPOPMIN,server,user,zmalloc_usable,redis-server,2.0,2.0,0.5,1.25,0.75,-1.5
ZPOPMIN,server,user,zslDelete,redis-server,3.5,3.5,2.0,2.75,0.75,-1.5
ZPOPMIN,client,kernel,__mod_timer,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,__netif_rx,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,__secure_computing,[kernel.kallsyms],1.0,1.5,1.0,1.0,0.5,0.0
ZPOPMIN,client,kernel,_raw_spin_unlock,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,check_stack_object,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,ep_remove,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,user,getMonotonicUs_posix,redis-benchmark,1.5,2.0,1.5,1.5,0.5,0.0
ZPOPMIN,client,kernel,hrtimer_try_to_cancel.part.0,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,inet_recvmsg,[kernel.kallsyms],0,1.0,1.0,0.5,0.5,1.0
ZPOPMIN,client,kernel,kmalloc_reserve,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,kmem_cache_alloc_noprof,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,ktime_get,[kernel.kallsyms],3.0,4.5,5.0,4.0,0.5,2.0
ZPOPMIN,client,kernel,napi_consume_skb,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,netif_rx_internal,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,page_counter_uncharge,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,user,recv@plt,redis-benchmark,0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,skb_copy_datagram_iter,[kernel.kallsyms],0.5,1.0,0.5,0.5,0.5,0.0
ZPOPMIN,client,kernel,skb_page_frag_refill,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,tcp_event_data_recv,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,tcp_schedule_loss_probe,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,tcp_skb_entail,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,timespec64_add_safe,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,validate_xmit_skb,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,ACLCheckCommandPerm,redis-server,0,1.0,1.0,0.5,0.5,1.0
ZPOPMIN,server,kernel,__tcp_select_window,[kernel.kallsyms],3.0,2.5,1.0,2.0,0.5,-2.0
ZPOPMIN,server,user,aeProcessEvents.part.0,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,begin_current_label_crit_section,[kernel.kallsyms],0,1.0,1.0,0.5,0.5,1.0
ZPOPMIN,server,user,dictUnlink,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,hasActiveChildProcess,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,je_arena_ralloc,redis-server,2.5,3.5,3.5,3.0,0.5,1.0
ZPOPMIN,server,user,je_realloc,redis-server,1.5,2.0,1.5,1.5,0.5,0.0
ZPOPMIN,server,kernel,ktime_get_with_offset,[kernel.kallsyms],5.5,6.0,5.5,5.5,0.5,0.0
ZPOPMIN,server,kernel,netif_skb_features,[kernel.kallsyms],1.0,1.5,1.0,1.0,0.5,0.0
ZPOPMIN,server,kernel,page_counter_try_charge,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,pthread_mutex_unlock@plt,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,rb_next,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,readQueryFromClient,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,rw_verify_area,[kernel.kallsyms],0,1.0,1.0,0.5,0.5,1.0
ZPOPMIN,server,kernel,sched_balance_softirq,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,sdsfree,redis-server,0.5,1.5,1.5,1.0,0.5,1.0
ZPOPMIN,server,user,setDeferredArrayLen,redis-server,0.5,1.0,0.5,0.5,0.5,0.0
ZPOPMIN,server,kernel,sk_free,[kernel.kallsyms],1.0,2.0,2.0,1.5,0.5,1.0
ZPOPMIN,server,kernel,sk_reset_timer,[kernel.kallsyms],0.5,1.0,0.5,0.5,0.5,0.0
ZPOPMIN,server,kernel,skb_condense,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,string2ll,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,tcp_ack_update_rtt,[kernel.kallsyms],1.0,1.0,0,0.5,0.5,-1.0
ZPOPMIN,server,kernel,tcp_grow_window,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,tcp_try_rmem_schedule,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,kernel,tcp_wmem_schedule,[kernel.kallsyms],0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,trackingInvalidateKey,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,server,user,zmalloc,redis-server,19.0,19.5,19.0,19.0,0.5,0.0
ZPOPMIN,server,user,zsetDel,redis-server,0,0.5,0,0.0,0.5,0
ZPOPMIN,client,kernel,__mod_memcg_state,[kernel.kallsyms],0,1.0,1.5,0.75,0.25,1.5
ZPOPMIN,client,kernel,__rcu_read_lock,[kernel.kallsyms],4.0,2.5,0.5,2.25,0.25,-3.5
ZPOPMIN,client,kernel,__sk_mem_schedule,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,client,kernel,_copy_from_iter,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,client,kernel,aa_sk_perm,[kernel.kallsyms],2.0,1.5,0.5,1.25,0.25,-1.5
ZPOPMIN,client,user,aeDeleteFileEvent,redis-benchmark,0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,client,user,epoll_ctl,libc.so.6,0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,client,kernel,import_ubuf,[kernel.kallsyms],1.0,1.5,1.5,1.25,0.25,0.5
ZPOPMIN,client,kernel,ip_rcv,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,client,kernel,ip_rcv_finish_core,[kernel.kallsyms],2.5,2.5,2.0,2.25,0.25,-0.5
ZPOPMIN,client,kernel,ip_send_check,[kernel.kallsyms],1.0,1.5,1.5,1.25,0.25,0.5
ZPOPMIN,client,user,memcpy@plt,redis-benchmark,0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,client,kernel,memset,[kernel.kallsyms],2.5,2.0,1.0,1.75,0.25,-1.5
ZPOPMIN,client,kernel,mutex_unlock,[kernel.kallsyms],1.5,2.5,3.0,2.25,0.25,1.5
ZPOPMIN,client,user,redisBufferRead,redis-benchmark,1.5,2.0,2.0,1.75,0.25,0.5
ZPOPMIN,client,kernel,sock_recvmsg,[kernel.kallsyms],1.5,1.0,0,0.75,0.25,-1.5
ZPOPMIN,client,kernel,tcp_data_queue,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,client,kernel,tcp_do_parse_auth_options,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,client,kernel,tcp_mstamp_refresh,[kernel.kallsyms],1.5,1.0,0,0.75,0.25,-1.5
ZPOPMIN,client,kernel,tcp_poll,[kernel.kallsyms],5.5,6.0,6.0,5.75,0.25,0.5
ZPOPMIN,client,kernel,tcp_recvmsg_locked,[kernel.kallsyms],10.5,11.0,11.0,10.75,0.25,0.5
ZPOPMIN,client,kernel,tcp_send_ack,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,kernel,__memcg_slab_free_hook,[kernel.kallsyms],0.5,1.0,1.0,0.75,0.25,0.5
ZPOPMIN,server,kernel,__netif_receive_skb_core.constprop.0,[kernel.kallsyms],3.5,3.0,2.0,2.75,0.25,-1.5
ZPOPMIN,server,kernel,__netif_receive_skb_one_core,[kernel.kallsyms],1.0,1.5,1.5,1.25,0.25,0.5
ZPOPMIN,server,kernel,__secure_computing,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,user,__snprintf_chk@plt,redis-server,0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,kernel,__tcp_ack_snd_check,[kernel.kallsyms],1.0,1.0,0.5,0.75,0.25,-0.5
ZPOPMIN,server,user,addReplyDouble,redis-server,1.0,1.0,0.5,0.75,0.25,-0.5
ZPOPMIN,server,user,commandProcessed,redis-server,0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,kernel,cubictcp_cong_avoid,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,user,freeClientOriginalArgv,redis-server,0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,user,gettimeofday@plt,redis-server,0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,kernel,ip_local_deliver,[kernel.kallsyms],0,1.0,1.5,0.75,0.25,1.5
ZPOPMIN,server,user,keyIsExpired,redis-server,0,1.0,1.5,0.75,0.25,1.5
ZPOPMIN,server,kernel,ktime_get,[kernel.kallsyms],7.0,9.0,10.5,8.75,0.25,3.5
ZPOPMIN,server,kernel,mod_timer,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,user,moduleCallCommandFilters,redis-server,1.0,1.5,1.5,1.25,0.25,0.5
ZPOPMIN,server,kernel,netif_rx_internal,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,kernel,nf_hook_slow,[kernel.kallsyms],1.5,2.0,2.0,1.75,0.25,0.5
ZPOPMIN,server,user,processInputBuffer,redis-server,0.5,1.0,1.0,0.75,0.25,0.5
ZPOPMIN,server,user,propagate.part.0,redis-server,0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,user,pthread_mutex_unlock,libc.so.6,0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,user,replicationFeedMonitors,redis-server,0.5,1.0,1.0,0.75,0.25,0.5
ZPOPMIN,server,user,replicationFeedSlaves,redis-server,0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,kernel,skb_push,[kernel.kallsyms],0.5,0.5,0,0.25,0.25,-0.5
ZPOPMIN,server,kernel,tcp_schedule_loss_probe,[kernel.kallsyms],2.5,3.5,4.0,3.25,0.25,1.5
ZPOPMIN,server,kernel,tcp_send_delayed_ack,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,kernel,tcp_small_queue_check.isra.0,[kernel.kallsyms],0,0.5,0.5,0.25,0.25,0.5
ZPOPMIN,server,kernel,try_charge_memcg,[kernel.kallsyms],0.5,1.0,1.0,0.75,0.25,0.5
ZPOPMIN,client,kernel,__css_rstat_updated,[kernel.kallsyms],1.0,0.5,0,0.5,0.0,-1.0
ZPOPMIN,client,kernel,__ip_finish_output,[kernel.kallsyms],1.0,1.5,2.0,1.5,0.0,1.0
ZPOPMIN,client,kernel,__sk_mem_reduce_allocated,[kernel.kallsyms],1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,client,kernel,__tcp_cleanup_rbuf,[kernel.kallsyms],1.0,2.5,4.0,2.5,0.0,3.0
ZPOPMIN,client,kernel,__virt_addr_valid,[kernel.kallsyms],1.5,1.5,1.5,1.5,0.0,0.0
ZPOPMIN,client,user,counts_index_for,redis-benchmark,1.0,2.0,3.0,2.0,0.0,2.0
ZPOPMIN,client,kernel,dst_release,[kernel.kallsyms],1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,client,kernel,ep_start_scan,[kernel.kallsyms],1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,client,user,freeReplyObject,redis-benchmark,2.0,3.0,4.0,3.0,0.0,2.0
ZPOPMIN,client,user,hdr_record_values,redis-benchmark,3.0,2.0,1.0,2.0,0.0,-2.0
ZPOPMIN,client,kernel,hrtimer_setup_sleeper_on_stack,[kernel.kallsyms],0,0.5,1.0,0.5,0.0,1.0
ZPOPMIN,client,kernel,ip_finish_output2,[kernel.kallsyms],3.0,3.5,4.0,3.5,0.0,1.0
ZPOPMIN,client,kernel,ip_output,[kernel.kallsyms],1.0,2.0,3.0,2.0,0.0,2.0
ZPOPMIN,client,kernel,netif_skb_features,[kernel.kallsyms],1.0,0.5,0,0.5,0.0,-1.0
ZPOPMIN,client,kernel,sk_filter_trim_cap,[kernel.kallsyms],0,1.0,2.0,1.0,0.0,2.0
ZPOPMIN,client,kernel,skb_defer_free_flush,[kernel.kallsyms],2.0,2.0,2.0,2.0,0.0,0.0
ZPOPMIN,client,user,string2ll,redis-benchmark,1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,client,kernel,tcp_add_backlog,[kernel.kallsyms],0.5,0.5,0.5,0.5,0.0,0.0
ZPOPMIN,client,kernel,tcp_recv_timestamp,[kernel.kallsyms],1.0,0.5,0,0.5,0.0,-1.0
ZPOPMIN,client,kernel,tcp_v4_do_rcv,[kernel.kallsyms],1.5,1.0,0.5,1.0,0.0,-1.0
ZPOPMIN,server,kernel,__ip_local_out,[kernel.kallsyms],3.0,1.5,0,1.5,0.0,-3.0
ZPOPMIN,server,kernel,__sk_mem_raise_allocated,[kernel.kallsyms],0,1.0,2.0,1.0,0.0,2.0
ZPOPMIN,server,user,_addReplyToBuffer,redis-server,2.0,1.5,1.0,1.5,0.0,-1.0
ZPOPMIN,server,user,clock_gettime,libc.so.6,1.5,1.0,0.5,1.0,0.0,-1.0
ZPOPMIN,server,user,listDelNode,redis-server,3.0,3.0,3.0,3.0,0.0,0.0
ZPOPMIN,server,user,memcpy@plt,libc.so.6,1.0,0.5,0,0.5,0.0,-1.0
ZPOPMIN,server,user,moduleNotifyKeyspaceEvent,redis-server,1.0,0.5,0,0.5,0.0,-1.0
ZPOPMIN,server,kernel,netdev_core_pick_tx,[kernel.kallsyms],0.5,0.5,0.5,0.5,0.0,0.0
ZPOPMIN,server,user,prepareClientToWrite,redis-server,0.5,0.5,0.5,0.5,0.0,0.0
ZPOPMIN,server,kernel,rb_insert_color,[kernel.kallsyms],2.0,1.5,1.0,1.5,0.0,-1.0
ZPOPMIN,server,user,sdsdup,redis-server,4.5,4.0,3.5,4.0,0.0,-1.0
ZPOPMIN,server,user,setDeferredReply,redis-server,5.0,5.5,6.0,5.5,0.0,1.0
ZPOPMIN,server,user,strchrnul@plt,libc.so.6,1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,server,kernel,tcp_ack,[kernel.kallsyms],11.0,11.0,11.0,11.0,0.0,0.0
ZPOPMIN,server,kernel,tcp_rate_skb_delivered.part.0,[kernel.kallsyms],1.0,1.0,1.0,1.0,0.0,0.0
ZPOPMIN,server,kernel,tcp_recvmsg_locked,[kernel.kallsyms],0.5,0.5,0.5,0.5,0.0,0.0
ZPOPMIN,server,user,trimReplyUnusedTailSpace,redis-server,0.5,1.0,1.5,1.0,0.0,1.0
ZPOPMIN,server,user,writeCommandsDeniedByDiskError,redis-server,0,0.5,1.0,0.5,0.0,1.0
ZPOPMIN,client,kernel,__free_frozen_pages,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,__ip_queue_xmit,[kernel.kallsyms],1.5,1.0,1.0,1.25,-0.25,-0.5
ZPOPMIN,client,kernel,__memcg_slab_post_alloc_hook,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,__netif_receive_skb_one_core,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,__pcs_replace_empty_main,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,__pcs_replace_full_main,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,__refill_obj_stock,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,__sock_wfree,[kernel.kallsyms],0.5,1.0,2.0,1.25,-0.25,1.5
ZPOPMIN,client,kernel,__x64_sys_epoll_wait,[kernel.kallsyms],2.0,1.0,0.5,1.25,-0.25,-1.5
ZPOPMIN,client,user,clock_gettime@plt,redis-benchmark,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,dev_hard_start_xmit,[kernel.kallsyms],1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,client,kernel,do_epoll_ctl,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,enqueue_to_backlog,[kernel.kallsyms],4.5,3.0,2.0,3.25,-0.25,-2.5
ZPOPMIN,client,kernel,eth_type_trans,[kernel.kallsyms],1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,client,user,hdr_record_value,redis-benchmark,1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,client,kernel,hrtimer_active,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,ip_local_deliver,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,ip_local_deliver_finish,[kernel.kallsyms],1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,client,kernel,ip_protocol_deliver_rcu,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,ip_rcv_core,[kernel.kallsyms],1.5,0.5,0,0.75,-0.25,-1.5
ZPOPMIN,client,kernel,ipv4_dst_check,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,mem_cgroup_sk_charge,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,mod_memcg_state,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,netdev_core_pick_tx,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,nf_hook_slow,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,user,normalize_index,redis-benchmark,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,prandom_u32_state,[kernel.kallsyms],2.0,1.5,1.5,1.75,-0.25,-0.5
ZPOPMIN,client,kernel,qdisc_pkt_len_segs_init,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,rb_erase,[kernel.kallsyms],0.5,0.5,1.0,0.75,-0.25,0.5
ZPOPMIN,client,kernel,sched_balance_softirq,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,schedule_hrtimeout_range,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,set_normalized_timespec64,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,sk_reset_timer,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,skb_condense,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,skb_csum_hwoffload_help,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,tcp_data_ecn_check,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,tcp_inbound_hash,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,tcp_newly_delivered,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,tcp_options_write,[kernel.kallsyms],2.0,1.0,0.5,1.25,-0.25,-1.5
ZPOPMIN,client,kernel,tcp_sendmsg_locked,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,tcp_try_rmem_schedule,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,tcp_urg,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,client,kernel,tcp_v4_fill_cb,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,client,kernel,tcp_wfree,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,__netif_rx,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,__sk_mem_schedule,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,user,__snprintf_chk,libc.so.6,1.5,1.5,2.0,1.75,-0.25,0.5
ZPOPMIN,server,kernel,__usecs_to_jiffies,[kernel.kallsyms],3.0,3.0,3.5,3.25,-0.25,0.5
ZPOPMIN,server,kernel,__wake_up_sync_key,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,_copy_from_iter,[kernel.kallsyms],5.5,7.0,9.0,7.25,-0.25,3.5
ZPOPMIN,server,kernel,_raw_spin_lock_irq,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,afterSleep,redis-server,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,areClientsPaused,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,user,authRequired,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,backlog_lock_irq_save,[kernel.kallsyms],0.5,0.5,1.0,0.75,-0.25,0.5
ZPOPMIN,server,user,checkType,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,clear_highpages_kasan_tagged,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,dictFetchValue,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,dst_release,[kernel.kallsyms],7.5,7.0,7.0,7.25,-0.25,-0.5
ZPOPMIN,server,kernel,eth_type_trans,[kernel.kallsyms],1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,server,user,freeClientReplyValue,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,hrtimer_active,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,je_extent_heap_remove_first,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,user,je_tcache_event_hard,redis-server,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,ktime_get_ts64,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,lock_timer_base,[kernel.kallsyms],0.5,1.0,2.0,1.25,-0.25,1.5
ZPOPMIN,server,kernel,memcg_state_val_in_pages,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,notifyKeyspaceEvent,redis-server,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,user,pthread_mutex_trylock,libc.so.6,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,raw_local_deliver,[kernel.kallsyms],2.0,1.0,0.5,1.25,-0.25,-1.5
ZPOPMIN,server,user,sdsrange,redis-server,0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,sk_filter_trim_cap,[kernel.kallsyms],3.0,3.0,3.5,3.25,-0.25,0.5
ZPOPMIN,server,kernel,skb_clone_tx_timestamp,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,skb_release_head_state,[kernel.kallsyms],0.5,0.5,1.0,0.75,-0.25,0.5
ZPOPMIN,server,user,slowlogPushEntryIfNeeded,redis-server,0,0.5,1.5,0.75,-0.25,1.5
ZPOPMIN,server,kernel,sock_poll,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,user,strcasecmp@plt,redis-server,1.0,1.0,1.5,1.25,-0.25,0.5
ZPOPMIN,server,user,strchr@plt,redis-server,1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,server,user,string2ll.part.0,redis-server,2.0,1.0,0.5,1.25,-0.25,-1.5
ZPOPMIN,server,user,strlen@plt,libc.so.6,0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,tcp_ack_tstamp,[kernel.kallsyms],1.0,0.5,0.5,0.75,-0.25,-0.5
ZPOPMIN,server,kernel,tcp_data_ecn_check,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,tcp_delack_max,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,tcp_established_options,[kernel.kallsyms],1.0,1.5,2.5,1.75,-0.25,1.5
ZPOPMIN,server,kernel,tcp_event_data_recv,[kernel.kallsyms],1.5,0.5,0,0.75,-0.25,-1.5
ZPOPMIN,server,kernel,tcp_poll,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,tcp_try_coalesce,[kernel.kallsyms],0.5,0,0,0.25,-0.25,-0.5
ZPOPMIN,server,kernel,tcp_urg,[kernel.kallsyms],0,0,0.5,0.25,-0.25,0.5
ZPOPMIN,server,kernel,tcp_wfree,[kernel.kallsyms],5.0,4.0,3.5,4.25,-0.25,-1.5
ZPOPMIN,client,kernel,_raw_spin_lock,[kernel.kallsyms],1.5,1.5,2.5,2.0,-0.5,1.0
ZPOPMIN,client,kernel,backlog_lock_irq_save,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,client,kernel,ep_item_poll.isra.0,[kernel.kallsyms],7.0,7.0,8.0,7.5,-0.5,1.0
ZPOPMIN,client,user,hi_sdsrange,redis-benchmark,0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,client,kernel,kfree_skbmem,[kernel.kallsyms],0,0,1.0,0.5,-0.5,1.0
ZPOPMIN,client,kernel,kmem_cache_free,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,client,user,moveToNextTask,redis-benchmark,2.5,1.5,1.5,2.0,-0.5,-1.0
ZPOPMIN,client,kernel,security_socket_recvmsg,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,client,kernel,simple_copy_to_iter,[kernel.kallsyms],0,0.5,2.0,1.0,-0.5,2.0
ZPOPMIN,client,kernel,skb_free_head,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,user,LRU_CLOCK,redis-server,0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,server,kernel,__css_rstat_updated,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,kernel,__inet_lookup_established,[kernel.kallsyms],10.5,11.5,13.5,12.0,-0.5,3.0
ZPOPMIN,server,kernel,__kfree_skb,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,kernel,__pcs_replace_empty_main,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,server,user,addReplyProto,redis-server,0.5,0.5,1.5,1.0,-0.5,1.0
ZPOPMIN,server,user,checkClientOutputBufferLimits,redis-server,0,0,1.0,0.5,-0.5,1.0
ZPOPMIN,server,kernel,dev_hard_start_xmit,[kernel.kallsyms],1.0,0.5,1.0,1.0,-0.5,0.0
ZPOPMIN,server,kernel,finish_task_switch.isra.0,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,server,user,htNeedsResize,redis-server,1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,kernel,ip_skb_dst_mtu,[kernel.kallsyms],2.5,2.0,2.5,2.5,-0.5,0.0
ZPOPMIN,server,kernel,is_vmalloc_addr,[kernel.kallsyms],0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,server,user,je_arena_tcache_fill_small,redis-server,1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,kernel,kmalloc_reserve,[kernel.kallsyms],2.0,1.5,2.0,2.0,-0.5,0.0
ZPOPMIN,server,user,listAddNodeTail,redis-server,0.5,0,0.5,0.5,-0.5,0.0
ZPOPMIN,server,user,memcmp@plt,redis-server,1.5,0.5,0.5,1.0,-0.5,-1.0
ZPOPMIN,server,kernel,memset,[kernel.kallsyms],8.5,8.0,8.5,8.5,-0.5,0.0
ZPOPMIN,server,kernel,prandom_u32_state,[kernel.kallsyms],2.0,1.5,2.0,2.0,-0.5,0.0
ZPOPMIN,server,kernel,refill_stock,[kernel.kallsyms],0,0,1.0,0.5,-0.5,1.0
ZPOPMIN,server,kernel,security_file_permission,[kernel.kallsyms],1.5,1.0,1.5,1.5,-0.5,0.0
ZPOPMIN,server,kernel,security_socket_sendmsg,[kernel.kallsyms],1.5,0.5,0.5,1.0,-0.5,-1.0
ZPOPMIN,server,kernel,skb_network_protocol,[kernel.kallsyms],1.5,0.5,0.5,1.0,-0.5,-1.0
ZPOPMIN,server,kernel,tcp_data_queue,[kernel.kallsyms],0.5,0.5,1.5,1.0,-0.5,1.0
ZPOPMIN,server,kernel,tcp_newly_delivered,[kernel.kallsyms],0,0,1.0,0.5,-0.5,1.0
ZPOPMIN,server,kernel,tcp_rcv_sne_update,[kernel.kallsyms],1.0,0,0,0.5,-0.5,-1.0
ZPOPMIN,server,kernel,tcp_rearm_rto,[kernel.kallsyms],1.0,1.0,2.0,1.5,-0.5,1.0
ZPOPMIN,server,user,trackingInvalidateKeyRaw,redis-server,2.0,0.5,0,1.0,-0.5,-2.0
ZPOPMIN,client,kernel,__sk_dst_check,[kernel.kallsyms],1.0,0,0.5,0.75,-0.75,-0.5
ZPOPMIN,client,kernel,is_vmalloc_addr,[kernel.kallsyms],1.5,0,0,0.75,-0.75,-1.5
ZPOPMIN,client,kernel,ktime_get_with_offset,[kernel.kallsyms],1.5,1.0,2.0,1.75,-0.75,0.5
ZPOPMIN,client,kernel,memcg_state_val_in_pages,[kernel.kallsyms],1.0,0,0.5,0.75,-0.75,-0.5
ZPOPMIN,client,kernel,tcp_v4_rcv,[kernel.kallsyms],4.5,3.5,4.0,4.25,-0.75,-0.5
ZPOPMIN,server,kernel,__xfrm_policy_check2.constprop.0,[kernel.kallsyms],1.0,0,0.5,0.75,-0.75,-0.5
ZPOPMIN,server,kernel,_copy_to_iter,[kernel.kallsyms],1.5,1.0,2.0,1.75,-0.75,0.5
ZPOPMIN,server,user,addReply,redis-server,1.5,0,0,0.75,-0.75,-1.5
ZPOPMIN,server,user,addReply.part.0,redis-server,3.0,2.0,2.5,2.75,-0.75,-0.5
ZPOPMIN,server,user,addReplyBulkCBuffer,redis-server,1.0,0,0.5,0.75,-0.75,-0.5
ZPOPMIN,server,user,createStringObject,redis-server,1.0,0,0.5,0.75,-0.75,-0.5
ZPOPMIN,server,user,dictSdsCaseHash,redis-server,0,0,1.5,0.75,-0.75,1.5
ZPOPMIN,server,kernel,enqueue_to_backlog,[kernel.kallsyms],6.0,5.0,5.5,5.75,-0.75,-0.5
ZPOPMIN,server,kernel,hook_socket_sendmsg,[kernel.kallsyms],3.0,1.0,0.5,1.75,-0.75,-2.5
ZPOPMIN,server,user,je_arena_ralloc_no_move,redis-server,2.5,0.5,0,1.25,-0.75,-2.5
ZPOPMIN,server,kernel,loopback_xmit,[kernel.kallsyms],0.5,0,1.0,0.75,-0.75,0.5
ZPOPMIN,server,user,processCommand,redis-server,4.5,3.0,3.0,3.75,-0.75,-1.5
ZPOPMIN,server,kernel,tcp_inbound_hash,[kernel.kallsyms],2.0,1.0,1.5,1.75,-0.75,-0.5
ZPOPMIN,client,kernel,__ip_local_out,[kernel.kallsyms],1.0,0,1.0,1.0,-1.0,0.0
ZPOPMIN,client,kernel,__local_bh_enable_ip,[kernel.kallsyms],2.0,1.0,2.0,2.0,-1.0,0.0
ZPOPMIN,client,kernel,_find_next_bit,[kernel.kallsyms],1.5,0.5,1.5,1.5,-1.0,0.0
ZPOPMIN,client,kernel,kfree,[kernel.kallsyms],0.5,0,1.5,1.0,-1.0,1.0
ZPOPMIN,client,user,malloc,libc.so.6,1.0,0.5,2.0,1.5,-1.0,1.0
ZPOPMIN,client,kernel,process_backlog,[kernel.kallsyms],1.5,0,0.5,1.0,-1.0,-1.0
ZPOPMIN,client,kernel,tcp_ack_update_rtt,[kernel.kallsyms],1.0,0,1.0,1.0,-1.0,0.0
ZPOPMIN,client,kernel,tcp_queue_rcv,[kernel.kallsyms],1.5,0.5,1.5,1.5,-1.0,0.0
ZPOPMIN,server,kernel,__copy_skb_header,[kernel.kallsyms],2.5,0.5,0.5,1.5,-1.0,-2.0
ZPOPMIN,server,kernel,__pcs_replace_full_main,[kernel.kallsyms],1.5,0,0.5,1.0,-1.0,-1.0
ZPOPMIN,server,user,__vdso_clock_gettime,[vdso],23.0,18.5,16.0,19.5,-1.0,-7.0
ZPOPMIN,server,kernel,aa_file_perm,[kernel.kallsyms],1.0,0,1.0,1.0,-1.0,0.0
ZPOPMIN,server,kernel,cubictcp_acked,[kernel.kallsyms],0,0,2.0,1.0,-1.0,2.0
ZPOPMIN,server,kernel,ipv4_dst_check,[kernel.kallsyms],1.5,0,0.5,1.0,-1.0,-1.0
ZPOPMIN,server,kernel,ipv4_mtu,[kernel.kallsyms],0,0,2.0,1.0,-1.0,2.0
ZPOPMIN,server,kernel,napi_consume_skb,[kernel.kallsyms],2.0,1.0,2.0,2.0,-1.0,0.0
ZPOPMIN,server,kernel,populate_seccomp_data,[kernel.kallsyms],3.5,1.5,1.5,2.5,-1.0,-2.0
ZPOPMIN,server,kernel,sk_page_frag_refill,[kernel.kallsyms],0.5,0,1.5,1.0,-1.0,1.0
ZPOPMIN,server,kernel,tcp_measure_rcv_mss,[kernel.kallsyms],1.5,0,0.5,1.0,-1.0,-1.0
ZPOPMIN,server,kernel,tcp_skb_entail,[kernel.kallsyms],1.0,1.5,4.0,2.5,-1.0,3.0
ZPOPMIN,client,kernel,__alloc_skb,[kernel.kallsyms],3.0,1.5,2.5,2.75,-1.25,-0.5
ZPOPMIN,client,kernel,_raw_spin_unlock_irqrestore,[kernel.kallsyms],11.0,8.0,7.5,9.25,-1.25,-3.5
ZPOPMIN,client,kernel,bpf_skops_write_hdr_opt.isra.0,[kernel.kallsyms],0.5,0,2.0,1.25,-1.25,1.5
ZPOPMIN,client,kernel,native_queued_spin_lock_slowpath,[kernel.kallsyms],4.0,2.0,2.5,3.25,-1.25,-1.5
ZPOPMIN,client,kernel,tcp_rcv_space_adjust,[kernel.kallsyms],6.5,5.0,6.0,6.25,-1.25,-0.5
ZPOPMIN,server,kernel,__mod_memcg_state,[kernel.kallsyms],1.0,0,1.5,1.25,-1.25,0.5
ZPOPMIN,server,kernel,__sk_dst_check,[kernel.kallsyms],1.5,0,1.0,1.25,-1.25,-0.5
ZPOPMIN,server,user,dictFind,redis-server,12.0,8.0,6.5,9.25,-1.25,-5.5
ZPOPMIN,server,kernel,ip_send_check,[kernel.kallsyms],1.5,1.5,4.0,2.75,-1.25,2.5
ZPOPMIN,server,user,lookupKey,redis-server,1.0,0,1.5,1.25,-1.25,0.5
ZPOPMIN,server,user,lookupKeyWrite,redis-server,1.5,0,1.0,1.25,-1.25,-0.5
ZPOPMIN,server,user,resetClient,redis-server,4.0,2.0,2.5,3.25,-1.25,-1.5
ZPOPMIN,server,kernel,skb_try_coalesce,[kernel.kallsyms],1.0,0,1.5,1.25,-1.25,0.5
ZPOPMIN,server,kernel,tcp_rate_check_app_limited,[kernel.kallsyms],2.0,0.5,1.5,1.75,-1.25,-0.5
ZPOPMIN,server,kernel,tcp_stream_alloc_skb,[kernel.kallsyms],2.0,1.0,2.5,2.25,-1.25,0.5
ZPOPMIN,server,kernel,tcp_tso_segs,[kernel.kallsyms],3.5,1.5,2.0,2.75,-1.25,-1.5
ZPOPMIN,client,user,redisReaderFeed,redis-benchmark,2.5,0.5,1.5,2.0,-1.5,-1.0
ZPOPMIN,client,kernel,skb_release_data,[kernel.kallsyms],2.5,0.5,1.5,2.0,-1.5,-1.0
ZPOPMIN,server,user,_sdsnewlen,redis-server,2.0,0,1.0,1.5,-1.5,-1.0
ZPOPMIN,server,user,dictSdsKeyCompare,redis-server,4.0,3.0,5.0,4.5,-1.5,1.0
ZPOPMIN,server,user,je_tcache_bin_flush_small,redis-server,4.5,2.5,3.5,4.0,-1.5,-1.0
ZPOPMIN,server,kernel,kmem_cache_free,[kernel.kallsyms],2.0,0.5,2.0,2.0,-1.5,0.0
ZPOPMIN,server,kernel,tcp_current_mss,[kernel.kallsyms],5.0,5.0,8.0,6.5,-1.5,3.0
ZPOPMIN,server,kernel,tcp_send_mss,[kernel.kallsyms],0.5,0.5,3.5,2.0,-1.5,3.0
ZPOPMIN,server,user,je_malloc,redis-server,10.0,11.0,15.5,12.75,-1.75,5.5
ZPOPMIN,server,kernel,lock_sock_nested,[kernel.kallsyms],3.0,0.5,1.5,2.25,-1.75,-1.5
ZPOPMIN,server,kernel,tcp_add_backlog,[kernel.kallsyms],2.5,0,1.0,1.75,-1.75,-1.5
ZPOPMIN,server,kernel,tcp_queue_rcv,[kernel.kallsyms],4.5,2.5,4.0,4.25,-1.75,-0.5
ZPOPMIN,client,kernel,__release_sock,[kernel.kallsyms],6.0,3.0,4.0,5.0,-2.0,-2.0
ZPOPMIN,server,kernel,__seccomp_filter,[kernel.kallsyms],6.0,2.5,3.0,4.5,-2.0,-3.0
ZPOPMIN,client,kernel,__tcp_send_ack.part.0,[kernel.kallsyms],2.0,0,2.5,2.25,-2.25,0.5
ZPOPMIN,client,user,__vdso_clock_gettime,[vdso],9.0,5.5,6.5,7.75,-2.25,-2.5
ZPOPMIN,client,kernel,populate_seccomp_data,[kernel.kallsyms],4.5,2.0,4.0,4.25,-2.25,-0.5
ZPOPMIN,server,kernel,__alloc_skb,[kernel.kallsyms],8.5,5.0,6.0,7.25,-2.25,-2.5
ZPOPMIN,server,kernel,__virt_addr_valid,[kernel.kallsyms],4.0,2.5,5.5,4.75,-2.25,1.5
ZPOPMIN,client,kernel,_copy_to_iter,[kernel.kallsyms],21.5,19.5,22.5,22.0,-2.5,1.0
ZPOPMIN,client,user,recv,libc.so.6,2.5,0.5,3.5,3.0,-2.5,1.0
ZPOPMIN,server,user,call,redis-server,14.0,10.0,11.0,12.5,-2.5,-3.0
ZPOPMIN,client,kernel,mutex_lock,[kernel.kallsyms],6.0,4.0,7.5,6.75,-2.75,1.5
ZPOPMIN,client,kernel,tcp_established_options,[kernel.kallsyms],4.5,2.0,5.0,4.75,-2.75,0.5
ZPOPMIN,server,user,createEmbeddedStringObject,redis-server,3.5,0.5,3.0,3.25,-2.75,-0.5
ZPOPMIN,client,kernel,__seccomp_filter,[kernel.kallsyms],5.0,2.5,6.0,5.5,-3.0,1.0
ZPOPMIN,client,kernel,tcp_ack,[kernel.kallsyms],6.0,3.0,6.5,6.25,-3.25,0.5
ZPOPMIN,server,kernel,_raw_spin_lock_bh,[kernel.kallsyms],7.5,3.5,6.0,6.75,-3.25,-1.5
ZPOPMIN,server,kernel,_find_next_bit,[kernel.kallsyms],6.0,2.0,5.0,5.5,-3.5,-1.0
ZPOPMIN,client,kernel,__skb_datagram_iter,[kernel.kallsyms],10.5,7.5,12.0,11.25,-3.75,1.5
ZPOPMIN,client,kernel,lock_sock_nested,[kernel.kallsyms],10.0,3.5,5.0,7.5,-4.0,-5.0
ZPOPMIN,client,user,redisReaderGetReply,redis-benchmark,21.5,15.0,17.0,19.25,-4.25,-4.5
ZPOPMIN,client,kernel,ep_send_events,[kernel.kallsyms],8.5,4.0,9.0,8.75,-4.75,0.5
ZPOPMIN,server,user,zfree,redis-server,34.0,27.0,29.5,31.75,-4.75,-4.5
ZPOPMIN,client,kernel,sock_rfree,[kernel.kallsyms],19.5,9.0,9.0,14.25,-5.25,-10.5
ZPOPMIN,server,kernel,native_queued_spin_lock_slowpath,[kernel.kallsyms],10.5,4.0,8.5,9.5,-5.5,-2.0
ZPOPMIN,server,kernel,tcp_rcv_established,[kernel.kallsyms],16.0,12.0,19.0,17.5,-5.5,3.0