Re: 2.6.0 performance problems
From: Thomas Molina
Date: Tue Dec 30 2003 - 09:17:02 EST
On Mon, 29 Dec 2003, Linus Torvalds wrote:
>
>
> On Mon, 29 Dec 2003, Thomas Molina wrote:
> >
> > It certainly looks like DMA is enabled.
>
> Indeed. Can you do a simple kernel profile? Either using oprofile or just
> even the old profiler. It should show something (hopefully obvious), since
> your load seems to have a _huge_ system load.
OK, recompiled with oprofile support and have read the documentation. I
must confess a failure to understand what data out of oprofile you want,
nor how to extract that data. I apologize for my simplemindedness. Maybe
there is a reference to using oprofile for kernel development.
In any case, attachment one is the resulot of:
opreport `bk export linux-2.5 linux-2.6-testa`
attachment two is the result of:
opreport -l vmlinux > vmlinux.txt
where the second command was done after stopping profiling. Is any of
this close to what you wanted? 635490 78.4683 vmlinux
106995 13.2114 bk
39834 4.9186 libc-2.3.2.so
19080 2.3559 libperl.so
2720 0.3359 perl
1368 0.1689 oprofiled
1155 0.1426 top
603 0.0745 ld-2.3.2.so
591 0.0730 libpthread-0.60.so
423 0.0522 bash
323 0.0399 troff
314 0.0388 uhci_hcd
207 0.0256 sendmail.sendmail
188 0.0232 libproc.so.2.0.17
122 0.0151 usbcore
94 0.0116 cupsd
66 0.0081 less
64 0.0079 grotty
43 0.0053 rm
35 0.0043 init
26 0.0032 libncurses.so.5.3
22 0.0027 libstdc++.so.5.0.5
18 0.0022 opreport
14 0.0017 hermes
11 0.0014 man
7 8.6e-04 ls
7 8.6e-04 orinoco
7 8.6e-04 killall5
6 7.4e-04 gunzip
5 6.2e-04 syslogd
4 4.9e-04 sunrpc
4 4.9e-04 crond
3 3.7e-04 libtermcap.so.2.0.8
3 3.7e-04 tbl
2 2.5e-04 grep
2 2.5e-04 libgcc_s-3.3.2-20031023.so.1
2 2.5e-04 Socket.so
1 1.2e-04 af_packet
1 1.2e-04 gawk
1 1.2e-04 libcrypt-2.3.2.so
1 1.2e-04 libdl-2.3.2.so
1 1.2e-04 unix
1 1.2e-04 groff
1 1.2e-04 which
1 1.2e-04 libncursesw.so.5.3
1 1.2e-04 libz.so.1.2.0.7
1 1.2e-04 gpm
CPU: PIII, speed 648.076 MHz (estimated)
Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 324038
vma samples % symbol name
c014a1a0 195865 29.5231 module_text_address
c0118510 90031 13.5705 mark_offset_tsc
c0111920 49842 7.5128 mask_and_ack_8259A
c014a2e0 23263 3.5065 kallsyms_lookup
c0141bb0 15389 2.3196 kernel_text_address
c01d1600 15017 2.2635 ext3_find_entry
c0111550 14170 2.1359 enable_8259A_irq
c0272560 13515 2.0371 cfb_imageblit
c0120330 8149 1.2283 kernel_map_pages
c0205f00 7819 1.1786 __io_virt_debug
c0156560 7685 1.1584 poison_obj
c01564d0 7497 1.1300 store_stackinfo
c0158fa0 7192 1.0841 kmem_cache_free
c019d580 7153 1.0782 __d_lookup
c018e380 5132 0.7736 link_path_walk
c010a700 4851 0.7312 irq_entries_start
c0158b20 4754 0.7166 kmem_cache_alloc
c0251e80 4429 0.6676 ide_outb
c01192b0 3676 0.5541 apm_bios_call_simple
c01565b0 3582 0.5399 scan_poisoned_obj
c0158560 3400 0.5125 free_block
c011fda0 3229 0.4867 __change_page_attr
c011ff70 2619 0.3948 change_page_attr
c026a7c0 2597 0.3915 sys_outbuf
c010afa0 2526 0.3807 apic_timer_interrupt
c0157fe0 2203 0.3321 cache_alloc_refill
c0133d10 2160 0.3256 run_timer_softirq
c019b250 2087 0.3146 prune_dcache
c01c9310 2045 0.3082 find_next_usable_block
c0251e10 2042 0.3078 ide_inb
c010cf80 1877 0.2829 do_IRQ
c0205ca0 1821 0.2745 atomic_dec_and_lock
c0203920 1703 0.2567 radix_tree_lookup
c0121360 1627 0.2452 schedule
c012e400 1533 0.2311 do_softirq
c0206350 1528 0.2303 __copy_from_user_ll
c026fff0 1479 0.2229 bitfill32
c01da480 1472 0.2219 ext3_permission
c0199f00 1358 0.2047 dput
c017b4b0 1307 0.1970 __find_get_block_slow
c0120d70 1298 0.1956 scheduler_tick
c017d920 1245 0.1877 __find_get_block
c01243b0 1204 0.1815 __might_sleep
c01637e0 1190 0.1794 do_anonymous_page
c01dc6c0 1163 0.1753 do_get_write_access
c0113220 1151 0.1735 timer_interrupt
c011bca0 1125 0.1696 smp_apic_timer_interrupt
c018e2e0 1108 0.1670 do_lookup
c01cbb40 1069 0.1611 ext3_new_inode
c010a587 1033 0.1557 sysenter_past_esp
c02062e0 1015 0.1530 __copy_to_user_ll
c0242ac0 916 0.1381 __make_request
c013b350 914 0.1378 do_sigaction
c0206040 911 0.1373 strncpy_from_user
c0129cc0 907 0.1367 profile_hook
c0152690 906 0.1366 buffered_rmqueue
c012e540 904 0.1363 raise_softirq
c0151fa0 889 0.1340 __rmqueue
c0155fd0 888 0.1338 dbg_redzone1
c017e370 885 0.1334 __block_prepare_write
c014d5e0 864 0.1302 find_get_page
c01daeb0 849 0.1280 start_this_handle
c0132ac0 839 0.1265 __mod_timer
c01de700 838 0.1263 journal_stop
c01e09a0 817 0.1231 journal_commit_transaction
c026a800 812 0.1224 move_buf_aligned
c01cfe40 798 0.1203 ext3_do_update_inode
c01ea540 782 0.1179 journal_add_journal_head
c010ca40 774 0.1167 handle_IRQ_event
c024e650 774 0.1167 ide_do_request
c017f7a0 750 0.1130 block_write_full_page
c02404b0 748 0.1127 blk_rq_map_sg
c01e00e0 746 0.1124 __journal_file_buffer
c0157ac0 738 0.1112 cache_grow
c015d030 736 0.1109 shrink_list
c027a420 721 0.1087 i8042_interrupt
c0152550 709 0.1069 free_hot_cold_page
c0133b80 705 0.1063 update_one_process
c0156000 700 0.1055 dbg_redzone2
c0157a80 694 0.1046 kmem_flagcheck
c01a0610 691 0.1042 find_inode_fast
c0111390 674 0.1016 disable_8259A_irq
c0163d70 669 0.1008 do_no_page
c01cfae0 656 0.0989 ext3_read_inode
c010af80 643 0.0969 common_interrupt
c015db10 642 0.0968 shrink_cache
c019c880 640 0.0965 d_alloc
c010a63a 638 0.0962 restore_all
c015e160 635 0.0957 refill_inactive_zone
c015bf10 627 0.0945 __pagevec_lru_add
c01660d0 624 0.0941 do_mmap_pgoff
c014f420 610 0.0919 generic_file_aio_write_nolock
c01cf6d0 599 0.0903 ext3_get_inode_block
c0134180 590 0.0889 do_timer
c019f7b0 590 0.0889 inode_init_once
c018f2d0 579 0.0873 path_lookup
c0107d80 570 0.0859 __switch_to
c0152910 568 0.0856 __alloc_pages
c01ddd00 564 0.0850 journal_dirty_metadata
c01567a0 564 0.0850 slab_destroy
c01cd1f0 563 0.0849 ext3_get_block_handle
c014d430 560 0.0844 unlock_page
c01ab6f0 554 0.0835 __mark_inode_dirty
c017ad20 551 0.0831 wake_up_buffer
c0157890 544 0.0820 cache_init_objs
c0151f40 544 0.0820 prep_new_page
c0119560 533 0.0803 apm_cpu_idle
c0204e70 527 0.0794 number
c0121d40 523 0.0788 __wake_up
c010cb60 516 0.0778 note_interrupt
c0150880 499 0.0752 mempool_alloc
c024f230 496 0.0748 ide_intr
c0248110 486 0.0733 as_completed_request
c01559b0 485 0.0731 do_page_cache_readahead
c017d810 484 0.0730 bh_lru_install
c0241ac0 480 0.0724 get_request
c01194b0 477 0.0719 apm_do_idle
c0133ca0 472 0.0711 update_process_times
c0203b20 471 0.0710 radix_tree_delete
c013c4e0 467 0.0704 notifier_call_chain
c018d850 461 0.0695 permission
c011ecf0 460 0.0693 do_page_fault
c02603d0 455 0.0686 ide_build_dmatable
c0133580 453 0.0683 del_timer
c01ad730 443 0.0668 do_mpage_readpage
c0243210 442 0.0666 generic_make_request
c01a00d0 441 0.0665 prune_icache
c0177a90 439 0.0662 get_unused_fd
c0247d30 437 0.0659 as_update_iohist
c0248d00 428 0.0645 as_add_request
c01e4e30 426 0.0642 __journal_clean_checkpoint_list
c024d160 422 0.0636 ide_end_request
c01205b0 422 0.0636 try_to_wake_up
c01203e0 420 0.0633 recalc_task_prio
c0121c60 413 0.0623 preempt_schedule
c01668c0 405 0.0610 find_vma
c0133a90 404 0.0609 update_wall_time_one_tick
c01cd660 403 0.0607 ext3_getblk
c018d660 399 0.0601 getname
c018e1c0 396 0.0597 follow_mount
c0180270 395 0.0595 bio_alloc
c0151a40 388 0.0585 bad_range
c01d4f10 387 0.0583 init_once
c0156040 386 0.0582 dbg_userword
c01881a0 384 0.0579 generic_fillattr
c0243660 377 0.0568 __end_that_request_first
c0203f80 374 0.0564 rb_erase
c01cf800 372 0.0561 ext3_get_inode_loc
c01c9bb0 371 0.0559 ext3_new_block
c01ca430 369 0.0556 ext3_check_dir_entry
c01ccb60 368 0.0555 ext3_get_branch
c0251e90 368 0.0555 ide_outbsync
c0151b50 362 0.0546 free_pages_bulk
c016a3d0 359 0.0541 page_add_rmap
c01184a0 358 0.0540 sched_clock
c01dee60 356 0.0537 __journal_unfile_buffer
c02412b0 354 0.0534 blk_run_queues
c0133b40 353 0.0532 update_wall_time
c027e980 350 0.0528 increment_tail
c0253260 344 0.0519 ide_execute_command
c0124a10 339 0.0511 prepare_to_wait
c01a1aa0 338 0.0509 iget_locked
c0150b60 331 0.0499 mempool_free
c0132a00 329 0.0496 internal_add_timer
c0111340 322 0.0485 end_8259A_irq
c013e000 322 0.0485 supplemental_group_member
c013f530 321 0.0484 worker_thread
c01a0c20 319 0.0481 get_new_inode_fast
c0251ed0 319 0.0481 ide_outl
c013bf80 309 0.0466 sys_rt_sigaction
c0156600 308 0.0464 check_poison_obj
c01cca00 308 0.0464 ext3_block_to_path
c02402b0 307 0.0463 blk_recount_segments
c014d0e0 305 0.0460 add_to_page_cache
c0177ee0 305 0.0460 fd_install
c016a320 301 0.0454 page_referenced
c01a4320 298 0.0449 dnotify_parent
c01546c0 297 0.0448 __set_page_dirty_nobuffers
c0179c90 297 0.0448 filp_ctor
c015b780 297 0.0448 mark_page_accessed
c018f8a0 296 0.0446 may_open
c0188950 294 0.0443 cp_new_stat64
c01409c0 294 0.0443 rcu_do_batch
c010b1d8 290 0.0437 page_fault
c022f570 289 0.0436 do_con_write
c0161410 288 0.0434 zap_pte_range
c015a070 286 0.0431 reap_timer_fnc
c017d560 284 0.0428 __brelse
c01c9830 284 0.0428 ext3_try_to_allocate
c02540a0 279 0.0421 do_rw_taskfile
c0249090 275 0.0415 as_queue_empty
c01781b0 274 0.0413 sys_close
c0205f50 272 0.0410 memcpy
c010a511 271 0.0408 ret_from_intr
c0188250 271 0.0408 vfs_getattr
c015baa0 268 0.0404 release_pages
c024e390 267 0.0402 start_request
c019f5f0 266 0.0401 alloc_inode
c01ea760 265 0.0399 __journal_remove_journal_head
c015b5c0 265 0.0399 activate_page
c0248990 265 0.0399 as_dispatch_request
c018f580 262 0.0395 __user_walk
c017f930 262 0.0395 submit_bh
c018de00 261 0.0393 real_lookup
c0203dc0 258 0.0389 __rb_erase_color
c010e480 254 0.0383 disable_irq_nosync
c01ada40 254 0.0383 mpage_readpages
c019cc10 253 0.0381 d_instantiate
c019e0b0 252 0.0380 d_rehash
c017dfa0 250 0.0377 __block_write_full_page
c0203840 248 0.0374 radix_tree_insert
c02050f0 248 0.0374 vsnprintf
c0243c30 246 0.0371 get_io_context
c0165b40 246 0.0371 vma_merge
c01d2180 245 0.0369 add_dirent_to_buf
c017da00 243 0.0366 __getblk
c01e5ef0 243 0.0366 journal_cancel_revoke
c01db830 242 0.0365 journal_start
c0158880 239 0.0360 cache_flusharray
c0118a60 238 0.0359 delay_tsc
c0164520 237 0.0357 handle_mm_fault
c0180550 235 0.0354 bio_add_page
c027e9b0 234 0.0353 sync_buffer
c017a190 231 0.0348 __fput
c0177870 231 0.0348 dentry_open
c01ea980 231 0.0348 journal_put_journal_head
c017faa0 231 0.0348 ll_rw_block
c018fa70 231 0.0348 open_namei
c01bae10 226 0.0341 task_statm
c0165840 223 0.0336 vma_link
c01ac050 222 0.0335 sync_sb_inodes
c0162860 221 0.0333 do_wp_page
c010cd90 221 0.0333 enable_irq
c014d8d0 220 0.0332 find_lock_page
c0179e30 219 0.0330 filp_dtor
c01dd7a0 219 0.0330 journal_dirty_data
c0248790 217 0.0327 as_move_to_dispatch
c017d110 217 0.0327 grow_dev_page
c0159dc0 216 0.0326 drain_array
c01dd5f0 213 0.0321 journal_get_undo_access
c014ea20 212 0.0320 filemap_nopage
c01d4650 210 0.0317 ext3_journal_start
c01d1bc0 210 0.0317 ext3_lookup
c014d330 210 0.0317 page_waitqueue
c013e040 209 0.0315 in_group_p
c01e7040 209 0.0315 journal_write_metadata_buffer
c01d04b0 208 0.0314 ext3_reserve_inode_write
c015c1b0 207 0.0312 __pagevec_lru_add_active
c0247490 206 0.0311 as_find_arq_hash
c0121cd0 204 0.0307 __wake_up_common
c017a630 203 0.0306 file_move
c02177e0 202 0.0304 add_timer_randomness
c0247440 202 0.0304 as_add_arq_hash
c017dd10 202 0.0304 create_empty_buffers
c017ac10 201 0.0303 __constant_c_and_count_memset
c0240aa0 201 0.0303 blk_plug_device
c01601f0 201 0.0303 blk_queue_bounce
c0247620 200 0.0301 as_choose_req
c0248400 200 0.0301 as_remove_queued_request
c01552d0 199 0.0300 file_ra_state_init
c01672f0 198 0.0298 do_munmap
c01520c0 198 0.0298 rmqueue_bulk
c0267fc0 197 0.0297 fbcon_redraw
c0240ed0 197 0.0297 generic_unplug_device
c0203cc0 195 0.0294 rb_insert_color
c017d410 190 0.0286 __getblk_slow
c027eb80 190 0.0286 add_event_entry
c02601c0 187 0.0282 ide_build_sglist
c0178060 186 0.0280 sys_open
c01cd990 185 0.0279 walk_page_buffers
c0180210 184 0.0277 bio_destructor
c017ad60 184 0.0277 unlock_buffer
c01c8b30 181 0.0273 ext3_get_group_desc
c0248fa0 180 0.0271 as_insert_request
c01d03f0 180 0.0271 ext3_writepage_trans_blocks
c0241a00 179 0.0270 freed_request
c0165520 179 0.0270 remove_shared_vm_struct
c0135d70 178 0.0268 rm_from_queue
c0217990 177 0.0267 add_disk_randomness
c02433a0 176 0.0265 submit_bio
c01e7410 175 0.0264 __log_space_left
c0180f00 174 0.0262 bio_endio
c01cda80 172 0.0259 ext3_prepare_write
c01ab980 171 0.0258 __sync_single_inode
c017a2b0 171 0.0258 fget
c01dd1d0 171 0.0258 journal_get_write_access
c0203730 169 0.0255 radix_tree_preload
c016a4f0 167 0.0252 page_remove_rmap
c01d05e0 166 0.0250 ext3_dirty_inode
c0260110 166 0.0250 ide_dma_intr
c018dd40 165 0.0249 path_release
c0166930 163 0.0246 find_vma_prev
c017bf60 162 0.0244 inode_has_buffers
c0242400 161 0.0243 drive_stat_acct
c02438a0 161 0.0243 end_that_request_last
c0248cb0 160 0.0241 as_next_request
c0134170 160 0.0241 run_local_timers
c0247bc0 159 0.0240 as_can_break_anticipation
c0155450 158 0.0238 read_pages
c012e380 156 0.0235 current_kernel_time
c0179fc0 156 0.0235 get_empty_filp
c0166780 156 0.0235 get_unmapped_area
c01a2700 156 0.0235 iput
c023f600 155 0.0234 clear_queue_congested
c01c2160 154 0.0232 collect_sigign_sigcatch
c0266600 154 0.0232 putcs_aligned
c0161690 152 0.0229 unmap_vmas
c025cdf0 151 0.0228 ide_do_rw_disk
c01d46c0 149 0.0225 __ext3_journal_stop
c0248530 149 0.0225 as_remove_dispatched_request
c017a7e0 148 0.0223 file_kill
c01a1d60 147 0.0222 __insert_inode_hash
c0124d10 147 0.0222 finish_wait
c013f150 147 0.0222 queue_work
c0242490 146 0.0220 disk_round_stats
c01a3100 144 0.0217 notify_change
c02040b0 144 0.0217 rb_next
c0240cd0 141 0.0213 blk_remove_plug
c01a2db0 141 0.0213 inode_change_ok
c0260910 139 0.0210 __ide_dma_read
c02478b0 139 0.0210 as_antic_stop
c01db7e0 139 0.0210 new_handle
c0188350 139 0.0210 vfs_lstat
c0247550 138 0.0208 as_add_arq_rb
c01a3ff0 138 0.0208 dnotify_flush
c01139d0 138 0.0208 sys_mmap2
c01d50a0 136 0.0205 ext3_clear_inode
c02555a0 136 0.0205 ide_cmd_type_parser
c0252760 135 0.0203 ide_wait_stat
c01d0580 134 0.0202 ext3_mark_inode_dirty
c023f250 133 0.0200 elv_queue_empty
c01c2af0 133 0.0200 proc_pid_stat
c026fbb0 132 0.0199 soft_cursor
c0253df0 131 0.0197 SELECT_DRIVE
c0205e80 131 0.0197 __const_udelay
c01cd510 130 0.0196 ext3_get_block
c014d510 129 0.0194 __lock_page
c017cff0 129 0.0194 create_buffers
c017afa0 129 0.0194 end_buffer_write_sync
c017a430 129 0.0194 fget_light
c01e5630 129 0.0194 find_revoke_record
c025cc40 128 0.0193 lba_28_rw_disk
c02499c0 127 0.0191 as_set_request
c025ca80 124 0.0187 get_command
c01a2cd0 124 0.0187 wake_up_inode
c027e440 123 0.0185 get_exec_dcookie
c0198bb0 123 0.0185 locks_remove_posix
c017ad00 122 0.0184 bh_waitq_head
c01d4e90 122 0.0184 ext3_alloc_inode
c0248640 121 0.0182 as_remove_request
c01ccd70 121 0.0182 ext3_alloc_branch
c0243b40 121 0.0182 put_io_context
c0180050 120 0.0181 alloc_buffer_head
c014dc10 120 0.0181 find_or_create_page
c01656f0 119 0.0179 find_vma_prepare
c0166ea0 119 0.0179 unmap_region
c01caf70 117 0.0176 ext3_release_file
c0255500 117 0.0176 ide_handler_parser
c0150d80 117 0.0176 mempool_alloc_slab
c01d0470 116 0.0175 ext3_mark_iloc_dirty
c0255460 116 0.0175 ide_pre_handler_parser
c0152e30 116 0.0175 nr_free_pages
c0177810 115 0.0173 filp_open
c0123f30 115 0.0173 io_schedule
c0160790 115 0.0173 pte_alloc_map
c0150070 114 0.0172 generic_file_aio_write
c0198e30 114 0.0172 locks_remove_flock
c01be9d0 114 0.0172 pid_revalidate
c01248a0 114 0.0172 remove_wait_queue
c017e740 112 0.0169 __block_commit_write
c0247760 112 0.0169 as_find_next_arq
c01d2730 112 0.0169 ext3_add_entry
c0178120 112 0.0169 filp_close
c016aad0 111 0.0167 __pte_chain_free
c0178f70 111 0.0167 do_sync_write
c017fc50 111 0.0167 drop_buffers
c0159340 110 0.0166 kfree
c0133510 110 0.0166 mod_timer
c01a0680 110 0.0166 new_inode
c015b8e0 109 0.0164 __page_cache_release
c0165a90 109 0.0164 can_vma_merge_before
c017dab0 107 0.0161 __bread
c0180110 107 0.0161 init_buffer_head
c01a27d0 107 0.0161 inode_times_differ
c0152cb0 106 0.0160 __get_free_pages
c01245c0 106 0.0160 add_wait_queue
c019d240 106 0.0160 d_splice_alias
c023f100 106 0.0160 elv_next_request
c017bac0 106 0.0160 end_buffer_async_write
c01cb020 106 0.0160 ext3_file_write
c0188b40 106 0.0160 inode_add_bytes
c0179030 106 0.0160 vfs_write
c01b34d0 104 0.0157 eventpoll_init_file
c01675c0 104 0.0157 sys_munmap
c010b008 103 0.0155 error_code
c01cdc60 103 0.0155 ext3_ordered_commit_write
c02299e0 102 0.0154 conv_uni_to_pc
c023f2f0 101 0.0152 elv_set_request
c0150da0 100 0.0151 mempool_free_slab
c023f570 99 0.0149 elv_try_last_merge
c01d0260 99 0.0149 ext3_setattr
c01a26e0 99 0.0149 generic_drop_inode
c015b880 99 0.0149 lru_add_drain
c016ab40 99 0.0149 pte_chain_alloc
c0158d40 98 0.0148 __kmalloc
c0249120 98 0.0148 as_merge
c01ff940 98 0.0148 cap_vm_enough_memory
c01cd010 98 0.0148 ext3_splice_branch
c01a2f30 98 0.0148 inode_setattr
c0220260 98 0.0148 opost_block
c017db70 98 0.0148 set_bh_page
c01a2820 97 0.0146 update_atime
c017fbf0 96 0.0145 check_ttfb_buffer
c0190c30 95 0.0143 sys_unlink
c027e780 95 0.0143 take_tasks_mm
c019fa60 94 0.0142 clear_inode
c018f7d0 94 0.0142 vfs_create
c017f8f0 93 0.0140 end_bio_bh_io_sync
c01cfa70 93 0.0140 ext3_set_inode_flags
c02475e0 92 0.0139 as_find_arq_rb
c026a990 92 0.0139 fb_get_buffer_offset
c0149b20 92 0.0139 get_ksymbol
c0252680 91 0.0137 drive_is_ready
c01ccce0 91 0.0137 ext3_find_goal
c01cafd0 90 0.0136 ext3_open_file
c01e08f0 90 0.0136 journal_end_buffer_io_sync
c0153f80 88 0.0133 balance_dirty_pages_ratelimited
c0181280 88 0.0133 bio_phys_segments
c010a544 88 0.0133 resume_kernel
c0180430 87 0.0131 bio_put
c023ef10 86 0.0130 __elv_add_request
c01ad4e0 86 0.0130 mpage_end_io_read
c017adc0 85 0.0128 __wait_on_buffer
c01294b0 85 0.0128 console_conditional_schedule
c01ea490 84 0.0127 journal_alloc_journal_head
c01da650 83 0.0125 ext3_init_acl
c01ce740 83 0.0125 ext3_set_aops
c019fb20 82 0.0124 dispose_list
c01ae160 82 0.0124 mpage_writepages
c01ac470 82 0.0124 writeback_inodes
c0199ec0 81 0.0122 d_free
c014c880 80 0.0121 __remove_from_page_cache
c0248710 80 0.0121 as_fifo_expired
c0159fc0 80 0.0121 drain_array_locked
c0188ac0 80 0.0121 sys_lstat64
c0260ab0 78 0.0118 __ide_dma_begin
c0260ba0 78 0.0118 __ide_dma_test_irq
c0247b00 78 0.0118 as_close_req
c025af00 78 0.0118 default_end_request
c0203c20 78 0.0118 radix_tree_node_ctor
c0204110 78 0.0118 rb_prev
c025c970 77 0.0116 __ide_do_rw_disk
c017af70 77 0.0116 end_buffer_read_sync
c01770f0 76 0.0115 sys_access
c0171f40 75 0.0113 free_page_and_swap_cache
c01e0380 75 0.0113 journal_file_buffer
c0194980 75 0.0113 sys_select
c017fd40 75 0.0113 try_to_free_buffers
c01ea6d0 74 0.0112 journal_grab_journal_head
c014d490 73 0.0110 end_page_writeback
c0205e10 72 0.0109 __delay
c0124ea0 72 0.0109 autoremove_wake_function
c019d520 72 0.0109 d_lookup
c01cba40 72 0.0109 find_group_other
c0177510 72 0.0109 sys_chmod
c01d3950 71 0.0107 ext3_orphan_del
c017dbb0 71 0.0107 try_to_release_page
c017c380 70 0.0106 __set_page_dirty_buffers
c023f390 70 0.0106 elv_completed_request
c01ced70 70 0.0106 ext3_free_data
c01ce010 70 0.0106 ext3_ordered_writepage
c017ef80 70 0.0106 generic_commit_write
c015b820 70 0.0106 lru_cache_add_active
c0194520 69 0.0104 do_select
c0240600 69 0.0104 ll_back_merge_fn
c0249950 63 0.0095 as_put_request
c024eb20 63 0.0095 do_ide_request
c0260c90 62 0.0093 __ide_dma_count
c012c7b0 62 0.0093 next_thread
c01615a0 62 0.0093 zap_pmd_range
c01657a0 61 0.0092 __vma_link
c02424e0 60 0.0090 __blk_put_request
c0243cb0 60 0.0090 copy_io_context
c0260860 60 0.0090 ide_start_dma
c017d510 60 0.0090 mark_buffer_dirty
c0178e40 60 0.0090 vfs_read
c02497b0 58 0.0087 as_work_handler
c0166f80 58 0.0087 detach_vmas_to_be_unmapped
c023f210 58 0.0087 elv_remove_request
c0243880 58 0.0087 end_that_request_first
c01ccc50 58 0.0087 ext3_find_near
c018d8a0 58 0.0087 get_write_access
c027e4b0 58 0.0087 lookup_dcookie
c0206180 57 0.0086 __copy_user_intel
c01c03a0 57 0.0086 get_tgid_list
c017ef00 56 0.0084 block_prepare_write
c017a170 56 0.0084 fput
c0161610 56 0.0084 unmap_page_range
c0154350 55 0.0083 do_writepages
c02492c0 54 0.0081 as_merged_request
c0270e90 54 0.0081 bitcpy
c01ea320 54 0.0081 journal_blocks_per_page
c018f410 53 0.0080 __lookup_hash
c01a2a30 53 0.0080 i_waitq_head
c017ae90 52 0.0078 __set_page_buffers
c027e5d0 52 0.0078 add_user_ctx_switch
c0120390 52 0.0078 effective_prio
c01a23c0 52 0.0078 generic_forget_inode
c0190a70 52 0.0078 vfs_unlink
c022e7f0 51 0.0077 do_con_trol
c02477e0 50 0.0075 as_antic_expired
c0199e80 50 0.0075 d_callback
c023ee30 50 0.0075 elv_merge
c0152df0 50 0.0075 free_pages
c017d0a0 50 0.0075 init_page_buffers
c017df30 50 0.0075 unmap_underlying_metadata
c0203c80 49 0.0074 __rb_rotate_right
c0121cb0 49 0.0074 default_wake_function
c0166d30 49 0.0074 free_pgtables
c0229bd0 49 0.0074 set_selection
c01791c0 49 0.0074 sys_write
c0247390 48 0.0072 as_get_io_context
c0131f70 47 0.0071 access_process_vm
c017ff80 47 0.0071 block_sync_page
c0140990 47 0.0071 call_rcu
c01d3010 47 0.0071 ext3_create
c014f040 47 0.0071 generic_file_mmap
c01a2900 47 0.0071 inode_update_time
c0166dd0 47 0.0071 unmap_vma
c014d2e0 46 0.0069 add_to_page_cache_lru
c01bdd30 46 0.0069 pid_alive
c01c8bf0 46 0.0069 read_block_bitmap
c023f330 45 0.0068 elv_put_request
c01d2f90 45 0.0068 ext3_add_nondir
c0180010 45 0.0068 recalc_bh_state
c0247520 44 0.0066 as_find_first_arq
c01cb160 44 0.0066 read_inode_bitmap
c0154910 44 0.0066 test_clear_page_dirty
c01337b0 43 0.0065 cascade
c01ad600 43 0.0065 mpage_alloc
c02037d0 43 0.0065 radix_tree_extend
c02473e0 42 0.0063 as_remove_merge_hints
c01c8c80 42 0.0063 ext3_free_blocks
c0108060 42 0.0063 get_wchan
c01be740 42 0.0063 task_dumpable
c01812b0 41 0.0062 bio_hw_segments
c01ce440 41 0.0062 ext3_readpages
c02036d0 41 0.0062 radix_tree_node_alloc
c014f2f0 41 0.0062 remove_suid
c01a3090 41 0.0062 setattr_mask
c0260b00 40 0.0060 __ide_dma_end
c027e680 39 0.0059 add_sample_entry
c0152da0 38 0.0057 __free_pages
c0134390 38 0.0057 schedule_timeout
c0203c40 37 0.0056 __rb_rotate_left
c01da800 37 0.0056 ext3_acl_chmod
c01291f0 37 0.0056 release_console_sem
c02afdf0 37 0.0056 tcp_poll
c01e38d0 36 0.0054 __try_to_free_cp_buf
c027e720 36 0.0054 add_sample
c0247cf0 36 0.0054 as_can_anticipate
c014df50 36 0.0054 do_generic_mapping_read
c015b3c0 36 0.0054 rotate_reclaimable_page
c021b060 36 0.0054 tty_write
c0248000 35 0.0053 as_update_arq
c02678d0 35 0.0053 fbcon_cursor
c0228e00 35 0.0053 inverse_translate
c0160b60 33 0.0050 copy_page_range
c0152670 33 0.0050 free_hot_page
c027a860 33 0.0050 i8042_timer_func
c01bd450 33 0.0050 proc_info_read
c02609e0 32 0.0048 __ide_dma_write
c015be70 32 0.0048 __pagevec_release_nonlru
c01a4c90 32 0.0048 lookup_mnt
c027e760 32 0.0048 release_mm
c0140090 32 0.0048 schedule_work
c0243b10 31 0.0047 kblockd_schedule_work
c0161be0 30 0.0045 follow_page
c01df4a0 30 0.0045 journal_try_to_free_buffers
c01e61c0 30 0.0045 journal_write_revoke_records
c015ef80 29 0.0044 balance_pgdat
c01cdb90 29 0.0044 ext3_journal_dirty_data
c0194420 29 0.0044 max_select_fd
c029b1b0 27 0.0041 dev_watchdog
c01cc9b0 27 0.0041 ext3_discard_prealloc
c01800b0 27 0.0041 free_buffer_head
c0140c20 27 0.0041 rcu_process_callbacks
c01a20d0 26 0.0039 generic_delete_inode
c0125370 26 0.0039 mmgrab
c019c400 26 0.0039 select_parent
c0179160 26 0.0039 sys_read
c0166e70 26 0.0039 unmap_vma_list
c01e4ee0 25 0.0038 __journal_remove_checkpoint
c0247930 25 0.0038 as_antic_timeout
c01cc9c0 25 0.0038 ext3_alloc_block
c01cd8e0 25 0.0038 ext3_bread
c01ce4b0 25 0.0038 ext3_releasepage
c027e940 25 0.0038 get_slots
c0161c80 25 0.0038 get_user_pages
c015b7c0 25 0.0038 lru_cache_add
c01b7ef0 25 0.0038 mb_cache_shrink_fn
c018cc40 25 0.0038 pipe_poll
c01c1510 25 0.0038 proc_readdir
c010a52c 25 0.0038 resume_userspace
c015ce00 25 0.0038 shrink_slab
c0282650 25 0.0038 sock_poll
c01d15d0 24 0.0036 ext3_update_dx_flag
c01ad5c0 24 0.0036 mpage_bio_submit
c0231300 24 0.0036 screen_glyph
c0229a90 24 0.0036 sel_pos
c02669e0 23 0.0035 accel_putcs
c0188a80 23 0.0035 sys_stat64
c018dd80 22 0.0033 cached_lookup
c0289270 22 0.0033 datagram_poll
c027ea90 22 0.0033 sync_cpu_buffers
c019fa00 21 0.0032 __iget
c01e7b00 21 0.0032 journal_next_log_block
c0224030 21 0.0032 write_chan
c022d800 20 0.0030 csi_K
c01cebf0 20 0.0030 ext3_clear_blocks
c012e650 20 0.0030 tasklet_action
c017aed0 19 0.0029 __clear_page_buffers
c01cf130 19 0.0029 ext3_truncate
c0188b00 19 0.0029 sys_fstat64
c01d4ee0 18 0.0027 ext3_destroy_inode
c01cb1f0 18 0.0027 ext3_free_inode
c01ca510 18 0.0027 ext3_readdir
c01dd220 18 0.0027 journal_get_create_access
c011e5d0 18 0.0027 pte_alloc_one
c0182880 18 0.0027 sync_supers
c014e400 17 0.0026 __generic_file_aio_read
c0165760 17 0.0026 __vma_link_rb
c019daf0 17 0.0026 d_delete
c019f750 17 0.0026 destroy_inode
c014e300 17 0.0026 file_read_actor
c0188ce0 17 0.0026 inode_sub_bytes
c01a3340 17 0.0026 is_bad_inode
c0155da0 17 0.0026 page_cache_readahead
c0194320 17 0.0026 poll_freewait
c0266300 16 0.0024 fb_flashcursor
c017cc20 16 0.0024 invalidate_inode_buffers
c01df040 16 0.0024 journal_unfile_buffer
c01883b0 16 0.0024 vfs_fstat
c0194110 15 0.0023 filldir64
c0260560 15 0.0023 ide_destroy_dmatable
c0123510 15 0.0023 idle_cpu
c01e7cf0 15 0.0023 journal_bmap
c01ea520 15 0.0023 journal_free_journal_head
c01ea900 15 0.0023 journal_remove_journal_head
c01e6150 15 0.0023 journal_switch_revoke_table
c0140a40 15 0.0023 rcu_check_quiescent_state
c01a0860 15 0.0023 unlock_new_inode
c01882f0 15 0.0023 vfs_stat
c0247860 14 0.0021 as_antic_waitreq
c0178d80 14 0.0021 do_sync_read
c026a170 14 0.0021 fbcon_screen_pos
c01c12a0 14 0.0021 proc_lookup
c02427e0 13 0.0020 attempt_merge
c01a2790 13 0.0020 bmap
c0270bb0 13 0.0020 cfb_fillrect
c01cc870 13 0.0020 ext3_put_inode
c0272c90 13 0.0020 input_event
c01bf340 13 0.0020 pid_delete_dentry
c01bc940 13 0.0020 proc_pid_cmdline
c0140e80 13 0.0020 rcu_check_callbacks
c0133810 13 0.0020 second_overflow
c0139b40 13 0.0020 sigprocmask
c0129190 12 0.0018 acquire_console_sem
c027e590 12 0.0018 add_kernel_ctx_switch
c027e6b0 12 0.0018 add_us_sample
c011ab30 12 0.0018 apm_event_handler
c02490f0 12 0.0018 as_latter_request
c01de0d0 12 0.0018 journal_release_buffer
c015be30 11 0.0017 __pagevec_release
c0194370 11 0.0017 __pollwait
c01969b0 11 0.0017 __posix_lock_file
c01abe30 11 0.0017 __writeback_single_inode
c0184bc0 11 0.0017 blkdev_writepage
c01d3700 11 0.0017 ext3_orphan_add
c01c3e00 11 0.0017 get_vmalloc_info
c01cdff0 11 0.0017 journal_dirty_data_fn
c018f4e0 11 0.0017 lookup_hash
c01c04d0 11 0.0017 proc_pid_readdir
c01c32b0 11 0.0017 proc_pid_statm
c01bf870 11 0.0017 proc_pident_lookup
c022c970 11 0.0017 set_cursor
c0190310 11 0.0017 sys_mkdir
c0152d70 10 0.0015 __pagevec_free
c023f280 10 0.0015 elv_latter_request
c01d3d20 10 0.0015 ext3_unlink
c017bdd0 10 0.0015 mark_buffer_async_write
c0152e80 10 0.0015 nr_used_zone_pages
c02200b0 10 0.0015 opost
c01bb510 10 0.0015 proc_delete_inode
c01c42b0 10 0.0015 show_stat
c0139de0 10 0.0015 sys_rt_sigprocmask
c012d720 10 0.0015 sys_time
c018d730 10 0.0015 vfs_permission
c01ab930 10 0.0015 write_inode
c01255d0 9 0.0014 copy_mm
c0266350 9 0.0014 cursor_timer_handler
c023f480 9 0.0014 elv_rq_merge_ok
c01cc8a0 9 0.0014 ext3_delete_inode
c01d3190 9 0.0014 ext3_mkdir
c010c590 9 0.0014 math_state_restore
c0220e50 9 0.0014 n_tty_receive_buf
c01c0990 9 0.0014 proc_file_read
c0206120 9 0.0014 strnlen_user
c0253e90 8 0.0012 SELECT_MASK
c017d760 8 0.0012 __bread_slow
c01e4f70 8 0.0012 __journal_insert_checkpoint
c027e550 8 0.0012 add_cpu_switch
c01191d0 8 0.0012 apm_bios_call
c01cdfd0 8 0.0012 bget_one
c0242750 8 0.0012 blk_congestion_wait
c0230440 8 0.0012 con_flush_chars
c0230270 8 0.0012 con_write
c012d3f0 8 0.0012 do_setitimer
c01cc670 8 0.0012 ext3_forget
c01cb770 8 0.0012 find_group_orlov
c022c8c0 8 0.0012 hide_cursor
c01bba00 8 0.0012 proc_root_readdir
c0134f50 8 0.0012 recalc_sigpending
c01a92b0 8 0.0012 seq_printf
c01badf0 8 0.0012 task_vsize
c01e05b0 7 0.0011 __journal_refile_buffer
c022c7d0 7 0.0011 add_softcursor
c01cdfe0 7 0.0011 bput_one
c0230310 7 0.0011 con_write_room
c0167640 7 0.0011 do_brk
c0192e70 7 0.0011 do_fcntl
c017f8b0 7 0.0011 generic_block_bmap
c0123f50 7 0.0011 io_schedule_timeout
c0194300 7 0.0011 poll_initwait
c01bcde0 7 0.0011 proc_pid_wchan
c0276790 7 0.0011 psmouse_interrupt
c019c840 7 0.0011 shrink_dcache_memory
c019ba00 7 0.0011 shrink_dcache_sb
c0120850 7 0.0011 wake_up_process
c0154950 6 9.0e-04 __pdflush
c01193d0 6 9.0e-04 apm_get_event
c01874d0 6 9.0e-04 chrdev_open
c01606e0 6 9.0e-04 clear_page_tables
c010b058 6 9.0e-04 device_not_available
c012d1d0 6 9.0e-04 do_getitimer
c01cdf50 6 9.0e-04 ext3_bmap
c014e620 6 9.0e-04 generic_file_aio_read
c0255590 6 9.0e-04 ide_post_handler_parser
c01e6540 6 9.0e-04 kjournald
c015cf70 6 9.0e-04 may_write_to_queue
c02230c0 6 9.0e-04 read_chan
c0114410 6 9.0e-04 restore_fpu
c01a0570 6 9.0e-04 shrink_icache_memory
c027eb20 6 9.0e-04 timer_ping
c0190250 6 9.0e-04 vfs_mkdir
c0193d60 6 9.0e-04 vfs_readdir
c01ad470 6 9.0e-04 writeback_acquire
c0152f80 5 7.5e-04 __get_page_state
c012e5b0 5 7.5e-04 __tasklet_schedule
c01bb2e0 5 7.5e-04 de_put
c022beb0 5 7.5e-04 do_update_region
c023ee60 5 7.5e-04 elv_merged_request
c01d2e70 5 7.5e-04 ext3_delete_entry
c01d8e50 5 7.5e-04 ext3_xattr_delete_inode
c015f190 5 7.5e-04 kswapd
c01c3d00 5 7.5e-04 loadavg_read_proc
c018fef0 5 7.5e-04 lookup_create
c0125220 5 7.5e-04 mmput
c0224270 5 7.5e-04 normal_poll
c016aab0 5 7.5e-04 pte_chain_ctor
c0194970 5 7.5e-04 select_bits_free
c01cc7a0 5 7.5e-04 start_transaction
c01655c0 5 7.5e-04 sys_brk
c0193940 5 7.5e-04 sys_ioctl
c021de90 5 7.5e-04 tty_ioctl
c021cfe0 5 7.5e-04 tty_poll
c027eb10 5 7.5e-04 wq_sync_buffers
c01ea340 4 6.0e-04 __jbd_kmalloc
c0275390 4 6.0e-04 atkbd_interrupt
c011a680 4 6.0e-04 check_events
c0219770 4 6.0e-04 check_tty_count
c0125e60 4 6.0e-04 copy_files
c01266a0 4 6.0e-04 copy_process
c0189360 4 6.0e-04 copy_strings
c0267620 4 6.0e-04 fbcon_clear
c0267830 4 6.0e-04 fbcon_putcs
c022d530 4 6.0e-04 lf
c0240890 4 6.0e-04 ll_merge_requests_fn
c01b5d60 4 6.0e-04 load_elf_binary
c0195620 4 6.0e-04 locks_init_lock
c0195e80 4 6.0e-04 locks_wake_up_blocks
c0124fd0 4 6.0e-04 mm_init
c010a54e 4 6.0e-04 need_resched
c0185340 4 6.0e-04 nr_blockdev_pages
c01c0c70 4 6.0e-04 proc_file_lseek
c01be8e0 4 6.0e-04 proc_pid_make_inode
c029fd20 4 6.0e-04 rt_check_expire
c0194940 4 6.0e-04 select_bits_alloc
c02797b0 4 6.0e-04 serio_interrupt
c015ec80 4 6.0e-04 shrink_zone
c0172630 4 6.0e-04 swap_info_get
c0278840 4 6.0e-04 synaptics_parse_hw_state
c0278a60 4 6.0e-04 synaptics_process_packet
c0194240 4 6.0e-04 sys_getdents64
c022cc00 4 6.0e-04 vc_cons_allocated
c0225d20 4 6.0e-04 vt_ioctl
c015f290 4 6.0e-04 wakeup_kswapd
c0154110 4 6.0e-04 wb_kupdate
c01ad4a0 4 6.0e-04 writeback_release
c0108bd0 3 4.5e-04 __down_trylock
c0129db0 3 4.5e-04 __unhash_process
c0266820 3 4.5e-04 accel_clear
c01404d0 3 4.5e-04 attach_pid
c0172c50 3 4.5e-04 can_share_swap_page
c0165ae0 3 4.5e-04 can_vma_merge_after
c01ff4d0 3 4.5e-04 cap_bprm_compute_creds
c026a2f0 3 4.5e-04 fbcon_invert_region
c01785c0 3 4.5e-04 generic_file_llseek
c011a620 3 4.5e-04 get_event
c0153000 3 4.5e-04 get_page_state
c01dada0 3 4.5e-04 get_transaction
c022d3d0 3 4.5e-04 gotoxy
c0155f20 3 4.5e-04 handle_ra_miss
c021b820 3 4.5e-04 init_dev
c0155000 3 4.5e-04 pdflush_operation
c011e640 3 4.5e-04 pgd_ctor
c018c5d0 3 4.5e-04 pipe_read
c01bb780 3 4.5e-04 proc_get_inode
c0134380 3 4.5e-04 process_timeout
c0177d70 3 4.5e-04 put_unused_fd
c021c190 3 4.5e-04 release_dev
c0204630 3 4.5e-04 rwsem_down_failed_common
c02055b0 3 4.5e-04 sprintf
c0278eb0 3 4.5e-04 synaptics_validate_byte
c017fb30 3 4.5e-04 sync_dirty_buffer
c012cef0 3 4.5e-04 sys_wait4
c016a670 3 4.5e-04 try_to_unmap_one
c021a930 3 4.5e-04 tty_hung_up_p
c0248070 3 4.5e-04 update_write_batch
c02179d0 2 3.0e-04 SHATransform
c017d5a0 2 3.0e-04 __bforget
c019eb80 2 3.0e-04 __d_path
c0135b30 2 3.0e-04 __dequeue_signal
c01084f0 2 3.0e-04 __down
c027e640 2 3.0e-04 add_cookie_switch
c02472e0 2 3.0e-04 alloc_as_io_context
c011abc0 2 3.0e-04 apm_mainloop
c01957f0 2 3.0e-04 assign_type
c0275300 2 3.0e-04 atkbd_report_key
c017e7d0 2 3.0e-04 block_read_full_page
c01cc760 2 3.0e-04 blocks_for_truncate
c022c0b0 2 3.0e-04 build_attr
c0187f80 2 3.0e-04 cdev_get
c018b490 2 3.0e-04 compute_creds
c0230330 2 3.0e-04 con_chars_in_buffer
c02302c0 2 3.0e-04 con_put_char
c01b5540 2 3.0e-04 create_elf_tables
c022d630 2 3.0e-04 csi_J
c018daf0 2 3.0e-04 deny_write_access
c018ba60 2 3.0e-04 do_execve
c010a360 2 3.0e-04 do_signal
c0163220 2 3.0e-04 do_swap_page
c01281c0 2 3.0e-04 do_syslog
c0124ef0 2 3.0e-04 dup_task_struct
c0166990 2 3.0e-04 expand_stack
c01ce420 2 3.0e-04 ext3_readpage
c01d0210 2 3.0e-04 ext3_write_inode
c0193590 2 3.0e-04 fasync_helper
c02681d0 2 3.0e-04 fbcon_scroll
c01983e0 2 3.0e-04 fcntl_setlk
c0166cb0 2 3.0e-04 find_extend_vma
c014dcd0 2 3.0e-04 find_get_pages
c01356c0 2 3.0e-04 flush_signal_handlers
c01183d0 2 3.0e-04 get_offset_tsc
c018d2c0 2 3.0e-04 get_pipe_inode
c0136820 2 3.0e-04 group_send_sig_info
c010a050 2 3.0e-04 handle_signal
c0112390 2 3.0e-04 init_new_context
c01e5420 2 3.0e-04 insert_revoke_hash
c01e2fb0 2 3.0e-04 journal_brelse_array
c022b7e0 2 3.0e-04 kbd_keycode
c0195600 2 3.0e-04 locks_alloc_lock
c01956f0 2 3.0e-04 locks_copy_lock
c0195f60 2 3.0e-04 locks_delete_lock
c0149c80 2 3.0e-04 module_address_lookup
c02746b0 2 3.0e-04 mousedev_abs_event
c0274800 2 3.0e-04 mousedev_event
c021fec0 2 3.0e-04 n_tty_chars_in_buffer
c0224bf0 2 3.0e-04 n_tty_ioctl
c028be40 2 3.0e-04 netif_receive_skb
c01c3cb0 2 3.0e-04 proc_calc_metrics
c01bc0e0 2 3.0e-04 proc_exe_link
c01bdac0 2 3.0e-04 proc_pid_readlink
c01bb980 2 3.0e-04 proc_root_lookup
c01bfad0 2 3.0e-04 proc_tgid_base_lookup
c0140a00 2 3.0e-04 rcu_start_batch
c0204240 2 3.0e-04 rwsem_wake
c0114940 2 3.0e-04 save_i387
c01a8ba0 2 3.0e-04 seq_read
c01a7cf0 2 3.0e-04 set_fs_pwd
c0189930 2 3.0e-04 setup_arg_pages
c0175c50 2 3.0e-04 si_swapinfo
c0204e30 2 3.0e-04 skip_atoi
c01931b0 2 3.0e-04 sys_fcntl64
c012d7e0 2 3.0e-04 sys_gettimeofday
c0178c80 2 3.0e-04 sys_llseek
c0190950 2 3.0e-04 sys_rmdir
c01234f0 2 3.0e-04 task_prio
c01a8ed0 2 3.0e-04 traverse
c01cc800 2 3.0e-04 try_to_extend_transaction
c021ce10 2 3.0e-04 tty_release
c022c1c0 2 3.0e-04 update_attr
c0205580 2 3.0e-04 vsprintf
c012cb10 2 3.0e-04 wait_task_zombie
c01d7203 1 1.5e-04 .text.lock.super
c01352e0 1 1.5e-04 __exit_sighand
c014ca60 1 1.5e-04 __filemap_fdatawrite
c0165a10 1 1.5e-04 __insert_vm_struct
c01e5050 1 1.5e-04 __journal_drop_transaction
c01df250 1 1.5e-04 __journal_try_to_free_buffer
c0193690 1 1.5e-04 __kill_fasync
c0203980 1 1.5e-04 __lookup
c0217930 1 1.5e-04 add_mouse_randomness
c0247810 1 1.5e-04 as_antic_waitnext
c02494a0 1 1.5e-04 as_merged_requests
c02410c0 1 1.5e-04 blk_unplug_timeout
c0167880 1 1.5e-04 build_mmap_rb
c0128bf0 1 1.5e-04 call_console_drivers
c0188030 1 1.5e-04 cdev_put
c0135e00 1 1.5e-04 check_kill_permission
c0229ad0 1 1.5e-04 clear_selection
c01e6520 1 1.5e-04 commit_timeout
c0230490 1 1.5e-04 con_open
c01fd8b0 1 1.5e-04 copy_semundo
c01895d0 1 1.5e-04 copy_strings_kernel
c0125e20 1 1.5e-04 count_open_files
c027e1b0 1 1.5e-04 cpu_buffer_reset
c0135ce0 1 1.5e-04 dequeue_signal
c0140670 1 1.5e-04 detach_pid
c0127720 1 1.5e-04 do_fork
c0112ba0 1 1.5e-04 do_gettimeofday
c01bd9f0 1 1.5e-04 do_proc_readlink
c0179360 1 1.5e-04 do_readv_writev
c01826c0 1 1.5e-04 drop_super
c012ca30 1 1.5e-04 eligible_child
c0128d30 1 1.5e-04 emit_log_char
c027ef10 1 1.5e-04 event_buffer_read
c0189c40 1 1.5e-04 exec_mmap
c01a33f0 1 1.5e-04 expand_fd_array
c01ca330 1 1.5e-04 ext3_bg_has_super
c01cc5d0 1 1.5e-04 ext3_count_free_inodes
c01d7260 1 1.5e-04 ext3_follow_link
c01d3b90 1 1.5e-04 ext3_rmdir
c0217b20 1 1.5e-04 extract_entropy
c026a1f0 1 1.5e-04 fbcon_getxy
c0267790 1 1.5e-04 fbcon_putc
c0193760 1 1.5e-04 file_ioctl
c0169ce0 1 1.5e-04 filemap_sync_pte
c0140780 1 1.5e-04 find_task_by_pid
c018a040 1 1.5e-04 flush_old_exec
c0135080 1 1.5e-04 flush_sigqueue
c0107750 1 1.5e-04 flush_thread
c021ea10 1 1.5e-04 flush_to_ldisc
c02e1bb0 1 1.5e-04 fn_hash_lookup
c017b060 1 1.5e-04 fsync_super
c0186e50 1 1.5e-04 generic_writepages
c01ca400 1 1.5e-04 get_dtype
c01a49d0 1 1.5e-04 get_filesystem_list
c01390f0 1 1.5e-04 get_signal_to_deliver
c0172150 1 1.5e-04 get_swap_page
c022d490 1 1.5e-04 gotoxay
c01801a0 1 1.5e-04 init_buffer
c01956d0 1 1.5e-04 init_once
c015c5b0 1 1.5e-04 invalidate_complete_page
c015cab0 1 1.5e-04 invalidate_mapping_pages
c02a5ae0 1 1.5e-04 ip_local_deliver
c02a5d40 1 1.5e-04 ip_rcv
c01e9070 1 1.5e-04 journal_check_used_features
c01de0f0 1 1.5e-04 journal_forget
c01e7d90 1 1.5e-04 journal_get_descriptor_buffer
c01e5d90 1 1.5e-04 journal_revoke
c01e9140 1 1.5e-04 journal_set_features
c022b0f0 1 1.5e-04 k_shift
c022bb30 1 1.5e-04 kbd_event
c01b59f0 1 1.5e-04 load_elf_interp
c0195f10 1 1.5e-04 locks_insert_lock
c01e7790 1 1.5e-04 log_wait_commit
c01c3f70 1 1.5e-04 meminfo_read_proc
c0274cc0 1 1.5e-04 mousedev_packet
c0168b10 1 1.5e-04 move_one_page
c01adb80 1 1.5e-04 mpage_readpage
c01bf370 1 1.5e-04 name_to_int
c028ba60 1 1.5e-04 netif_rx
c0189b00 1 1.5e-04 open_exec
c01b5510 1 1.5e-04 padzero
c02a5560 1 1.5e-04 peer_check_expire
c011e820 1 1.5e-04 pgd_dtor
c018cdb0 1 1.5e-04 pipe_read_fasync
c018cfc0 1 1.5e-04 pipe_write_release
c0230c90 1 1.5e-04 poke_blanked_console
c0128da0 1 1.5e-04 printk
c01bb5e0 1 1.5e-04 proc_alloc_inode
c01bb660 1 1.5e-04 proc_destroy_inode
c01bfd80 1 1.5e-04 proc_pid_flush
c01bc540 1 1.5e-04 proc_root_link
c028bff0 1 1.5e-04 process_backlog
c0189620 1 1.5e-04 put_dirty_page
c012af70 1 1.5e-04 put_fs_struct
c022a710 1 1.5e-04 puts_queue
c0129e50 1 1.5e-04 release_task
c0110010 1 1.5e-04 release_x86_irqs
c018b720 1 1.5e-04 remove_arg_zero
c02a09c0 1 1.5e-04 rt_intern_hash
c02041c0 1 1.5e-04 rwsem_down_read_failed
c0120bd0 1 1.5e-04 schedule_tail
c018b7b0 1 1.5e-04 search_binary_handler
c01a90a0 1 1.5e-04 seq_lseek
c0192210 1 1.5e-04 set_close_on_exec
c0109bb0 1 1.5e-04 setup_frame
c02887a0 1 1.5e-04 skb_recv_datagram
c0285950 1 1.5e-04 sock_alloc_send_pskb
c0282c60 1 1.5e-04 sock_create
c0166fe0 1 1.5e-04 split_vma
c0175e30 1 1.5e-04 swap_duplicate
c01729c0 1 1.5e-04 swap_entry_free
c0171200 1 1.5e-04 swap_readpage
c0278f00 1 1.5e-04 synaptics_process_byte
c0134270 1 1.5e-04 sys_alarm
c0177240 1 1.5e-04 sys_chdir
c0107fe0 1 1.5e-04 sys_execve
c013e480 1 1.5e-04 sys_getrlimit
c0178bc0 1 1.5e-04 sys_lseek
c0134560 1 1.5e-04 sys_nanosleep
c013e0a0 1 1.5e-04 sys_newuname
c0194ff0 1 1.5e-04 sys_poll
c013dda0 1 1.5e-04 sys_setsid
c0109520 1 1.5e-04 sys_sigreturn
c0283f30 1 1.5e-04 sys_socketcall
c0179740 1 1.5e-04 sys_writev
c010a62f 1 1.5e-04 syscall_exit
c02c1810 1 1.5e-04 tcp_connect
c02300c0 1 1.5e-04 tioclinux
c016a950 1 1.5e-04 try_to_unmap
c021d060 1 1.5e-04 tty_fasync
c021c900 1 1.5e-04 tty_open
c02243f0 1 1.5e-04 tty_wait_until_sent
c02d2a00 1 1.5e-04 udp_queue_rcv_skb
c02d3290 1 1.5e-04 udp_rcv
c01c3eb0 1 1.5e-04 uptime_read_proc
c0190760 1 1.5e-04 vfs_rmdir
c0163030 1 1.5e-04 vmtruncate
c0120880 1 1.5e-04 wake_up_state
c01ad490 1 1.5e-04 writeback_in_progress