请教各位大神,底层用nvme 盘,最近内核碰到异常,但是没有找到nvme盘相关的类似这种 0000:5f:00.0: Failed status: 3, reset controller/Cancelling I/O 629 QID 44 错误,iostatus 也正常,只是业务线程变成D状态:
Mar 25 07:26:43 ssd-10 kernel: [12118908.620934] INFO: task kswapd1:227 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.621105] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.621417] kswapd1 D 0000000000000000 0 227 2 0x00000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.621733] ffff8810395938f0 0000000000000046 ffff881039c5de20 ffff881039593fd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.622061] ffff881039593fd8 ffff881039593fd8 ffff881039c5de20 ffff88010731b558
Mar 25 07:26:43 ssd-10 kernel: [12118908.622384] ffff88010731b560 7fffffffffffffff ffff881039c5de20 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.622705] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.622867] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.623038] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.623207] [<ffffffff812f0ce4>] ? blk_finish_plug+0x14/0x40
Mar 25 07:26:43 ssd-10 kernel: [12118908.623395] [<ffffffffa079ba84>] ? _xfs_buf_ioapply+0x334/0x460 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.627805] [<ffffffff8168cbd6>] wait_for_completion+0x116/0x170
Mar 25 07:26:43 ssd-10 kernel: [12118908.627979] [<ffffffff810c54e0>] ? wake_up_state+0x20/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.628156] [<ffffffffa079dbc4>] ? xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628332] [<ffffffffa079d7b6>] xfs_buf_submit_wait+0xe6/0x1d0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628509] [<ffffffffa079dbc4>] xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628684] [<ffffffffa07a611b>] xfs_reclaim_inode+0x34b/0x370 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628862] [<ffffffffa07a63a7>] xfs_reclaim_inodes_ag+0x267/0x390 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629039] [<ffffffff810c54a3>] ? wake_up_process+0x23/0x40
Mar 25 07:26:43 ssd-10 kernel: [12118908.629217] [<ffffffffa07a7033>] xfs_reclaim_inodes_nr+0x33/0x40 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629397] [<ffffffffa07b6a05>] xfs_fs_free_cached_objects+0x15/0x20 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629709] [<ffffffff812010d8>] prune_super+0xe8/0x170
Mar 25 07:26:43 ssd-10 kernel: [12118908.629877] [<ffffffff811947b3>] shrink_slab+0x163/0x330
Mar 25 07:26:43 ssd-10 kernel: [12118908.630047] [<ffffffff811f5b37>] ? vmpressure+0x87/0x90
Mar 25 07:26:43 ssd-10 kernel: [12118908.630212] [<ffffffff811985b1>] balance_pgdat+0x4b1/0x5e0
Mar 25 07:26:43 ssd-10 kernel: [12118908.630376] [<ffffffff81198853>] kswapd+0x173/0x450
Mar 25 07:26:43 ssd-10 kernel: [12118908.630541] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
Mar 25 07:26:43 ssd-10 kernel: [12118908.630706] [<ffffffff811986e0>] ? balance_pgdat+0x5e0/0x5e0
Mar 25 07:26:43 ssd-10 kernel: [12118908.630872] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
Mar 25 07:26:43 ssd-10 kernel: [12118908.631043] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:26:43 ssd-10 kernel: [12118908.631215] [<ffffffff81697758>] ret_from_fork+0x58/0x90
Mar 25 07:26:43 ssd-10 kernel: [12118908.631380] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:26:43 ssd-10 kernel: [12118908.631589] INFO: task tabletserver:196922 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.631901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.632213] tabletserver D 0000000000000000 0 196922 1 0x00000080
Mar 25 07:26:43 ssd-10 kernel: [12118908.632527] ffff8804307fb6e0 0000000000000082 ffff88001d84de20 ffff8804307fbfd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.632848] ffff8804307fbfd8 ffff8804307fbfd8 ffff88001d84de20 ffff88202d05f630
Mar 25 07:26:43 ssd-10 kernel: [12118908.633174] 7fffffffffffffff ffff88001d84de20 0000000000000002 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.633494] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.633648] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.633811] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.633998] [<ffffffffa07bba76>] ? xfs_trans_add_item+0x46/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634180] [<ffffffffa07cccf5>] ? _xfs_trans_bjoin+0x45/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634360] [<ffffffffa07cd227>] ? xfs_trans_read_buf_map+0x247/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634670] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:26:43 ssd-10 kernel: [12118908.634846] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635033] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635201] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:26:43 ssd-10 kernel: [12118908.635363] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:26:43 ssd-10 kernel: [12118908.635535] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635710] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635889] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636068] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636251] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636430] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636605] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636781] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636966] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637142] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637308] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:26:43 ssd-10 kernel: [12118908.637482] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637658] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637832] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.638003] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:26:43 ssd-10 kernel: [12118908.638168] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:26:43 ssd-10 kernel: [12118908.638330] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:26:43 ssd-10 kernel: [12118908.638493] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:26:43 ssd-10 kernel: [12118908.638656] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:26:43 ssd-10 kernel: [12118908.638820] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:26:43 ssd-10 kernel: [12118908.638993] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:26:43 ssd-10 kernel: [12118908.639158] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.639321] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:26:43 ssd-10 kernel: [12118908.639490] INFO: task tabletserver:196951 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.639799] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.640117] tabletserver D 0000000000000000 0 196951 1 0x00000080
Mar 25 07:26:43 ssd-10 kernel: [12118908.640431] ffff8808597fba00 0000000000000082 ffff88003ddbedd0 ffff8808597fbfd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.640752] ffff8808597fbfd8 ffff8808597fbfd8 ffff88003ddbedd0 ffff88202d05f630
Mar 25 07:26:43 ssd-10 kernel: [12118908.641079] 7fffffffffffffff ffff88003ddbedd0 0000000000000002 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.641399] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.641553] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.641716] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.641903] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642083] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642250] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:26:43 ssd-10 kernel: [12118908.642425] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642592] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:26:43 ssd-10 kernel: [12118908.642753] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:26:43 ssd-10 kernel: [12118908.642933] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643108] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643283] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643458] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643638] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643817] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644002] [<ffffffffa07b0c0c>] xfs_iunlink+0x4c/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644179] [<ffffffffa07b0d76>] xfs_droplink+0x56/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644354] [<ffffffffa07b1a1f>] xfs_remove+0x21f/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644529] [<ffffffffa07ac08a>] xfs_vn_unlink+0x5a/0xa0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644694] [<ffffffff8120e9f1>] vfs_unlink+0xe1/0x150
Mar 25 07:26:43 ssd-10 kernel: [12118908.644866] [<ffffffff8120fcae>] do_unlinkat+0x26e/0x2b0
Mar 25 07:26:43 ssd-10 kernel: [12118908.645030] [<ffffffff81204071>] ? SYSC_newlstat+0x41/0x60
Mar 25 07:26:43 ssd-10 kernel: [12118908.645194] [<ffffffff81210bf6>] SyS_unlink+0x16/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.645357] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:27:01 ssd-10 systemd[1]: Started Session 501314 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Starting Session 501314 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Started Session 501315 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Starting Session 501315 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Started Session 501317 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Starting Session 501317 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Started Session 501316 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Starting Session 501316 of user root.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366046] INFO: task kswapd1:227 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366221] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366532] kswapd1 D 0000000000000000 0 227 2 0x00000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.366847] ffff8810395938f0 0000000000000046 ffff881039c5de20 ffff881039593fd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.367178] ffff881039593fd8 ffff881039593fd8 ffff881039c5de20 ffff88010731b558
Mar 25 07:28:43 ssd-10 kernel: [12119028.367498] ffff88010731b560 7fffffffffffffff ffff881039c5de20 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.367819] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.367981] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.368153] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.368323] [<ffffffff812f0ce4>] ? blk_finish_plug+0x14/0x40
Mar 25 07:28:43 ssd-10 kernel: [12119028.368506] [<ffffffffa079ba84>] ? _xfs_buf_ioapply+0x334/0x460 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.368674] [<ffffffff8168cbd6>] wait_for_completion+0x116/0x170
Mar 25 07:28:43 ssd-10 kernel: [12119028.368843] [<ffffffff810c54e0>] ? wake_up_state+0x20/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.369019] [<ffffffffa079dbc4>] ? xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369200] [<ffffffffa079d7b6>] xfs_buf_submit_wait+0xe6/0x1d0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369377] [<ffffffffa079dbc4>] xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369553] [<ffffffffa07a611b>] xfs_reclaim_inode+0x34b/0x370 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369731] [<ffffffffa07a63a7>] xfs_reclaim_inodes_ag+0x267/0x390 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369900] [<ffffffff810c54a3>] ? wake_up_process+0x23/0x40
Mar 25 07:28:43 ssd-10 kernel: [12119028.370084] [<ffffffffa07a7033>] xfs_reclaim_inodes_nr+0x33/0x40 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.370264] [<ffffffffa07b6a05>] xfs_fs_free_cached_objects+0x15/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.370575] [<ffffffff812010d8>] prune_super+0xe8/0x170
Mar 25 07:28:43 ssd-10 kernel: [12119028.370742] [<ffffffff811947b3>] shrink_slab+0x163/0x330
Mar 25 07:28:43 ssd-10 kernel: [12119028.370907] [<ffffffff811f5b37>] ? vmpressure+0x87/0x90
Mar 25 07:28:43 ssd-10 kernel: [12119028.371079] [<ffffffff811985b1>] balance_pgdat+0x4b1/0x5e0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371244] [<ffffffff81198853>] kswapd+0x173/0x450
Mar 25 07:28:43 ssd-10 kernel: [12119028.371410] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.371574] [<ffffffff811986e0>] ? balance_pgdat+0x5e0/0x5e0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371739] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371901] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:28:43 ssd-10 kernel: [12119028.372079] [<ffffffff81697758>] ret_from_fork+0x58/0x90
Mar 25 07:28:43 ssd-10 kernel: [12119028.372243] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:28:43 ssd-10 kernel: [12119028.372452] INFO: task tabletserver:196922 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.372760] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.373075] tabletserver D 0000000000000000 0 196922 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.373391] ffff8804307fb6e0 0000000000000082 ffff88001d84de20 ffff8804307fbfd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.373712] ffff8804307fbfd8 ffff8804307fbfd8 ffff88001d84de20 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.374038] 7fffffffffffffff ffff88001d84de20 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.374360] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.374514] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.374676] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.374854] [<ffffffffa07bba76>] ? xfs_trans_add_item+0x46/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375045] [<ffffffffa07cccf5>] ? _xfs_trans_bjoin+0x45/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375226] [<ffffffffa07cd227>] ? xfs_trans_read_buf_map+0x247/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375536] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.375712] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375890] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376061] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.376225] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.376396] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376571] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376746] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376921] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377108] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377289] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377463] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377640] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377816] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377992] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.378163] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.378337] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.378512] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.382937] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.383107] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.383270] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:28:43 ssd-10 kernel: [12119028.383433] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:28:43 ssd-10 kernel: [12119028.383596] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.383758] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:28:43 ssd-10 kernel: [12119028.383922] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.384095] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:28:43 ssd-10 kernel: [12119028.384261] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.384422] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:28:43 ssd-10 kernel: [12119028.384589] INFO: task tabletserver:196928 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.384897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.385211] tabletserver D 0000000000000000 0 196928 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.385526] ffff880f813df6e0 0000000000000082 ffff88005aebce70 ffff880f813dffd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.385847] ffff880f813dffd8 ffff880f813dffd8 ffff88005aebce70 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.386171] 7fffffffffffffff ffff88005aebce70 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.386492] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.386646] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.386808] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.386995] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387172] [<ffffffffa0774ccf>] ? xfs_btree_log_block+0x7f/0xf0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387340] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.387506] [<ffffffff810ce400>] ? update_curr+0x40/0x190
Mar 25 07:28:43 ssd-10 kernel: [12119028.387680] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387847] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.388016] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.388189] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388364] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388538] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388714] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388894] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389080] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389255] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389431] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389608] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389783] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389950] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.390129] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390306] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390481] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390646] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.390809] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:28:43 ssd-10 kernel: [12119028.390979] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:28:43 ssd-10 kernel: [12119028.391142] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.391304] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:28:43 ssd-10 kernel: [12119028.391468] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.391632] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:28:43 ssd-10 kernel: [12119028.391796] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.391966] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:28:43 ssd-10 kernel: [12119028.392135] INFO: task tabletserver:196951 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.392443] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.392754] tabletserver D 0000000000000000 0 196951 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.393075] ffff8808597fba00 0000000000000082 ffff88003ddbedd0 ffff8808597fbfd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.393396] ffff8808597fbfd8 ffff8808597fbfd8 ffff88003ddbedd0 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.393716] 7fffffffffffffff ffff88003ddbedd0 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.394044] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.394199] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.394362] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.394540] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.394718] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.394886] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.395068] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395236] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.395397] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.395568] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395742] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395917] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396102] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396282] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396460] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396637] [<ffffffffa07b0c0c>] xfs_iunlink+0x4c/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396812] [<ffffffffa07b0d76>] xfs_droplink+0x56/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396992] [<ffffffffa07b1a1f>] xfs_remove+0x21f/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.397169] [<ffffffffa07ac08a>] xfs_vn_unlink+0x5a/0xa0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.397334] [<ffffffff8120e9f1>] vfs_unlink+0xe1/0x150
Mar 25 07:28:43 ssd-10 kernel: [12119028.397497] [<ffffffff8120fcae>] do_unlinkat+0x26e/0x2b0
Mar 25 07:28:43 ssd-10 kernel: [12119028.397661] [<ffffffff81204071>] ? SYSC_newlstat+0x41/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.397825] [<ffffffff81210bf6>] SyS_unlink+0x16/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.397995] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
///////////
Mar 25 07:26:43 ssd-10 kernel: [12118908.620934] INFO: task kswapd1:227 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.621105] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.621417] kswapd1 D 0000000000000000 0 227 2 0x00000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.621733] ffff8810395938f0 0000000000000046 ffff881039c5de20 ffff881039593fd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.622061] ffff881039593fd8 ffff881039593fd8 ffff881039c5de20 ffff88010731b558
Mar 25 07:26:43 ssd-10 kernel: [12118908.622384] ffff88010731b560 7fffffffffffffff ffff881039c5de20 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.622705] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.622867] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.623038] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.623207] [<ffffffff812f0ce4>] ? blk_finish_plug+0x14/0x40
Mar 25 07:26:43 ssd-10 kernel: [12118908.623395] [<ffffffffa079ba84>] ? _xfs_buf_ioapply+0x334/0x460 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.627805] [<ffffffff8168cbd6>] wait_for_completion+0x116/0x170
Mar 25 07:26:43 ssd-10 kernel: [12118908.627979] [<ffffffff810c54e0>] ? wake_up_state+0x20/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.628156] [<ffffffffa079dbc4>] ? xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628332] [<ffffffffa079d7b6>] xfs_buf_submit_wait+0xe6/0x1d0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628509] [<ffffffffa079dbc4>] xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628684] [<ffffffffa07a611b>] xfs_reclaim_inode+0x34b/0x370 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.628862] [<ffffffffa07a63a7>] xfs_reclaim_inodes_ag+0x267/0x390 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629039] [<ffffffff810c54a3>] ? wake_up_process+0x23/0x40
Mar 25 07:26:43 ssd-10 kernel: [12118908.629217] [<ffffffffa07a7033>] xfs_reclaim_inodes_nr+0x33/0x40 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629397] [<ffffffffa07b6a05>] xfs_fs_free_cached_objects+0x15/0x20 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.629709] [<ffffffff812010d8>] prune_super+0xe8/0x170
Mar 25 07:26:43 ssd-10 kernel: [12118908.629877] [<ffffffff811947b3>] shrink_slab+0x163/0x330
Mar 25 07:26:43 ssd-10 kernel: [12118908.630047] [<ffffffff811f5b37>] ? vmpressure+0x87/0x90
Mar 25 07:26:43 ssd-10 kernel: [12118908.630212] [<ffffffff811985b1>] balance_pgdat+0x4b1/0x5e0
Mar 25 07:26:43 ssd-10 kernel: [12118908.630376] [<ffffffff81198853>] kswapd+0x173/0x450
Mar 25 07:26:43 ssd-10 kernel: [12118908.630541] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
Mar 25 07:26:43 ssd-10 kernel: [12118908.630706] [<ffffffff811986e0>] ? balance_pgdat+0x5e0/0x5e0
Mar 25 07:26:43 ssd-10 kernel: [12118908.630872] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
Mar 25 07:26:43 ssd-10 kernel: [12118908.631043] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:26:43 ssd-10 kernel: [12118908.631215] [<ffffffff81697758>] ret_from_fork+0x58/0x90
Mar 25 07:26:43 ssd-10 kernel: [12118908.631380] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:26:43 ssd-10 kernel: [12118908.631589] INFO: task tabletserver:196922 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.631901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.632213] tabletserver D 0000000000000000 0 196922 1 0x00000080
Mar 25 07:26:43 ssd-10 kernel: [12118908.632527] ffff8804307fb6e0 0000000000000082 ffff88001d84de20 ffff8804307fbfd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.632848] ffff8804307fbfd8 ffff8804307fbfd8 ffff88001d84de20 ffff88202d05f630
Mar 25 07:26:43 ssd-10 kernel: [12118908.633174] 7fffffffffffffff ffff88001d84de20 0000000000000002 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.633494] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.633648] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.633811] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.633998] [<ffffffffa07bba76>] ? xfs_trans_add_item+0x46/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634180] [<ffffffffa07cccf5>] ? _xfs_trans_bjoin+0x45/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634360] [<ffffffffa07cd227>] ? xfs_trans_read_buf_map+0x247/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.634670] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:26:43 ssd-10 kernel: [12118908.634846] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635033] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635201] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:26:43 ssd-10 kernel: [12118908.635363] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:26:43 ssd-10 kernel: [12118908.635535] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635710] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.635889] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636068] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636251] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636430] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636605] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636781] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.636966] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637142] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637308] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:26:43 ssd-10 kernel: [12118908.637482] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637658] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.637832] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.638003] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:26:43 ssd-10 kernel: [12118908.638168] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:26:43 ssd-10 kernel: [12118908.638330] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:26:43 ssd-10 kernel: [12118908.638493] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:26:43 ssd-10 kernel: [12118908.638656] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:26:43 ssd-10 kernel: [12118908.638820] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:26:43 ssd-10 kernel: [12118908.638993] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:26:43 ssd-10 kernel: [12118908.639158] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.639321] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:26:43 ssd-10 kernel: [12118908.639490] INFO: task tabletserver:196951 blocked for more than 120 seconds.
Mar 25 07:26:43 ssd-10 kernel: [12118908.639799] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:26:43 ssd-10 kernel: [12118908.640117] tabletserver D 0000000000000000 0 196951 1 0x00000080
Mar 25 07:26:43 ssd-10 kernel: [12118908.640431] ffff8808597fba00 0000000000000082 ffff88003ddbedd0 ffff8808597fbfd8
Mar 25 07:26:43 ssd-10 kernel: [12118908.640752] ffff8808597fbfd8 ffff8808597fbfd8 ffff88003ddbedd0 ffff88202d05f630
Mar 25 07:26:43 ssd-10 kernel: [12118908.641079] 7fffffffffffffff ffff88003ddbedd0 0000000000000002 0000000000000000
Mar 25 07:26:43 ssd-10 kernel: [12118908.641399] Call Trace:
Mar 25 07:26:43 ssd-10 kernel: [12118908.641553] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:26:43 ssd-10 kernel: [12118908.641716] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:26:43 ssd-10 kernel: [12118908.641903] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642083] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642250] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:26:43 ssd-10 kernel: [12118908.642425] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.642592] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:26:43 ssd-10 kernel: [12118908.642753] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:26:43 ssd-10 kernel: [12118908.642933] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643108] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643283] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643458] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643638] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.643817] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644002] [<ffffffffa07b0c0c>] xfs_iunlink+0x4c/0x160 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644179] [<ffffffffa07b0d76>] xfs_droplink+0x56/0x60 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644354] [<ffffffffa07b1a1f>] xfs_remove+0x21f/0x340 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644529] [<ffffffffa07ac08a>] xfs_vn_unlink+0x5a/0xa0 [xfs]
Mar 25 07:26:43 ssd-10 kernel: [12118908.644694] [<ffffffff8120e9f1>] vfs_unlink+0xe1/0x150
Mar 25 07:26:43 ssd-10 kernel: [12118908.644866] [<ffffffff8120fcae>] do_unlinkat+0x26e/0x2b0
Mar 25 07:26:43 ssd-10 kernel: [12118908.645030] [<ffffffff81204071>] ? SYSC_newlstat+0x41/0x60
Mar 25 07:26:43 ssd-10 kernel: [12118908.645194] [<ffffffff81210bf6>] SyS_unlink+0x16/0x20
Mar 25 07:26:43 ssd-10 kernel: [12118908.645357] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:27:01 ssd-10 systemd[1]: Started Session 501314 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Starting Session 501314 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Started Session 501315 of user root.
Mar 25 07:27:01 ssd-10 systemd[1]: Starting Session 501315 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Started Session 501317 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Starting Session 501317 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Started Session 501316 of user root.
Mar 25 07:28:01 ssd-10 systemd[1]: Starting Session 501316 of user root.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366046] INFO: task kswapd1:227 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366221] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.366532] kswapd1 D 0000000000000000 0 227 2 0x00000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.366847] ffff8810395938f0 0000000000000046 ffff881039c5de20 ffff881039593fd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.367178] ffff881039593fd8 ffff881039593fd8 ffff881039c5de20 ffff88010731b558
Mar 25 07:28:43 ssd-10 kernel: [12119028.367498] ffff88010731b560 7fffffffffffffff ffff881039c5de20 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.367819] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.367981] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.368153] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.368323] [<ffffffff812f0ce4>] ? blk_finish_plug+0x14/0x40
Mar 25 07:28:43 ssd-10 kernel: [12119028.368506] [<ffffffffa079ba84>] ? _xfs_buf_ioapply+0x334/0x460 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.368674] [<ffffffff8168cbd6>] wait_for_completion+0x116/0x170
Mar 25 07:28:43 ssd-10 kernel: [12119028.368843] [<ffffffff810c54e0>] ? wake_up_state+0x20/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.369019] [<ffffffffa079dbc4>] ? xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369200] [<ffffffffa079d7b6>] xfs_buf_submit_wait+0xe6/0x1d0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369377] [<ffffffffa079dbc4>] xfs_bwrite+0x24/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369553] [<ffffffffa07a611b>] xfs_reclaim_inode+0x34b/0x370 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369731] [<ffffffffa07a63a7>] xfs_reclaim_inodes_ag+0x267/0x390 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.369900] [<ffffffff810c54a3>] ? wake_up_process+0x23/0x40
Mar 25 07:28:43 ssd-10 kernel: [12119028.370084] [<ffffffffa07a7033>] xfs_reclaim_inodes_nr+0x33/0x40 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.370264] [<ffffffffa07b6a05>] xfs_fs_free_cached_objects+0x15/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.370575] [<ffffffff812010d8>] prune_super+0xe8/0x170
Mar 25 07:28:43 ssd-10 kernel: [12119028.370742] [<ffffffff811947b3>] shrink_slab+0x163/0x330
Mar 25 07:28:43 ssd-10 kernel: [12119028.370907] [<ffffffff811f5b37>] ? vmpressure+0x87/0x90
Mar 25 07:28:43 ssd-10 kernel: [12119028.371079] [<ffffffff811985b1>] balance_pgdat+0x4b1/0x5e0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371244] [<ffffffff81198853>] kswapd+0x173/0x450
Mar 25 07:28:43 ssd-10 kernel: [12119028.371410] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.371574] [<ffffffff811986e0>] ? balance_pgdat+0x5e0/0x5e0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371739] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
Mar 25 07:28:43 ssd-10 kernel: [12119028.371901] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:28:43 ssd-10 kernel: [12119028.372079] [<ffffffff81697758>] ret_from_fork+0x58/0x90
Mar 25 07:28:43 ssd-10 kernel: [12119028.372243] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Mar 25 07:28:43 ssd-10 kernel: [12119028.372452] INFO: task tabletserver:196922 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.372760] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.373075] tabletserver D 0000000000000000 0 196922 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.373391] ffff8804307fb6e0 0000000000000082 ffff88001d84de20 ffff8804307fbfd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.373712] ffff8804307fbfd8 ffff8804307fbfd8 ffff88001d84de20 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.374038] 7fffffffffffffff ffff88001d84de20 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.374360] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.374514] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.374676] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.374854] [<ffffffffa07bba76>] ? xfs_trans_add_item+0x46/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375045] [<ffffffffa07cccf5>] ? _xfs_trans_bjoin+0x45/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375226] [<ffffffffa07cd227>] ? xfs_trans_read_buf_map+0x247/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375536] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.375712] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.375890] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376061] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.376225] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.376396] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376571] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376746] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.376921] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377108] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377289] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377463] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377640] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377816] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.377992] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.378163] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.378337] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.378512] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.382937] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.383107] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.383270] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:28:43 ssd-10 kernel: [12119028.383433] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:28:43 ssd-10 kernel: [12119028.383596] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.383758] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:28:43 ssd-10 kernel: [12119028.383922] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.384095] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:28:43 ssd-10 kernel: [12119028.384261] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.384422] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:28:43 ssd-10 kernel: [12119028.384589] INFO: task tabletserver:196928 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.384897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.385211] tabletserver D 0000000000000000 0 196928 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.385526] ffff880f813df6e0 0000000000000082 ffff88005aebce70 ffff880f813dffd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.385847] ffff880f813dffd8 ffff880f813dffd8 ffff88005aebce70 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.386171] 7fffffffffffffff ffff88005aebce70 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.386492] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.386646] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.386808] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.386995] [<ffffffffa07cd61f>] ? xfs_trans_log_buf+0x8f/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387172] [<ffffffffa0774ccf>] ? xfs_btree_log_block+0x7f/0xf0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387340] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.387506] [<ffffffff810ce400>] ? update_curr+0x40/0x190
Mar 25 07:28:43 ssd-10 kernel: [12119028.387680] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.387847] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.388016] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.388189] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388364] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388538] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388714] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.388894] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389080] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389255] [<ffffffffa078d404>] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389431] [<ffffffffa078de58>] xfs_dialloc+0xe8/0x280 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389608] [<ffffffffa07af611>] xfs_ialloc+0x71/0x550 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389783] [<ffffffffa07afb63>] xfs_dir_ialloc+0x73/0x1f0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.389950] [<ffffffff8168b9a2>] ? down_write+0x12/0x30
Mar 25 07:28:43 ssd-10 kernel: [12119028.390129] [<ffffffffa07affab>] xfs_create+0x27b/0x6e0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390306] [<ffffffffa07ac5fb>] xfs_vn_mknod+0xbb/0x250 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390481] [<ffffffffa07ac7c3>] xfs_vn_create+0x13/0x20 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.390646] [<ffffffff8120b3fd>] vfs_create+0xcd/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.390809] [<ffffffff8120ddcf>] do_last+0xbef/0x12a0
Mar 25 07:28:43 ssd-10 kernel: [12119028.390979] [<ffffffff8120e542>] path_openat+0xc2/0x490
Mar 25 07:28:43 ssd-10 kernel: [12119028.391142] [<ffffffff8120f41d>] ? putname+0x3d/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.391304] [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Mar 25 07:28:43 ssd-10 kernel: [12119028.391468] [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Mar 25 07:28:43 ssd-10 kernel: [12119028.391632] [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Mar 25 07:28:43 ssd-10 kernel: [12119028.391796] [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.391966] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Mar 25 07:28:43 ssd-10 kernel: [12119028.392135] INFO: task tabletserver:196951 blocked for more than 120 seconds.
Mar 25 07:28:43 ssd-10 kernel: [12119028.392443] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 25 07:28:43 ssd-10 kernel: [12119028.392754] tabletserver D 0000000000000000 0 196951 1 0x00000080
Mar 25 07:28:43 ssd-10 kernel: [12119028.393075] ffff8808597fba00 0000000000000082 ffff88003ddbedd0 ffff8808597fbfd8
Mar 25 07:28:43 ssd-10 kernel: [12119028.393396] ffff8808597fbfd8 ffff8808597fbfd8 ffff88003ddbedd0 ffff88202d05f630
Mar 25 07:28:43 ssd-10 kernel: [12119028.393716] 7fffffffffffffff ffff88003ddbedd0 0000000000000002 0000000000000000
Mar 25 07:28:43 ssd-10 kernel: [12119028.394044] Call Trace:
Mar 25 07:28:43 ssd-10 kernel: [12119028.394199] [<ffffffff8168c7f9>] schedule+0x29/0x70
Mar 25 07:28:43 ssd-10 kernel: [12119028.394362] [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
Mar 25 07:28:43 ssd-10 kernel: [12119028.394540] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.394718] [<ffffffffa07bcb27>] ? kmem_zone_alloc+0x97/0x130 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.394886] [<ffffffff8168bac8>] __down_common+0x108/0x154
Mar 25 07:28:43 ssd-10 kernel: [12119028.395068] [<ffffffffa079cba6>] ? _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395236] [<ffffffff8168bb31>] __down+0x1d/0x1f
Mar 25 07:28:43 ssd-10 kernel: [12119028.395397] [<ffffffff810b6971>] down+0x41/0x50
Mar 25 07:28:43 ssd-10 kernel: [12119028.395568] [<ffffffffa079c99c>] xfs_buf_lock+0x3c/0xd0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395742] [<ffffffffa079cba6>] _xfs_buf_find+0x176/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.395917] [<ffffffffa079cd9a>] xfs_buf_get_map+0x2a/0x240 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396102] [<ffffffffa079d910>] xfs_buf_read_map+0x30/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396282] [<ffffffffa07cd1f1>] xfs_trans_read_buf_map+0x211/0x400 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396460] [<ffffffffa078d35d>] xfs_read_agi+0x9d/0x110 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396637] [<ffffffffa07b0c0c>] xfs_iunlink+0x4c/0x160 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396812] [<ffffffffa07b0d76>] xfs_droplink+0x56/0x60 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.396992] [<ffffffffa07b1a1f>] xfs_remove+0x21f/0x340 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.397169] [<ffffffffa07ac08a>] xfs_vn_unlink+0x5a/0xa0 [xfs]
Mar 25 07:28:43 ssd-10 kernel: [12119028.397334] [<ffffffff8120e9f1>] vfs_unlink+0xe1/0x150
Mar 25 07:28:43 ssd-10 kernel: [12119028.397497] [<ffffffff8120fcae>] do_unlinkat+0x26e/0x2b0
Mar 25 07:28:43 ssd-10 kernel: [12119028.397661] [<ffffffff81204071>] ? SYSC_newlstat+0x41/0x60
Mar 25 07:28:43 ssd-10 kernel: [12119028.397825] [<ffffffff81210bf6>] SyS_unlink+0x16/0x20
Mar 25 07:28:43 ssd-10 kernel: [12119028.397995] [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
--
FROM 36.112.24.*