outage: March 2009 Archives

it looks like I forgot to set dom0-min-mem to something reasonable in /etc/xen/xend-config.sxp. Yes, that was the problem.
sshd: page allocation failure. order:0, mode:0x20

Call Trace:
  [] __alloc_pages+0x299/0x2b2
 [] cache_alloc_refill+0x2b2/0x535
 [] __kmalloc+0xb0/0xf0
 [] pskb_expand_head+0x51/0x137
 [] :bridge:br_dev_queue_push_xmit+0x13e/0x1ad
 [] :bridge:br_nf_post_routing+0x17a/0x195
 [] nf_iterate+0x41/0x7d
 [] :bridge:br_dev_queue_push_xmit+0x0/0x1ad
 [] nf_hook_slow+0x58/0xbc
 [] :bridge:br_dev_queue_push_xmit+0x0/0x1ad
 [] :bridge:br_forward_finish+0x3f/0x51
 [] :bridge:br_nf_forward_finish+0xf7/0xff
 [] :bridge:br_nf_forward_ip+0x14d/0x15d
 [] nf_iterate+0x41/0x7d
 [] :bridge:br_forward_finish+0x0/0x51
 [] nf_hook_slow+0x58/0xbc
 [] :bridge:br_forward_finish+0x0/0x51
 [] :bridge:__br_forward+0x0/0x6d
 [] :bridge:__br_forward+0x59/0x6d
 [] netif_be_start_xmit+0x0/0x4a6
 [] :bridge:br_flood+0x7d/0xc6
 [] :bridge:br_handle_frame_finish+0xe3/0xf8
 [] :bridge:br_nf_pre_routing_finish+0x2ed/0x2fc
 [] :bridge:br_nf_pre_routing_finish+0x0/0x2fc
 [] nf_hook_slow+0x58/0xbc
 [] :bridge:br_nf_pre_routing_finish+0x0/0x2fc
 [] :bridge:br_nf_pre_routing+0x611/0x62f
 [] nf_iterate+0x41/0x7d
 [] :bridge:br_handle_frame_finish+0x0/0xf8
 [] nf_hook_slow+0x58/0xbc
 [] :bridge:br_handle_frame_finish+0x0/0xf8
 [] notify_remote_via_irq+0x2b/0x67
 [] :bridge:br_handle_frame+0x16e/0x1a2
 [] netif_receive_skb+0x1ca/0x2ea
 [] process_backlog+0xd0/0x182
 [] net_rx_action+0xe3/0x24b
 [] __do_softirq+0x83/0x117
 [] call_softirq+0x1c/0x28
 [] do_softirq+0x6a/0xed
 [] do_hypervisor_callback+0x1e/0x2c
  [] .text.lock.spinlock+0x0/0x8a
 [] key_lookup+0xf/0x68
 [] lookup_user_key+0x13c/0x1e4
 [] keyctl_revoke_key+0x14/0x37
 [] tracesys+0xab/0xb5

Mem-info:
DMA per-cpu:
cpu 0 hot: high 0, batch 1 used:0
cpu 0 cold: high 0, batch 1 used:0
cpu 1 hot: high 0, batch 1 used:0
cpu 1 cold: high 0, batch 1 used:0
cpu 2 hot: high 0, batch 1 used:0
cpu 2 cold: high 0, batch 1 used:0
cpu 3 hot: high 0, batch 1 used:0
cpu 3 cold: high 0, batch 1 used:0
cpu 4 hot: high 0, batch 1 used:0
cpu 4 cold: high 0, batch 1 used:0
cpu 5 hot: high 0, batch 1 used:0
cpu 5 cold: high 0, batch 1 used:0
cpu 6 hot: high 0, batch 1 used:0
cpu 6 cold: high 0, batch 1 used:0
cpu 7 hot: high 0, batch 1 used:0
cpu 7 cold: high 0, batch 1 used:0
DMA32 per-cpu:
cpu 0 hot: high 186, batch 31 used:30
cpu 0 cold: high 62, batch 15 used:11
cpu 1 hot: high 186, batch 31 used:178
cpu 1 cold: high 62, batch 15 used:24
cpu 2 hot: high 186, batch 31 used:168
cpu 2 cold: high 62, batch 15 used:14
cpu 3 hot: high 186, batch 31 used:160
cpu 3 cold: high 62, batch 15 used:12
cpu 4 hot: high 186, batch 31 used:97
cpu 4 cold: high 62, batch 15 used:5
cpu 5 hot: high 186, batch 31 used:146
cpu 5 cold: high 62, batch 15 used:0
cpu 6 hot: high 186, batch 31 used:83
cpu 6 cold: high 62, batch 15 used:5
cpu 7 hot: high 186, batch 31 used:123
cpu 7 cold: high 62, batch 15 used:2
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:        5424kB (0kB HighMem)
Active:36633 inactive:756 dirty:18 writeback:3 unstable:0 free:1356 slab:28807 mapped:918 pagetables:1660
DMA free:4012kB min:8kB low:8kB high:12kB active:164kB inactive:0kB present:2120kB pages_scanned:8415 all_unreclaimable? yes
lowmem_reserve[]: 0 1002 1002 1002
DMA32 free:1412kB min:4044kB low:5052kB high:6064kB active:146368kB inactive:3024kB present:1026160kB pages_scanned:21160 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 1*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4012kB
DMA32: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
264192 pages of RAM
22597 reserved pages
5161 pages shared
0 pages swap cached

I rebooted the box (using Xen's CTRL-A 3 times escape sequence) looks like the people on stables get a free month.

About this Archive

This page is a archive of entries in the outage category from March 2009.

outage: December 2008 is the previous archive.

outage: April 2009 is the next archive.

Find recent content on the main index or look in the archives to find all content.