[d-kernel] Повис сервер

[d-kernel] Повис сервер

[Grigory Batalov]

[-- Attachment #1: Type: text/plain, Size: 4924 bytes --]

Здравствуйте!

Недавно у нас повис шлюз и находящиеся на нём OVZ-контейнеры.
Он пинговался, и сквозь него можно было зайти по ssh на одну машину
в локальной сети. А в контейнеры и на сам шлюз не получалось.

Соответствующий кусок лога:
...
Jan 12 13:14:49 gate sshd[16590]: Unknown username from 203.130.198.102
Jan 12 13:14:52 gate sshd[16618]: Unknown username from 203.130.198.102
Jan 12 13:14:55 gate kernel: Unable to handle kernel NULL pointer dereference at 0000000000000010 RIP: 
Jan 12 13:14:55 gate kernel:  [__rb_rotate_left+7/91] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel:  [<ffffffff802249a6>] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel: PGD c3660067 PUD a4748067 PMD 0 
Jan 12 13:14:55 gate kernel: Oops: 0000 [340] SMP 
Jan 12 13:14:55 gate kernel: CPU: 0 
Jan 12 13:14:55 gate kernel: Modules linked in: ipt_ULOG ip_gre usblp simfs vznetdev vzethdev vzrst vzcpt vzdquota vzmon vzdev xt_tcpudp ipt_ttl ipt_TCPMSS iptable_mangle ipt_tos ipt_REJECT af_packet ac thermal processor button ide_cd cdrom parport_pc parport floppy psmouse pcspkr sunrpc iptable_nat ip_nat iptable_filter ip_tables x_tables usbhid tg3 ohci_hcd usbcore i2c_amd756 k8temp i2c_amd8111 i2c_core hwmon amd_rng sg dm_mod ip_conntrack_pptp ip_conntrack nfnetlink rtc ext3 jbd mbcache arcmsr sata_sil libata sd_mod scsi_mod ide_disk ide_generic amd74xx generic ide_core
Jan 12 13:14:55 gate kernel: Pid: 16667, comm: sshd Not tainted 2.6.18-ovz-smp-alt11 #1
Jan 12 13:14:55 gate kernel: RIP: 0060:[__rb_rotate_left+7/91]  [__rb_rotate_left+7/91] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel: RIP: 0060:[<ffffffff802249a6>]  [<ffffffff802249a6>] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel: RSP: 0068:ffff81009e111df0  EFLAGS: 00010286
Jan 12 13:14:55 gate kernel: RAX: 0000000000000000 RBX: ffff8100cfca7188 RCX: 0000000000000000
Jan 12 13:14:55 gate kernel: RDX: 0000000000000000 RSI: ffffffff8055fa00 RDI: ffff81004d5cb088
Jan 12 13:14:55 gate kernel: RBP: ffff81004d5cb088 R08: ffff81006c9e3088 R09: ffff81009e111c08
Jan 12 13:14:55 gate kernel: R10: 000000001150af17 R11: 00000000c43c0f84 R12: ffff8100cfca7188
Jan 12 13:14:55 gate kernel: R13: ffff81002df1a688 R14: ffffffff8055fa00 R15: 0000000000000021
Jan 12 13:14:55 gate kernel: FS:  00002aaaac3edb00(0000) GS:ffffffff80587000(0000) knlGS:00000000b7ebc6c0
Jan 12 13:14:55 gate kernel: CS:  0060 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 12 13:14:55 gate kernel: CR2: 0000000000000010 CR3: 00000000a4321000 CR4: 00000000000006e0
Jan 12 13:14:55 gate kernel: Process sshd (pid: 16667, veid=107, threadinfo ffff81009e110000, task ffff8100f3040c10)
Jan 12 13:14:55 gate kernel: Stack:  ffffffff80211282 ffff8100cfca7198 0000000000000009 ffff81002df1a680
Jan 12 13:14:55 gate kernel:  ffff81002df1a680 ffff81009e111ea8 ffffffff803031a3 0000000000000000
Jan 12 13:14:55 gate kernel:  1f3f000000000000 00000065ffffffff ffffffff804bbe60 0000000000000000
Jan 12 13:14:55 gate kernel: Call Trace:
Jan 12 13:14:55 gate kernel:  [rb_insert_color+178/218] rb_insert_color+0xb2/0xda
Jan 12 13:14:55 gate kernel:  [<ffffffff80211282>] rb_insert_color+0xb2/0xda
Jan 12 13:14:55 gate kernel:  [key_alloc+626/750] key_alloc+0x272/0x2ee
Jan 12 13:14:55 gate kernel:  [<ffffffff803031a3>] key_alloc+0x272/0x2ee
Jan 12 13:14:55 gate kernel:  [keyring_alloc+41/94] keyring_alloc+0x29/0x5e
Jan 12 13:14:55 gate kernel:  [<ffffffff80304192>] keyring_alloc+0x29/0x5e
Jan 12 13:14:55 gate kernel:  [alloc_uid_keyring+120/166] alloc_uid_keyring+0x78/0xa6
Jan 12 13:14:55 gate kernel:  [<ffffffff8030584c>] alloc_uid_keyring+0x78/0xa6
Jan 12 13:14:55 gate kernel:  [alloc_uid+274/429] alloc_uid+0x112/0x1ad
Jan 12 13:14:55 gate kernel:  [<ffffffff8028bd9b>] alloc_uid+0x112/0x1ad
Jan 12 13:14:55 gate kernel:  [set_user+15/146] set_user+0xf/0x92
Jan 12 13:14:55 gate kernel:  [<ffffffff8028ea22>] set_user+0xf/0x92
Jan 12 13:14:55 gate kernel:  [sys_setresuid+321/544] sys_setresuid+0x141/0x220
Jan 12 13:14:55 gate kernel:  [<ffffffff80290cb3>] sys_setresuid+0x141/0x220
Jan 12 13:14:55 gate kernel:  [system_call+126/131] system_call+0x7e/0x83
Jan 12 13:14:55 gate kernel:  [<ffffffff8025b23a>] system_call+0x7e/0x83
Jan 12 13:14:55 gate kernel: DWARF2 unwinder stuck at system_call+0x7e/0x83
Jan 12 13:14:55 gate kernel: Leftover inexact backtrace:
Jan 12 13:14:55 gate kernel: 
Jan 12 13:14:55 gate kernel: 
Jan 12 13:14:55 gate kernel: Code: 48 8b 51 10 49 83 e0 fc 48 85 d2 48 89 57 08 74 0c 48 8b 02 
Jan 12 13:14:55 gate kernel: RIP  [__rb_rotate_left+7/91] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel: RIP  [<ffffffff802249a6>] __rb_rotate_left+0x7/0x5b
Jan 12 13:14:55 gate kernel:  RSP <ffff81009e111df0>
Jan 12 13:14:55 gate kernel: CR2: 0000000000000010
Jan 14 15:24:02 gate syslogd 1.4.1: restart.
...

Ядро 2.6.18-ovz-smp-alt11 x86_64

-- 
 Grigory Batalov,
 ALT Linux Team

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [d-kernel] Повис сервер

[Sergey Vlasov]

[-- Attachment #1: Type: text/plain, Size: 1040 bytes --]

On Fri, Jan 18, 2008 at 10:28:50PM +0300, Grigory Batalov wrote:
> Jan 12 13:14:55 gate kernel: RIP: 0060:[<ffffffff802249a6>]  [<ffffffff802249a6>] __rb_rotate_left+0x7/0x5b

> Jan 12 13:14:55 gate kernel: Call Trace:
> Jan 12 13:14:55 gate kernel:  [<ffffffff80211282>] rb_insert_color+0xb2/0xda
> Jan 12 13:14:55 gate kernel:  [<ffffffff803031a3>] key_alloc+0x272/0x2ee
> Jan 12 13:14:55 gate kernel:  [<ffffffff80304192>] keyring_alloc+0x29/0x5e
> Jan 12 13:14:55 gate kernel:  [<ffffffff8030584c>] alloc_uid_keyring+0x78/0xa6
> Jan 12 13:14:55 gate kernel:  [<ffffffff8028bd9b>] alloc_uid+0x112/0x1ad
> Jan 12 13:14:55 gate kernel:  [<ffffffff8028ea22>] set_user+0xf/0x92
> Jan 12 13:14:55 gate kernel:  [<ffffffff80290cb3>] sys_setresuid+0x141/0x220
> Jan 12 13:14:55 gate kernel:  [<ffffffff8025b23a>] system_call+0x7e/0x83

> Ядро 2.6.18-ovz-smp-alt11 x86_64

http://bugzilla.kernel.org/show_bug.cgi?id=7727, оно же CVE-2007-0006;
исправлено в kernel-image-ovz-smp-2.6.18-alt13 и
kernel-image-std-smp-2.6.18-alt5.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [d-kernel] Повис сервер

[Grigory Batalov]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

On Fri, 18 Jan 2008 23:03:41 +0300, Sergey Vlasov wrote:

> > Jan 12 13:14:55 gate kernel: RIP: 0060:[<ffffffff802249a6>]  [<ffffffff802249a6>] __rb_rotate_left+0x7/0x5b
> 
> > Jan 12 13:14:55 gate kernel: Call Trace:
> > Jan 12 13:14:55 gate kernel:  [<ffffffff80211282>] rb_insert_color+0xb2/0xda
> > Jan 12 13:14:55 gate kernel:  [<ffffffff803031a3>] key_alloc+0x272/0x2ee
> > Jan 12 13:14:55 gate kernel:  [<ffffffff80304192>] keyring_alloc+0x29/0x5e
> > Jan 12 13:14:55 gate kernel:  [<ffffffff8030584c>] alloc_uid_keyring+0x78/0xa6
> > Jan 12 13:14:55 gate kernel:  [<ffffffff8028bd9b>] alloc_uid+0x112/0x1ad
> > Jan 12 13:14:55 gate kernel:  [<ffffffff8028ea22>] set_user+0xf/0x92
> > Jan 12 13:14:55 gate kernel:  [<ffffffff80290cb3>] sys_setresuid+0x141/0x220
> > Jan 12 13:14:55 gate kernel:  [<ffffffff8025b23a>] system_call+0x7e/0x83
> 
> > Ядро 2.6.18-ovz-smp-alt11 x86_64
> 
> http://bugzilla.kernel.org/show_bug.cgi?id=7727, оно же CVE-2007-0006;
> исправлено в kernel-image-ovz-smp-2.6.18-alt13 и
> kernel-image-std-smp-2.6.18-alt5.

  Спасибо, обновлюсь. Надо было самому следить за ядрами =).

-- 
 Grigory Batalov,
 ALT Linux Team

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]