Bug #732
msgr: leaking sockets when connection retrying?
| Status: | Resolved | Start date: | 01/21/2011 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | msgr | Spent time: | 0.25 hour | |
| Target version: | v0.24.2 | |||
| Source: | Severity: | |||
| Backport: | Reviewed: | |||
| Tags: |
Description
2011-01-20 11:14:03.283311 7f9b5ab2e710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987b8a60 sd=999 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:06.283581 7f9b5aa2d710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987bc230 sd=1000 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:09.283812 7f9b5a92c710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987bca60 sd=1001 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:12.284062 7f9b5a82b710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987c0260 sd=1002 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:15.284313 7f9b5a72a710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987c4050 sd=1003 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:18.284582 7f9b5a629710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987c0ca0 sd=1004 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:21.284855 7f9b5a528710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987c48c0 sd=1005 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:24.285112 7f9b5a427710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987c8050 sd=1006 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:27.285386 7f9b5a326710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987c8750 sd=1007 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:30.285655 7f9b5a225710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987cc260 sd=1008 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:33.285924 7f9b5a124710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987c8c80 sd=1009 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:36.286194 7f9b5a023710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987d0050 sd=1010 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:39.286426 7f9b59f22710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987d07b0 sd=1011 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:42.286716 7f9b59e21710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987d4050 sd=1012 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:45.287009 7f9b59d20710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987d47b0 sd=1013 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:48.287272 7f9b59c1f710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987d8050 sd=1014 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:51.287647 7f9b59b1e710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987d8790 sd=1015 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:54.287910 7f9b59a1d710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987dc050 sd=1016 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:14:57.288207 7f9b5991c710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987dc790 sd=1017 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:00.288465 7f9b5981b710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987e0050 sd=1018 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:03.288678 7f9b5971a710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987e0790 sd=1019 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:06.288938 7f9b59619710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987e4050 sd=1020 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:09.289203 7f9b59518710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6791/0 pipe(0x7f9b987e47b0 sd=1021 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:12.289477 7f9b59417710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987e8050 sd=1022 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:15.289717 7f9b59316710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6789/0 pipe(0x7f9b987e87b0 sd=1023 pgs=0 cs=0 l=0).fault first fault 2011-01-20 11:15:18.289962 7f9b59215710 -- 10.0.1.201:0/12255 >> 10.0.1.201:6790/0 pipe(0x7f9b987ec050 sd=-1 pgs=0 cs=0 l=0).connect couldn't created socket Too many open files msg/SimpleMessenger.cc: In function 'int SimpleMessenger::Pipe::connect()': msg/SimpleMessenger.cc:1003: FAILED assert(0) ceph version 0.24.1 (commit:5cf9570faa7d92871639b5bfe7307936e8870915) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x39) [0x6abcfe] 2: (SimpleMessenger::Pipe::connect()+0x21f) [0x5f2aad] 3: (SimpleMessenger::Pipe::writer()+0x16e) [0x5f6560] 4: (SimpleMessenger::Pipe::Writer::entry()+0x1c) [0x5a8ff0] 5: (Thread::_entry_func(void*)+0x23) [0x5fc237] 6: (()+0x68ba) [0x7f9ba732d8ba] 7: (clone()+0x6d) [0x7f9ba522d02d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Associated revisions
msgr: always start reaper
If we didn't explicitly bind (i.e. are a client), then we don't start
the accepter. That's fine. But the reaper thread start was also
conditional, when it shouldn't be; otherwise the client can't clean up
old Pipes (and their sockets).
Fixes: #732
Signed-off-by: Sage Weil <sage@newdream.net>
msgr: always start reaper
If we didn't explicitly bind (i.e. are a client), then we don't start
the accepter. That's fine. But the reaper thread start was also
conditional, when it shouldn't be; otherwise the client can't clean up
old Pipes (and their sockets).
Fixes: #732
Signed-off-by: Sage Weil <sage@newdream.net>
History
#1 Updated by Sage Weil over 2 years ago
- Status changed from New to Resolved