-1E - RADOS Gateway refactor into library internal APIs¶
scuttlemonkey ok, I just refreshed the feed 10:23
scuttlemonkey it may reload once more 10:23
fghaas joshd: yes indeed 10:23
*** xiaoxi has quit IRC 10:24
joshd fghaas: lmb: I'm curious what async-replicated rbd would solve beyond sending rbd differentials between snapshots to another cluster 10:25
joshd fghaas: lmb: if you send diffs frequently they start looking pretty similar 10:26
fghaas joshd: well now that you mention it, shipping snapshot diffs might actually be better than streaming replication 10:28
scuttlemonkey both video feeds should be live now 10:28
fghaas because it would allow people to have a backup site trail x hours behind the live site on purpose 10:28
scuttlemonkey we'll get the sessions started up again in another 2 mins or so 10:28
fghaas joshd: is snapshot diff shipping coming for dumpling? 10:28
gregaf wido: if you want to talk, the Google+ app supports hangouts and should open automatically if you go to the page in the browser; if you're just watching you can open it on the youtube site 10:29
wido gregaf: I'm in already :) 10:29
joshd fghaas: I've got good news for you then, diff import/export are already there in cuttlefish 10:29
gregaf good good 10:30
fghaas joshd: I do know that, but is there a daemon that will actually ship offsite or do people still have to handcraft that? 10:30
rturk Hey all - #ceph-summit2 is for the second track 10:30
rturk head over there if you're in track 2 :) 10:30
joshd fghaas: shipping isn't handled by anything right now 10:31
rturk and if you want to participate in the discussion, send scuttlemonkey a ping 10:31
fghaas (joshd, in case you're wondering why lmb isn't responding, he stepped out for a bit, will probably throw his comments in when he returns) 10:31
*** nwat has joined #ceph-summit1 10:31
*** maclin has quit IRC 10:32
joshd fghaas: perhaps the rgw-geo agent could do similar things for rbd, I haven't thought much about it yet 10:32
*** nwat has quit IRC 10:32
fghaas joshd: ok. so what people would have to do is write a cron job that exports a diff, ships it, and then applies it on the other end? that's ok for testing, but people might dislike it for the real deal 10:32
paravoid that's not accurate 10:33
paravoid rgw isn't CGI, it's FastCGI 10:33
paravoid and FastCGI isn't an old, deprecated way of running apps 10:33
paravoid and fairly high performant, better than Python for sure 10:33
*** sjustlaptop has quit IRC 10:34
joshd fghaas: I agree the management of diffs could certainly be made easier, it sounds like you might have some suggestions. I haven't thought about it much yet 10:34
joshd fghaas: I just wanted the basic tools in place first for cuttlefish 10:35
*** Ryan_Lane has joined #ceph-summit1 10:36
fghaas sounds a bit like the old Postgres WAL shipping. Which isn't quite as elegant as streaming, low-level RADOS replication, but probably a far better option for Glance/Cinder than an alternative where Glance images are stored as Swift objects via rgw (no objection to rgw at all, but in this case I'd greatly prefer native RBD) 10:37
fghaas at any rate, joshd, our discussion is going OT for this channel, better switch this to #ceph :) 10:37
loicd there is a project to move the swift API out of the swift implementation. That was discussed during the openstack summit but I don't have a link handy. 10:42
*** cdl has joined #ceph-summit1 10:42
*** jtran has left #ceph-summit1 10:47
gregaf we already have so many logs for RGW, and coordinating consumers can be pretty annoying 10:47
scuttlemonkey need an aggregator log tool like juju's debug-log or something :P 10:48
gregaf sagewk was talking about keeping an event log and I don't think that's much less appealing to external services than a librgw that you can embed in your app to do all the real work ;) 10:49
loicd joshd: do you remember about it ? ( the swift library ). Although it's already implemented, it might save time on upgrading and chasing the changes. But I don't really know anything there ;-) 10:52
*** Ryan_Lane has quit IRC 10:53
joshd loicd: yeah, I talked to some swift folks about it, and first plan is pluggable fs backends for gluster/zfs on the backend servers, and later maybe pluggable higher-level backends (like ceph) beneath the proxy server later 10:54
paravoid a low-level librgw library that doesn't include the HTTP bits sounds useful in general 10:54
gregaf sagewk: not sure how behind this is your discussion, but the problem with get and put is all the streaming of those requests, right? 10:54
paravoid hooking the logic directly into applications 10:54
sagewk that's the devil in the details, yeah 10:54
paravoid while also having radosgw on top of that 10:54
scuttlemonkey (the youtube feed is about 1m behind the hangout) 10:54
paravoid for example, we could use librgw PHP bindings for mediawiki to access directly the Ceph cluster without a gateway 10:55
joshd loicd: it might be useful for keeping up with the apis, like new keystone etc, but probably a lot easier with librgw 10:55
loicd it looks like it could leverage librgw indeed :-) 10:56
joshd loicd: but the swift guys basically aren't going to do the work to make the proxy backends pluggable (and certainly not soon), so we'd need to do that as well if we wanted to use it 10:56
loicd ah 10:56
paravoid youtube is flaky dammit 10:58
scuttlemonkey yeah, a bit 10:58
loicd youtube 10:59
loicd darn 10:59
paravoid don't assume a MQ 11:01
paravoid maybe a fork/exec or a unix socket (for performance) 11:01
gregaf I'd feel better about a unix socket than a blind fork, yikes 11:04
paravoid could the georeplication logs be reused for this? 11:04
gregaf reused for which? 11:06
paravoid the swift API has object expiration already 11:07
paravoid http://docs.openstack.org/developer/swift/overview_expiring_objects.html 11:07
paravoid X-Delete-At & X-Delete-After 11:07
paravoid but this isn't LRU unfortunately 11:08
paravoid I wouldn't know about S3, sorry :) 11:08
*** Aaron|away has joined #ceph-summit1 11:10
gregaf wow, watching the etherpad edits in real-time and then hearing the conversation a minute later is very confusing 11:10
Aaron|away swift temp urls :) 11:10
*** Aaron|away is now known as AaronSchulz 11:10
* loicd lost connectivity, changing location 11:10
paravoid AaronSchulz: faster than you! 11:11
*** loicd has quit IRC 11:11
sagewk what i stoken server? 11:11
michael_dreamhost with s3 you pull acess tokens for mobile apps say 11:12
AaronSchulz paravoid: I was listing but not on IRC 11:12
paravoid gregaf: better than having the stream be interrupted every third minute or so 11:13
gregaf so we're talking about token servers in terms of something like Keystone, right? 11:13
paravoid at least I can keep up with the pad :) 11:13
joao I propose waiting for everyone on the hangout to wait until able to sync IRC with video :p 11:13