Project

General

Profile

-1F - Enforced bucket-level quotas in RGW » History » Version 1

Jessica Mack, 06/22/2015 05:46 AM

1 1 Jessica Mack
h1. -1F - Enforced bucket-level quotas in RGW
2 1 Jessica Mack
3 1 Jessica Mack
<pre>
4 1 Jessica Mack
*** mtk has left #ceph-summit1	11:16
5 1 Jessica Mack
scuttlemonkey	wido/gregaf: one of you guys want to take over sage's notetaking duties for this session?	11:16
6 1 Jessica Mack
wido	I have to go afk in a moment	11:16
7 1 Jessica Mack
scuttlemonkey	k	11:17
8 1 Jessica Mack
liwang	hi	11:17
9 1 Jessica Mack
wido	sorry	11:17
10 1 Jessica Mack
scuttlemonkey	li!	11:17
11 1 Jessica Mack
paravoid	wido: don't get me wrong on the varnish-specific stuff	11:17
12 1 Jessica Mack
scuttlemonkey	so great you could make it	11:17
13 1 Jessica Mack
paravoid	wido: I love Varnish and we heavily use it	11:17
14 1 Jessica Mack
wido	paravoid: No problem :)	11:17
15 1 Jessica Mack
paravoid	wido: I just don't think the two layers should be intermixed	11:17
16 1 Jessica Mack
liwang	yes	11:17
17 1 Jessica Mack
wido	No, they shouldn't indeed. I just wanted to point out the caching issues	11:17
18 1 Jessica Mack
liwang	does the invitation sent	11:18
19 1 Jessica Mack
wido	I want to have a "hook" to tell the cache to purge object X	11:18
20 1 Jessica Mack
scuttlemonkey	liwang: in approximately 42 minutes we'll have a break and I'll make sure you get added to the speakers	11:18
21 1 Jessica Mack
wido	And don't do the caching in RGW, since Varnish is much better in HTTP caching	11:18
22 1 Jessica Mack
paravoid	sure, that's not a bad idea	11:18
23 1 Jessica Mack
paravoid	you could invalidate with VCL too	11:18
24 1 Jessica Mack
paravoid	catch PUT/POST and call out to purge (what was previously set obj.ttl=0s)	11:19
25 1 Jessica Mack
liwang	i mean for the bucket-level quota session	11:19
26 1 Jessica Mack
paravoid	thinking Swift here, not sure about S3	11:19
27 1 Jessica Mack
wido	paravoid: True, but if you have multiple Varnish caches, you still get cache inconsistency	11:19
28 1 Jessica Mack
scuttlemonkey	liwang: you want to join as a speaker? Or just as an observer?	11:19
29 1 Jessica Mack
wido	It will only purge locally	11:19
30 1 Jessica Mack
paravoid	indeed	11:20
31 1 Jessica Mack
scuttlemonkey	youtube stream is at: http://youtu.be/rbVFozB9qls	11:20
32 1 Jessica Mack
wido	I'm thinking about having 10 Varnish caches	11:20
33 1 Jessica Mack
liwang	I registered as interested party for this session	11:20
34 1 Jessica Mack
paravoid	we have layered varnishes with consistent hashing among them	11:20
35 1 Jessica Mack
scuttlemonkey	ahhh!	11:20
36 1 Jessica Mack
paravoid	if you don't do that, then you cache the same URLs multiple times	11:20
37 1 Jessica Mack
wido	paravoid: True, you can do very cool stuff with Varnish that way	11:20
38 1 Jessica Mack
paravoid	it's better to shard your cache	11:20
39 1 Jessica Mack
wido	So you have multiple layers of Varnish servers?	11:21
40 1 Jessica Mack
scuttlemonkey	liwang: url sent via private message in irc	11:21
41 1 Jessica Mack
*** paravoid_ has joined #ceph-summit1	11:26
42 1 Jessica Mack
*** paravoid is now known as Guest4609	11:27
43 1 Jessica Mack
*** paravoid_ is now known as paravoid	11:27
44 1 Jessica Mack
paravoid	damn	11:27
45 1 Jessica Mack
paravoid	what did I miss?	11:27
46 1 Jessica Mack
paravoid	21:21 < wido> So you have multiple layers of Varnish servers?	11:27
47 1 Jessica Mack
paravoid	wido: yes	11:27
48 1 Jessica Mack
Guest4609	yes	11:27
49 1 Jessica Mack
*** Guest4609 has quit IRC	11:27
50 1 Jessica Mack
wido	paravoid: Ah, ok. I just tried to avoid that	11:28
51 1 Jessica Mack
wido	So you have them all as each others backend	11:28
52 1 Jessica Mack
wido	Would love to see that VCL :)	11:28
53 1 Jessica Mack
paravoid	not a VCL	11:28
54 1 Jessica Mack
wido	Inline C?	11:28
55 1 Jessica Mack
paravoid	it's a custom director, for consistent hashing	11:28
56 1 Jessica Mack
wido	Ah, that's cool	11:28
57 1 Jessica Mack
paravoid	https://gerrit.wikimedia.org/r/gitweb?p=operations/debs/varnish.git;a=commit;h=885b7d65fb7d93d0b3caa3fbf59a90fc2fffbaff	11:30
58 1 Jessica Mack
paravoid	anyway, this is getting offtopic	11:31
59 1 Jessica Mack
*** Elbandi_ has joined #ceph-summit1	11:32
60 1 Jessica Mack
sergey	I think Google App Engine can do a "503 Over Quota"	11:36
61 1 Jessica Mack
*** ksp has left #ceph-summit1	11:36
62 1 Jessica Mack
wido	paravoid: Aha! Indeed offtopic, but I met one of the Wikimedia people at a VUG in Amsterdam. Got inspired :)	11:39
63 1 Jessica Mack
*** Ryan_Lane has joined #ceph-summit1	11:39
64 1 Jessica Mack
paravoid	that was Mark, the commiter of the above	11:39
65 1 Jessica Mack
mikedawson	sagewk: what is considered a large size Teuthology ceph cluster? What is a long-running test?	11:40
66 1 Jessica Mack
sagewk	mikedawson: other channel :)	11:40
67 1 Jessica Mack
scuttlemonkey	^^ (#ceph-summit2)	11:41
68 1 Jessica Mack
mikedawson	what is considered a large size Teuthology ceph cluster? What is a long-running test?	11:41
69 1 Jessica Mack
mikedawson	sorry	11:41
70 1 Jessica Mack
*** ccourtaut has quit IRC	11:49
71 1 Jessica Mack
scuttlemonkey	new Track 1 video is live on the event page and can also be accessed here: http://youtu.be/OUxEk87RcnI	12:03
72 1 Jessica Mack
</pre>