Activity
From 11/10/2013 to 12/09/2013
12/09/2013
- 05:59 PM Revision 702cc58d (calamari-clients): NWL feedback on widgets
- - move widget icon to top left
- move status icon to top right
- center context at bottom - 04:31 PM Feature #6958 (Resolved): BE: Set uniform data resolution and retention policy
- Set data retention for all inputs to 1 week,
Set resolution to all inputs to 60 seconds - 04:25 PM Feature #6886: BE: Cluster IOPS for 1.x
- Diamond bits are currently @here: https://github.com/jcsp/Diamond/tree/cluster-stats
- 04:23 PM Feature #6886 (In Progress): BE: Cluster IOPS for 1.x
- 04:12 PM Feature #6886: BE: Cluster IOPS for 1.x
- 04:17 PM Feature #6837: Setup: script to host local repo and install Calamari packages
- 04:09 PM Feature #6837 (Need More Info): Setup: script to host local repo and install Calamari packages
- 04:13 PM Feature #6887 (Resolved): BE: report total pg count
- 04:11 PM Feature #6818 (In Progress): UI: Dash: Create new Cluster IOPS status widget
- 04:11 PM Feature #6816 (In Progress): UI: Dash: Create new Pool status widget
- 04:11 PM Feature #6817 (In Progress): UI: Dash: Create new Host status widget
- 04:10 PM Feature #6822 (Resolved): BE: Dash: Return list of pools and quota sizes
- 04:08 PM Feature #6821 (Resolved): BE: Dash: List of hostnames running mon and OSD services
- 04:07 PM Fix #6640 (Resolved): UI: notification if the UI fails to load config.json
- 03:58 PM Feature #6823 (Resolved): UI: Graph: Use client side JS library to generate graphs
- 11:16 AM Feature #6823: UI: Graph: Use client side JS library to generate graphs
- We've run into a minor problem with the graphite backend returning inconsistent results sets, different step values f...
- 11:18 AM Feature #6563: UI: Graph: Add option to control time range
- See 6823 for details about a graphite bug that we need to work around concerning inconsistent step values being retur...
12/07/2013
- 06:07 AM Revision 23036aab (calamari): cephrestapi: daemonize and properly log uwsgi; clean up $PIDFILE
- - use uwsgi -d <logfile> to daemonize directly rather than
backgrounding with &
- actually send log messages to the... - 06:07 AM Revision a82801f0 (calamari): postinst: move edited diamond.conf interval to correct section, cha...
- graphite isn't accumulating data faster than 60s anyway right now; don't
collect data just to throw it away.
Fixes: ... - 12:54 AM Revision 625f50b2 (calamari-clients): Add a 50ms delay between json requests to graphite
12/06/2013
- 11:00 PM Revision 49181609 (calamari-clients): Clear legend too
- 10:57 PM Revision 32d3c863 (calamari-clients): Clear current graph when next one fails to load.
- 10:50 PM Revision 37e7f23e (calamari-clients): Bind to 'input' event instead of 'change'
- Change is ambigiously defined by the W3C for range inputs.
Input seems to work correctly for firefox and chrome.
Pre... - 10:35 PM Revision d11ef11b (calamari-clients): Fix slider reset on graph page change
- 10:25 PM Revision ac4592c2 (calamari): Merge pull request #28 from inktankstorage/wip-6887
- Calamari 1.x additions:
#6822: Return list of pools and quota sizes
#6821: List of hostnames running mon and OSD serv... - 10:18 PM Revision 12fdcc14 (calamari-clients): Graphs: Improve error reporting
- 10:05 PM Bug #6949 (Resolved): uwsgi not daemonizing nor logging properly, leaves PIDFILE
- as in subject. clean all this up.
- 10:03 PM Fix #6948 (Resolved): diamond.conf interval setting in wrong place, also 30s is wasted
- diamond.conf is edited by the postinstalls to add "interval = 30" to the end, but
# it needs to go in the collecto... - 09:53 PM Revision aa42ad9c (calamari-clients): Fix regression in URL cleaning
- 09:45 PM Revision c30cd392 (calamari-clients): Debounce the graph rendering, not the control.
- This was preventing updates on the UI when sliding the range around.
Now it updates the slider UI but waits 500ms bef... - 08:09 PM Revision 11b45960 (calamari-clients): Fix bad checkin
- 07:55 PM Revision 852fe190 (calamari-clients): Switch back to stack
- - Dan thinks it's useful.
- 07:51 PM Revision c2cfd22b (calamari-clients): Use line graphs instead of stack for cpu
- it's too confusing without all the metrics
- 07:38 PM Revision 34e0db2e (calamari-clients): Remove warning icon on each reload of graph
- 07:23 PM Revision c403946b (calamari-clients): Bug: pass timeout value through to pollers
- The timeout value was being set on the poller code but not being passed
to each of the individual pollers, so it used... - 07:11 PM Revision da491eaa (calamari-clients): Add a debounce handler for range slider
- 07:09 PM Revision 5e2c0101 (calamari-clients): Move veil code to cover the network request
- 05:50 PM Feature #6946 (Rejected): investigate firewall config for Diamond agents
- "anyone can send to 2003" is our current policy, and we probably ought to be a little tighter than that, particularly...
- 01:30 AM Revision 28de2e5d (calamari-clients): Update render from keys
- 01:26 AM Revision d240035e (calamari-clients): Update xlabel too
- 01:21 AM Revision 860cacc7 (calamari-clients): Simplest range code that would work
- 12:26 AM Revision e58cb95a (calamari-clients): Avoid text wrap in range control
- 12:12 AM Revision 54d4315a (calamari-clients): Add a graph range control to UI
12/05/2013
- 10:03 PM Revision edfda6b5 (calamari-clients): Add Labels to the X & Y Axis
- - placeholder for time range
- 07:54 PM Revision 385dadc0 (calamari-clients): Refactor: Extract Functions
- Break dygraphLoader into smaller functions
- add jsonRequest()
- add basic error handling back when a graph fails t... - 06:11 PM Revision 554a1f60 (calamari-clients): Separate processing json from rendering
- Split the processing and rendering into separate calls to the event
loop. This should prevent the browser from feelin... - 06:11 PM Revision b8e66bda (calamari-clients): DynaGraph Enhancements
- - re-use dynagraph instances to prevent memory leaks
- save new instances to jquery.data('graph') per graph-card ele... - 06:11 PM Revision a9c160f3 (calamari-clients): Reset Graphs carefully when re-using instances
- Reset stackedGraph and stepPlot
Reset fillGraph
Fill graph seems to be set by stacked graph
Resetting has some odd ... - 06:11 PM Revision f783128e (calamari-clients): Reset zoom on changing graph.
- 06:11 PM Revision 98e2103d (calamari-clients): Remove Avail metrics in stacked graphs
- - Remove 'Avail' metric from target list
- Remove 'Available' custom label
For disk summaries this is redundant info... - 02:45 PM Feature #6563 (In Progress): UI: Graph: Add option to control time range
- 02:44 PM Feature #6823 (In Progress): UI: Graph: Use client side JS library to generate graphs
- In bs3-exp branch
- 01:30 PM Revision 90383fc4 (calamari): Fix "calamari-ctl initialize" trying to create
- the same user twice when run twice (this
command should be idempotent) - 01:30 PM Revision 7f73a2a9 (calamari): Send supervisor stdout/stderr to files
- instead of PIPE, to avoid risk of deadlock
if the pipes fill up. - 12:39 AM Revision 72258a12 (calamari-clients): Remove unused widget
- 12:29 AM Revision e437c82d (calamari-clients): Convert Disk stats to stacked graphs
- 12:24 AM Revision 49b6a6bc (calamari-clients): Add stacked graph to memory plots
- 12:20 AM Revision 84e005b0 (calamari-clients): Switch to new json-array graphite format
- - create a graphites defaults and combine this with overrides
12/04/2013
- 08:16 PM Revision 6788251d (calamari-clients): Add missing X axis legend
- 08:11 PM Revision fa04f8d0 (calamari-clients): Add a custom legend
- - use custom labels for each graph generator
- style legend - 06:24 PM Revision 75e3b569 (calamari-clients): Add formatting for Kilo Mega Billions
- 06:17 PM Revision bb8d0698 (calamari-clients): Start adding options to Bytes based graphs
- 05:57 PM Revision 034cbf75 (calamari): Add 'make' to dependencies
- 04:04 PM Revision 75386720 (calamari): Remove the in_cluster attribute from ServiceStatus
- Fix a couple of typos
- 07:46 AM Revision e44ba48a (calamari-clients): Enhance graph generator code for dygraphs
- - Enhance per Host Graphs so they can accept more parameters
- add unique titles to each graph
- Make aggregate grap...
12/03/2013
- 09:31 PM Revision ab382c89 (calamari-clients): Increase network time outs to 10 seconds.
- 3 seconds is too short.
- 08:55 PM Revision 23e8ac40 (calamari): Merge pull request #29 from inktankstorage/wip-2.0-gregory
- Fix a salt sytax error in the carbon state
- 08:13 PM Revision a543aaf8 (calamari): Fix a salt sytax error in the carbon state
- 08:01 PM Revision cdffb6cb (calamari-clients): Remove display format configs from helper
- Move them to the graphwall where they are specified
- 04:38 PM Revision dddbf111 (calamari): Fix test_osd_out
- 04:38 PM Revision 2201dca6 (calamari): Fix silently failing to authenticate
- 04:28 PM Feature #6817: UI: Dash: Create new Host status widget
- Sorry I wasn't clear. My proposal is that the context shows which hosts have which role. i.e how many hosts are actin...
- 04:25 PM Feature #6817: UI: Dash: Create new Host status widget
- you say correct, but then negate the assertion in my question, so I'm unclear :)
So if there are 4 hosts, 3 runnin... - 04:20 PM Feature #6817: UI: Dash: Create new Host status widget
- Correct (hosts:4, 3 mon/4 osd). The goal is to show how many hosts are participating in the cluster and what processe...
- 04:15 PM Feature #6817: UI: Dash: Create new Host status widget
- Reviewing John's branch, I've had occasion to question the intent here: it seems like this is trying to express the "...
- 04:02 PM Feature #6886: BE: Cluster IOPS for 1.x
- So I interviewed Sage, and I think the actual situation is that:
1) OSD-centric io counters are available from the... - 03:32 PM Fix #6844 (Resolved): stop doing npm install/bower install in clean target
- cb3e2a3d416cb32ff27a86550e170321a3051907 in v1.0
- 03:30 PM Fix #6839 (Resolved): calamari-webapp must also install/enable mod_proxy_http
- cea81dfee11e2c401f0ef390735a83212ae51392 in v1.0
- 03:30 PM Fix #6840 (Resolved): limit graphite to using Django 1.5.5
- f3aa114a69341a76d17fa2def2c12cc68058f41c in v1.0
- 03:29 PM Fix #6841 (Resolved): make process uses global dir, limiting to one at a time and ugly. Fix that.
- 567c02e2005e2997053d216490fc6f52de42704f in v1.0; issue number not in commit msg.
- 03:28 PM Fix #6842 (Resolved): ceph/models.py: boilerplate error (update_attempt_time was really update_time)
- c35ce00dc1a9f4e32b744611ad86634ff87dd326 in v1.0. Issue not noted in commit msg.
- 03:27 PM Fix #6848 (Resolved): Time still jumps to six hours off randomly
- a95b3dd5d50822433d2133161a744ce6ea6f0ad9 in v1.0. Didn't put bug ID in commit msg. <sigh>
- 03:26 PM Fix #6848: Time still jumps to six hours off randomly
- 03:02 PM Revision c570c828 (calamari): Bind ZeroRPC before spawning thread, to catch error earlier
- if it happens.
- 02:10 PM Revision 6617ecaf (calamari): Fix a case where services weren't shut down if
- HTTP connections failed during CalmariCtl.start()
- 01:21 PM Revision 21b89241 (calamari): Fix some paths I broke
- 01:21 PM Revision 974db999 (calamari): Fix carbon installation in vagrant
- 01:21 PM Revision 5e131989 (calamari): s/calameri/calamari/
- 01:21 PM Revision f7eeb151 (calamari): msgpack doesn't like defaultdict, ensure the
- structures we pass around don't contain any.
- 07:26 AM Revision 9682a97d (calamari-clients): Dynamically resize the graphs based on container
- - dygraphs looks at the position style of the div to detect dynamic
resizing.
- use bootstraps markup to control the ... - 06:46 AM Revision 196d51f4 (calamari-clients): WIP - generate graph positions dynamically
- 01:18 AM Revision e3012ea9 (calamari-clients): medium 9 columns with height @ golden ratio
- 01:02 AM Revision d5ca149e (calamari-clients): Clean lint warnings
- 12:59 AM Revision 6a506dcc (calamari-clients): Change the graphwall layout
- 12:25 AM Revision 81463593 (calamari-clients): Wrong this reference
12/02/2013
- 11:41 PM Revision b89c7825 (calamari-clients): Test DyGraph Code
- 09:44 PM Revision 60723bee (calamari-clients): Add gauge Helper so card disappears
- 09:44 PM Revision 04e0becd (calamari-clients): Avoid crashing on App object
- 09:15 PM Revision 45119627 (calamari-clients): Placeholder IOPS widget
- 06:08 PM Revision 0be65a4f (calamari): DB+User setup from calamarictl
- Big piece of collateral damage here is moving the 'calamari'
django app out into a first class module so that we can
... - 06:06 PM Bug #6925 (Won't Fix): graphite's logging throws "[error] No handlers could be found for logger "...
- graphite's logging configuration must be slightly wrong.
- 04:07 PM Revision 0c525f1c (calamari): Vagrant+Salt for setting up a dev-mode calamari server instance
- 04:07 PM Revision 362a82fc (calamari): Update the annoying absolute paths to work in vagrant by default
- 04:07 PM Revision bbccafb6 (calamari): Update README with instructions on setting up
- a calamari 2.0 development environment.
- 04:07 PM Revision cc825195 (calamari): Opening the pandora's box of multiple vagrant files
- The old 'bootstrap.sh' one is in vagrant/production
A new one for running a dev-mode system on ubuntu
is in vagrant/d... - 12:39 PM Revision 9635f31d (calamari): Update requirements
- 11:59 AM Revision 40bc7dab (calamari): Fix loading /dashboard/ (was trying to read defunct DB table)
- 03:50 AM Feature #6917 (Resolved): Populate /info/ resource with real data
- Currently calamari.views.info is serving static (bogus) values.
11/29/2013
- 07:16 PM Revision cd8c9713 (calamari): FEATURE #6859: REST access to salt keys
- 05:17 PM Revision 7627f8b2 (calamari): Enable recovery, so that we don't have to wait
- for a cluster to say hello after restarting
calamari server. - 04:18 PM Revision 3fd391b9 (calamari): Persist sync_objects.
- Still to do:
* Validate DB at startup
* Recover ClusterMonitor state from DB at startup
* Limit number of records ...
11/28/2013
- 04:44 AM Revision a2a355de (calamari): Merge branch 'wip-make-cleanup' into v1.0
- 04:22 AM Revision 839e2b0c (calamari): Merge branch 'v1.0'
- 04:21 AM Revision b3c45f18 (calamari): deb httpd conf: don't set WSGISocketPrefix
- Default (apparently /var/run/apache2) is appropriate for
deb; for rh/centos, /etc/httpd/run/wsgi makes sense
Signed-... - 01:28 AM Revision 1a753c59 (calamari-clients): Add skeleton IOPS widget
- 01:27 AM Revision 28e0120e (calamari-clients): Add Dygraphs to bower
- 01:17 AM Revision 05c29a63 (calamari-clients): Add dygraphs to requirejs
- - add a simple define wrapper to export Dygraph to requirejs
11/27/2013
- 11:54 PM Revision 761c4e68 (calamari-clients): Issue #6815
- Add a warning state if any red pg states exist
- 11:33 PM Revision b6a371ce (calamari): Merge branch 'v1.0'
- 11:32 PM Revision dcb59b06 (calamari): More httpd.conf cleanup: remove head of rh version of calamari.conf
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 11:31 PM Revision 03e5e154 (calamari-clients): Issue #6819
- Add a warning icon at 90% usage
- 11:18 PM Revision 6f7a9aeb (calamari-clients): Issue #6814
- Show a warning if no mon's are in quorum
- 11:13 PM Revision 3128c07d (calamari-clients): Issue #6813
- 10:42 PM Revision 0eba304d (calamari-clients): Remove more dead code
- 10:29 PM Revision 7b202f6b (calamari-clients): Remove dead code
- 10:16 PM Revision 6a5c9b7d (calamari): Add clients submodule.
- 09:28 PM Revision cb3e2a3d (calamari): Makefile: make clean: more efficient in ui dirs
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 09:28 PM Revision 41d5828a (calamari): cleanup: move restapi conf files to conf/ with the rest of them
- also remove unused uwsgi.startup
Signed-off-by: Dan Mick <dan.mick@inktank.com> - 09:28 PM Revision 758266d0 (calamari): Move conf/upstart to conf/init for better conformance with other dirs
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 07:51 PM Revision 6dada3ae (calamari-clients): Enhance Gauge Helpers - add icon events
- Auto enhance gauge elements with event handers for:
status:{ok|warn|fail}
Triggering these on a Marionette ItemVie... - 07:05 PM Revision 9c6cebe2 (calamari-clients): Remove demo text
- - replace with a fa-refresh animation
- 07:01 PM Revision 82ad7ad0 (calamari-clients): Be consistent with PGMap use an &
- 06:41 PM Revision 3f59f7b9 (calamari-clients): Missing a state 'inconsistent'
- 06:32 PM Revision 40a4e346 (calamari-clients): Add a pgstat subtext to pgmap
- Show the most interesting stats in order of importance
- 05:54 PM Revision 0d84e6f5 (calamari-clients): Add a extra detail line to pgmap
- - add a place in the UI for extra details about PGs
- move the legend to the right of the canvas - 05:25 PM Revision ae5a1d78 (calamari): Couple of things I missed:
- * Handle socket.error from reverse DNS
* Get maybe-reverse-dns-derived server names for 'host' attribute
in fixup... - 05:01 PM Revision 03db1adc (calamari): Refactor pool/osd-specific logic into RequestFactory
- 04:01 PM Feature #6816: UI: Dash: Create new Pool status widget
- Base skeleton UI is done. Waiting for new APIs which should come from John's branch after merge. Faking using OSD dat...
- 04:00 PM Feature #6817: UI: Dash: Create new Host status widget
- UI is mostly there. Waiting on merge from John's branch. Right now I'm faking this using the OSD API.
- 03:59 PM Feature #6814 (Resolved): UI: Dash: Update MON status widget
- 03:57 PM Feature #6814: UI: Dash: Update MON status widget
- Commit 6f7a9aeb9abac1cdab1e75dda8137e51c4dc5c63
- 03:58 PM Feature #6813 (Resolved): UI: Dash: Update OSD status widget
- 03:58 PM Feature #6813: UI: Dash: Update OSD status widget
- Commit 3128c07dbc291e8fe49bf154b3e6c3c237bb7b03
- 03:10 PM Feature #6813: UI: Dash: Update OSD status widget
- Setting this value to 80% failed for now; my gut says this should probably be calculated based on which pools are in ...
- 03:56 PM Feature #6815: UI: Dash: Create new PG status extended widget with heatmap
- Commit 761c4e68d12f73c698fb1ed59ffbc1b3fc64488e
- 03:55 PM Feature #6815 (Resolved): UI: Dash: Create new PG status extended widget with heatmap
- 03:56 PM Feature #6819 (Resolved): UI: Dash: Update usage status widget
- Commit 03e5e1543b59d4980ff3f928085a284afc71b5d3
- 01:38 PM Revision cf5b5249 (calamari): Rename and quota operations for pools
- 12:52 PM Revision 5fdf9dad (calamari): Some basic OPTIONS feedback for RPCView
- 09:47 AM Feature #6907 (Resolved): Validation for POST & PATCH operations
In addition to types of attributes, should be checking (at least):
* Name is unique
* pgp_num is <= pg_num
I...- 08:47 AM Feature #6906 (New): Manage pool snapshots
C
osd pool mksnap <poolname> <snap> make snapshot <snap> in <pool>
R
'pool snaps' in OSD map
D
osd ...- 08:45 AM Feature #6864 (Resolved): Pool rename
- https://github.com/jcsp/calamari/commit/cf5b52492ebb50a9d2e8fb2ada78cdcbe098b4eb
- 12:56 AM Revision c91f722b (calamari): Merge branch 'v1.0'
- 12:53 AM Revision a95b3dd5 (calamari): Run mod_wsgi in daemon mode; bizarre bugs result otherwise
- The symptom: _convert_to_unix_ms would sometimes be 6 hours off; that
behavior correlated with the 't' datetime.datet... - 12:41 AM Revision 00f1f30a (calamari-clients): Fix pgmap rendering bug
- - Round X & Y coordinates down otherwise Firefox renders a different
bitmap from Chrome.
- Add a legend programmati...
11/26/2013
- 10:32 PM Revision f0a2ce09 (calamari-clients): Try a new icon position (nwl)
- This leaves us room for the tick in the bottom right and more text
space. - 05:21 PM Revision 7eb2be7c (calamari): FEATURE #6887: Report total PGs on each OSD
- 05:21 PM Revision d125cf13 (calamari): FEATURE #6821: List of hostnames running mon and OSD services
- Implement /cluster/<id>/server to return a list of
servers, where each server has a 'services' attribute
which is a l... - 05:21 PM Revision 80b3e6dd (calamari): Generalise CRUSH map hostname lookup:
- - OSDs now just have to be descendents of hosts, not
children
- If the 'osd' and 'host' crush types get renamed, ... - 05:20 PM Revision 57794bdc (calamari): FEATURE #6822: Return list of pools and quota sizes
- At /cluster/<cluster_id>/pool, return a list of:
{
"id": 1, # Django PK, ignore it.
"cluster": 1, # Django P... - 05:00 PM Revision 07b0ca54 (calamari): Use flake8 for pre-commit checks
- 04:51 PM Revision db738d43 (calamari): Flesh out pool handling:
- * Handle all the writeable attributes in creation and updates
* Expose CRUSH rules and rulesets in REST API so that ... - 09:25 AM Feature #6822: BE: Dash: Return list of pools and quota sizes
- The attribute names: the quota_max_[objects_bytes] are from the osd map, so I like to keep those the same, while the ...
- 06:09 AM Revision 3e6feb17 (calamari-clients): Tweak the layouts for mobile
- - switch to a 3 wide layout for small
- 05:02 AM Revision ec559581 (calamari-clients): Remove trailing space
- 04:58 AM Revision 929d9ecc (calamari-clients): Move text to right handside to give space for icon
- 12:22 AM Revision 298c5445 (calamari-clients): Add a pools widget using info in OSD
- We'll be getting a different API later, this is a standin
- 12:21 AM Revision ff77a9cf (calamari-clients): Remove formatting check, jsbeautify decides
11/25/2013
- 08:39 PM Revision 4dcb6d0d (calamari-clients): Hosts dashboard widget
- - using the osd map as the initial data source, missing mon info
- 06:41 PM Revision 716f3b27 (calamari-clients): Fix warnings
- 01:30 PM Feature #6822: BE: Dash: Return list of pools and quota sizes
- Created #6898 + targeted for v2.0 to make sure I don't forget we would like to provide the per-pool PG info.
- 01:08 PM Feature #6822: BE: Dash: Return list of pools and quota sizes
- Let's leave per-pool PG counts for now: I'd rather implement forward-looking stuff in the 2.0 codebase, as much of th...
- 12:00 PM Feature #6822: BE: Dash: Return list of pools and quota sizes
- If it's cheap to add, I would also love pg_states counts per pool. At some point, we'd like to add PG Map views for e...
- 11:51 AM Feature #6822: BE: Dash: Return list of pools and quota sizes
- Could we make the fields a little more uniform?
e.g.
quota_max_bytes maps to bytes_used
Could we have
quo... - 04:57 AM Feature #6822 (In Progress): BE: Dash: Return list of pools and quota sizes
- https://github.com/inktankstorage/calamari/tree/wip-6822
- 01:29 PM Feature #6898 (New): Per-pool PG details and state counters
(Re. Yan's comment on #6822)
This is two things:
- When getting the list of PGs, it should be possible to fil...- 01:05 PM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- The field you should be using is 'name' (which is equal to hostname if hostname is set, but may be something else if ...
- 11:54 AM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- I'm a little bit concerned with hostname not always being in the data structure, but other than that, I think this wi...
- 08:34 AM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- https://github.com/inktankstorage/calamari/tree/wip-6821
Borrowed the reverse dns path from Dan's wip-shortname br... - 05:03 AM Feature #6821 (In Progress): BE: Dash: List of hostnames running mon and OSD services
- 12:15 PM Revision feb11791 (calamari): Add /login/ route
- This allows use in 'runserver' mode, i.e. without apache.
- 12:10 PM Revision d5d4c63c (calamari): Fix absolute /opt/ paths so that running in
- development mode works.
- 11:57 AM Feature #6887: BE: report total pg count
- Ahem. Take 2.
- 11:56 AM Feature #6887: BE: report total pg count
- As discussed during standup, we're actually talking about this API entry point
http://mira035.front.sepia.ceph.com... - 08:45 AM Feature #6887 (In Progress): BE: report total pg count
- Added an 'all' section in pg counters, see attached.
- 08:05 AM Feature #6887: BE: report total pg count
- Oh never mind, I see that the calamari code is munging this before handing it back, so the total count is indeed bein...
- 08:04 AM Feature #6887: BE: report total pg count
- Hmm, anecdotally the state counts do seem to add up to the total PG count, I have 768 and here's the 'ceph status' sn...
- 10:10 AM Feature #6886: BE: Cluster IOPS for 1.x
As Dan pointed out in chat, Ceph has had these statistics for some time, although they are in the form of counters ...- 05:05 AM Bug #6875: Some statistics only have 5 minute resolution
- From chat:
dan.mick
00:34 cpu stats are indeed being sent every 300s from mira045; it never got the newer agent pac...
11/23/2013
- 05:11 PM Revision 844d4b1e (calamari): Passing tests for test_cluster_removal and test_two_clusters
- ... and a bunch of whitespace shrapnel.
- 03:57 PM Revision 95b0dfe0 (calamari): A pre-commit hook for PEP8+Pyflakes to keep me honest
- 02:13 AM Revision ea7c8fc5 (calamari-clients): Style PGMAP after nwl mock
- - Make widget larger (9 cols)
- Add active and clean metric
- WIP - 12:40 AM Revision 536b4620 (calamari-clients): Fix var. In strict mode
- 12:31 AM Revision 69bd59df (calamari-clients): Issue #6640 - provide an alert on err config.json
- - Don't let jQuery parse the JSON, do it explicitly in code so you can
catch the error and update the promise.
- No...
11/22/2013
- 04:33 PM Fix #6640: UI: notification if the UI fails to load config.json
- I've added basic error handling to the config loader. It will report an alert on any important error. It's pretty cle...
- 03:14 PM Feature #6887 (Resolved): BE: report total pg count
- Currently counts of pgs in various states are sent, but the total count of PGs is not. Since PGs can have multiple s...
- 03:08 PM Feature #6886 (Resolved): BE: Cluster IOPS for 1.x
Neil reckons Sage told him that we could get global (ie not per pool) IOPS stats out of existing dumpling releases....- 02:56 PM Feature #6815 (In Progress): UI: Dash: Create new PG status extended widget with heatmap
- 02:55 PM Feature #6819 (In Progress): UI: Dash: Update usage status widget
- 02:55 PM Feature #6814 (In Progress): UI: Dash: Update MON status widget
- 02:54 PM Feature #6813 (In Progress): UI: Dash: Update OSD status widget
- 02:51 PM Feature #6651: UI: Review Health Widget to remove unnecessary Warning states
- Aide memoire: the issue here is that if we remove the HEALTH_WARN etc state outright, we would fail to report some cl...
- 02:45 PM Feature #6651: UI: Review Health Widget to remove unnecessary Warning states
- 02:39 PM Feature #6651 (Rejected): UI: Review Health Widget to remove unnecessary Warning states
- We've redesigned to avoid the health widget
- 02:47 PM Feature #6653 (Resolved): UI: Dashboard v2 mockup
- 02:44 PM Revision 188796eb (calamari): Create a Sphinx doctree for development documentation
- 02:39 PM Fix #6572 (Resolved): UI: Migrate Twitter Bootstrap 2 -> 3
- 02:38 PM Feature #6012: BE: fixup monitor criticality classification based on email thread
- For the benefit of my memory: apparently this is about creating a middle-ground state for mons, between quorum and no...
- 02:32 PM Feature #6883 (Resolved): Graphite statistics names should be based on FSIDs, not hostnames
Namespacing statistics under hostnames is bad because:
- Ceph services can potentially be relocated between host...- 02:20 PM Feature #6881 (New): Diamond collector should use librados directly
- This is an efficiency thing.
Currently it wraps the ceph command line. This is inefficient, doing lots of process... - 02:15 PM Feature #6880 (Resolved): Job execution robustness and error reporting
Roughly this:...- 02:12 PM Feature #6879 (Resolved): OSD modification (in/out/up/down)
In out, in out, shake it all about.
- 12:15 PM Feature #6320: BE: create static/virtual config.json file to enable UI->backend
- Dan: I have a feeling this might already be fixed?
- 12:11 PM Feature #5562: BE: Reconcile mon info against Diamond/Graphite
- One liner, any clues on what this meant?
- 12:09 PM Feature #5541: BE: Multiple Clusters in Diamond
- Fixed this while I was doing something else
https://github.com/jcsp/Diamond/commit/a9791b0bd58747d3ac119c67d213514d3... - 12:01 PM Feature #5540 (Closed): BE: Update Architecture Diagram
- Superseded by 2.0 work.
- 11:52 AM Feature #5802: BE: use apache to serve authenticated static files
- I'm not sure we actually need to worry so much about authenticating access to static files, as there's no information...
- 11:49 AM Feature #6822: BE: Dash: Return list of pools and quota sizes
- For pool space usage, 1.x code is already getting the 'df' info in _populate_space, just not storing the per-pool stu...
- 11:40 AM Feature #6092: BE: robust OSD-to-HOST conversion (or different Diamond metric names)
- After giving this some thought I'm not sure there's that much we can do in 1.x to make hostname resolution truly robu...
- 11:37 AM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- For the initial widget, the goal is just to say "this many machines are in the cluster". We won't yet get the widget ...
- 11:22 AM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- So we discussed this yesterday, what I took away was:
- We actually want hostnames rather than IPs here
- Althoug... - 11:32 AM Feature #6876 (New): Efficient latest-value querying for graphite stats
Some values, such as space used in a pool, are by nature time series, but we're often interested in just getting th...- 11:17 AM Bug #6875 (Resolved): Some statistics only have 5 minute resolution
Querying some statistics (e.g. CPU) only gets 5 minute resolution. Graphite is giving you per-minute datapoints, b...- 10:58 AM Feature #6869 (Resolved): Generate and expose event log
When something noteworthy happens, synthesize an event:
- Health summary state changing
- OSD(s) going online/o...- 10:44 AM Feature #6868 (Rejected): Websockets notifications
This is a broad ticket, split it up before starting work.
Think about:
- Packaging of ZeroGW, packages exist ...- 10:40 AM Feature #6867 (New): Security for graphite connections
Two main things to consider:
1. How is the collector to know that it's sending data to an authorized receiver?
...- 10:30 AM Feature #6866 (New): Backend support for account recovery by email
Including:
- Configurable (via REST) SMTP server
- Ability to trigger a reset email+token by username or email
...- 08:40 AM Feature #6865 (Resolved): Update pgp_num when setting pg_num on a pool
- Little bit involved:
- Set pg_num
- Wait for PGs to be created
- Set pgp_num - 08:38 AM Feature #6864 (Resolved): Pool rename
Handle modifications to 'name' attribute of pool resource (equivalent of 'ceph osd pool rename')- 08:36 AM Feature #6863 (New): Expose historical cluster state in REST API
It's pretty unambiguously simple for the backend to store a log of cluster map versions. However, the view that th...- 08:28 AM Feature #6862 (New): Gather and store historical cluster state
Parts to this:
- When synchronising cluster maps, request the range of versions from the last seen to the latest,...- 08:10 AM Feature #6861 (Resolved): Expose list of servers in REST API
For each server, we should expose at least:
* FQDN (acting as the ID)
* Salt grains (this includes things like ...- 08:09 AM Feature #6860 (Resolved): Persist cluster state and recover on service restart
As well as holding cluster state in memory where it's accessible via ZeroRPC, write it to disk in the background. ...- 07:58 AM Feature #6859 (Resolved): Expose salt minion key management in REST API
At a minimum, expose the equivalent of:
salt-key -L
salt-key -a <minion id>
salt-key -d <minion id>
This is t...- 07:54 AM Feature #6857 (Resolved): Create CentOS & Wheezy minion bootstrap script
This is a script that the Calamari server makes available via HTTP, so that connecting the Ceph server to Calamari ...- 07:50 AM Feature #6856 (Resolved): Package 1.2 backend and dependencies (CentOS & Wheezy)
Aspects of this include:
- Installing graphite in an embedding-friendly way
- Installing the salt master module...- 07:44 AM Feature #6855 (Rejected): Package Calamari variant of Salt minion
I don't want to use upstream the upstream Salt minion as is, because if the end user happens to be using salt thems...- 06:54 AM Revision 817fd285 (calamari-clients): Make margins smaller
- 06:36 AM Revision 75f1021c (calamari-clients): Use round instead of floor
- 06:32 AM Revision e850339d (calamari-clients): Add basic Mon functionality
- 06:07 AM Revision 3d9bcd76 (calamari-clients): OSD Widget implementation
- - add event handlers for update
- calc percentage of down OSDs - 05:08 AM Revision 8e32c17b (calamari-clients): Add prefix to select dropdown for graphs
- 04:50 AM Revision 4b384b57 (calamari): Merge branch 'v1.0'
- 04:49 AM Revision 40a48313 (calamari): Makefile: remove extra graphite/ from graphite venv. Bug in venv r...
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 04:46 AM Revision 9b2323bf (calamari): smoketest.py: use paramiko, get hostname from setup, noninteractive...
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 01:42 AM Revision b14747db (calamari-clients): More styling
- - change the default behavior of gauge.js so it show's the inverse of
the usage value. Instead of 33% used we have ...
11/21/2013
- 11:47 PM Revision 91938229 (calamari-clients): New styling guidelines from nwl
- - basic Mon and OSD widget layouts
- extract card styling from _dashboard.scss - 10:32 PM Revision 933e4773 (calamari-clients): Stop redrawing text
- 10:26 PM Revision 4d59fece (calamari-clients): Refactor guage layout to be more generic
- - stop referrring to specific widgets
- use generic a,b,c,d notations for 4 slots
- move the layout classes into the ... - 09:57 PM Revision 92d58e39 (calamari-clients): Update bower dependencies
- Add new version of
- upgrade to marionette 1.2.3
- added bower dependencies to repo - 08:22 PM Fix #6848 (Resolved): Time still jumps to six hours off randomly
- Yan notices that the front-end sometimes reports times that are way way off (like six hours off). Dan's spent a *lot...
- 08:16 PM Fix #6847 (Resolved): calamari-webapp should list individual architectures, not 'all'
- -webapp is the package that has binaries, and so needs to vary across architectures.
- 08:14 PM Fix #6846 (Resolved): remove log files/dirs when uninstalling
- Clean up log files when removing packages
- 08:13 PM Fix #6845 (Resolved): move all upstart scripts to init.d scripts
- Upstart just isn't widely-enough enabled; punt and make everything init.d. This involves
creating some scripts, add... - 08:12 PM Fix #6844 (Resolved): stop doing npm install/bower install in clean target
- clean is way too slow; instead, just early-out if node_modules/ doesn't exist
- 08:11 PM Feature #6843 (Resolved): Create some testing automation (smoketest.py to provision vm and set up...
- Script is mostly working to set up a single-VM (of any supporte distro) cluster, install all three packages, and pop...
- 08:08 PM Fix #6842 (Resolved): ceph/models.py: boilerplate error (update_attempt_time was really update_time)
- Just wrong code in an update shortly before 1.0. Both times are the same in 1.0.
- 08:07 PM Fix #6841 (Resolved): make process uses global dir, limiting to one at a time and ugly. Fix that.
- Hack the installed virtualenvs so that they can be built in a private directory but
installable into /opt. Evil out... - 08:06 PM Fix #6840 (Resolved): limit graphite to using Django 1.5.5
- Deprecated features disappear in Django 1.6 that graphite still uses. Limit our venv to
Django 1.5.5 for now. - 08:05 PM Fix #6839 (Resolved): calamari-webapp must also install/enable mod_proxy_http
- Both mod_proxy and mod_proxy_http need to be enabled for ProxyPass
to work as we're using it. Discovered by Patrick... - 06:00 PM Revision 348126ce (calamari): Some useful settings for measuring coverage
- 05:39 PM Revision f75f6c99 (calamari): Wait for API to start accepting connections before starting
- 05:38 PM Revision 963b352e (calamari): Oops, don't forget this file!
- 05:29 PM Feature #6838 (Duplicate): Setup: Build tarballs containing all Calamari , Ceph-Deploy and Ceph D...
- The tarball should have the following structure:...
- 05:28 PM Feature #6837 (Resolved): Setup: script to host local repo and install Calamari packages
- A script (setup.sh) should:
* Copy files from a configured source (loopback mounted ISO or tarball) and move them ... - 04:10 PM Revision 8bccd8f3 (calamari): Housekeeping:
- * Update simulated ceph cluster to support pool operations
* Fix event handling to use tags to differentiate message... - 03:51 PM Feature #6270 (Resolved): Doc: Simple install doc
- Finished for release. (Why isn't v1.0 still in the Target version?)
- 03:51 PM Feature #6275 (Resolved): Build: Create Red Hat 6.4 RPMs for Calamari
- Finished weeks ago.
11/20/2013
- 10:11 PM Revision 94135d92 (calamari): Actions modifying the OSD map wait for it
- to be updated on the calamari server
before indicating completion.
Reinstate DELETE for clusters, to make
the calama... - 04:24 PM Feature #6816: UI: Dash: Create new Pool status widget
- 04:05 PM Feature #6816: UI: Dash: Create new Pool status widget
- Based on 1.1 mockups, display total number of pools.
Context element to show pool with least amount of space left in... - 03:53 PM Feature #6816 (Resolved): UI: Dash: Create new Pool status widget
- 04:24 PM Feature #6818: UI: Dash: Create new Cluster IOPS status widget
- 04:07 PM Feature #6818: UI: Dash: Create new Cluster IOPS status widget
- Based on 1.1 mockups, display total cluster IOPS figure.
Display stylized line graph for IOPS over previous 24 hour ... - 03:54 PM Feature #6818 (Resolved): UI: Dash: Create new Cluster IOPS status widget
- 04:24 PM Feature #6819: UI: Dash: Update usage status widget
- 04:08 PM Feature #6819: UI: Dash: Update usage status widget
- Based on 1.1 mockups, display used/total alongside guage.
Context element to show %age used.
Alert to be generated ... - 03:54 PM Feature #6819 (Resolved): UI: Dash: Update usage status widget
- 04:24 PM Feature #6557: UI: Dash: Create Activity Stream widget
- 04:18 PM Feature #6557: UI: Dash: Create Activity Stream widget
- Move the activity stream to the bottom of the screen.
Events for the stream include:
* Show timestamp for success... - 04:24 PM Feature #6815: UI: Dash: Create new PG status extended widget with heatmap
- 04:11 PM Feature #6815: UI: Dash: Create new PG status extended widget with heatmap
- Based on 1.1 mockups, headline to display actual number of active&clean PGs to total number of PGs
Context element t... - 03:53 PM Feature #6815 (Resolved): UI: Dash: Create new PG status extended widget with heatmap
- 04:24 PM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- 04:13 PM Feature #6821: BE: Dash: List of hostnames running mon and OSD services
- For the host widget on the dashboard, we need to show the total number of unique hosts in the cluster. We currently g...
- 03:57 PM Feature #6821 (Resolved): BE: Dash: List of hostnames running mon and OSD services
- 04:24 PM Feature #6822: BE: Dash: Return list of pools and quota sizes
- 04:15 PM Feature #6822: BE: Dash: Return list of pools and quota sizes
- For the dashboard, we want to add a new widget which shows the total number of pools available as the headline. For t...
- 03:58 PM Feature #6822 (Resolved): BE: Dash: Return list of pools and quota sizes
- 04:24 PM Feature #6563: UI: Graph: Add option to control time range
- 04:24 PM Feature #6823: UI: Graph: Use client side JS library to generate graphs
- 04:01 PM Feature #6823 (Resolved): UI: Graph: Use client side JS library to generate graphs
- 04:21 PM Feature #6304 (Duplicate): UI: Dash: Show aggregate IOPS across cluster
- Dup with 6818
- 04:20 PM Feature #6555 (Closed): UI: Dash: Break up status widget into 3 separate widgets
- 04:19 PM Feature #6048: UI: Dash: Show MON nodes in visualization
- Dup with 6814
- 04:18 PM Feature #6048 (Duplicate): UI: Dash: Show MON nodes in visualization
- Dup of #6814
- 04:19 PM Feature #6556 (Duplicate): UI: Dash: Show aggregate Cluster IOPS
- Dup with 6818
- 04:15 PM Feature #6181 (Rejected): UI: Dash: Move ? (help) icons to be next to the numbers to which they r...
- 04:05 PM Feature #6817: UI: Dash: Create new Host status widget
- Based on 1.1 mockups, headline display of total number of unique hosts.
Context element to show number of OSD and MO... - 04:02 PM Feature #6817: UI: Dash: Create new Host status widget
- 03:53 PM Feature #6817 (Resolved): UI: Dash: Create new Host status widget
- 04:04 PM Feature #6814: UI: Dash: Update MON status widget
- Based on 1.1 mockups, display up actual and total
Context element to show %age of MONs in quorum.
Alert to be gener... - 04:02 PM Feature #6814: UI: Dash: Update MON status widget
- 03:53 PM Feature #6814 (Resolved): UI: Dash: Update MON status widget
- 04:03 PM Feature #6813: UI: Dash: Update OSD status widget
- Based on 1.1 mockups, display in/up actual and total
Context element to show %age of OSDs in/up.
Alert to be genera... - 03:52 PM Feature #6813 (Resolved): UI: Dash: Update OSD status widget
- Based on new 1.1 template.
- 06:33 AM Revision 7e1f6cfa (calamari): Add smoketest.py to try to automate much of the package setup/testing.
- Partially tested; seems to almost set up a precise physical machine
(problems seem to be with the package repo).
Sig...
11/19/2013
- 05:29 PM Revision 28b8f089 (calamari): Stub out some more tests
- 04:51 PM Revision 11fdc9ce (calamari): Add GET for pools to REST API
- 04:32 PM Revision 94866833 (calamari): Getting data for the REST API from RAM:
- * Refactor cthulhu.manager stuff out so that
I can import parts without importing everything
* gevent-ize webapp/... - 05:53 AM Revision 503c588b (calamari-clients): Use a back canvas to render to and copy contents
- - Add an layer for text
- turn down alpha on clean pgs - 01:16 AM Revision 5f27f322 (calamari-clients): Hex WIP
- 01:11 AM Revision 263a51a7 (calamari-clients): WIP: drow some pixels onto canvas
- PoC. Still have to better understand KinectJS
11/18/2013
- 10:56 PM Revision 8efac369 (calamari-clients): Define layouts for different numbers of PGs
- Layouts for 15k, 30k, 60k and 100k PGs.
Includes scaling factors, and precalculated height & widths based on a
maximu... - 09:32 PM Revision a3d42f96 (calamari-clients): WIP: Integrate basic canvas support using kinetic
- 08:54 PM Revision c67c568b (calamari-clients): Add kinetic js
- 04:54 PM Revision 269117b1 (calamari): More tests
11/16/2013
- 02:57 AM Revision ce5f4958 (calamari-clients): Refactor app.js
- - extract Marionette.Application into it's own module
- 12:24 AM Revision f89d6fa6 (calamari-clients): Add initial Hex CSS
11/15/2013
- 04:45 PM Revision a68e229b (calamari): TestMonitoring.test_osd_out passing in simulation
- 07:56 AM Revision 650cf9e8 (calamari-clients): Try only importing the compass we use
- shaves about 1 second off
- 06:15 AM Revision d5c63476 (calamari-clients): Fix jshint warnings
- 02:05 AM Revision 2452c5f6 (calamari): Update calamari-client-examples.md with current behavior, bash script
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 01:21 AM Revision 5dfb0489 (calamari-clients): Add missing gauge class
- 01:19 AM Revision 2f67d6b7 (calamari-clients): Re-organize the SASS files after groking @import
11/14/2013
- 08:37 PM Revision c35ce00d (calamari): ceph/models.py: boilerplate error (update_attempt_time was really u...
- Signed-off-by: Dan Mick <dan.mick@inktank.com>
- 06:43 PM Revision 519bc1b5 (calamari): Sketching tests
- 03:09 PM Revision ae4617e8 (calamari): Hook minion sim into test
- 02:16 PM Revision 77809c0e (calamari): Bones of tests for calamari server
- 05:28 AM Revision 32854598 (calamari-clients): Update mon title
- 01:38 AM Revision 6ee18452 (calamari-clients): Change headlines to be functional
- 01:06 AM Revision c2f99639 (calamari-clients): Add a status line
- 12:11 AM Revision 6f992a68 (calamari-clients): Format big numbers using humanize
11/13/2013
- 11:53 PM Revision 8d67f36f (calamari-clients): Add a heartbeat space
- 03:23 PM Revision 6fa56f6d (calamari): Brain fart: the StatsSender is in the same process as CephCluster, no
- need to use XMLRPC between them.
- 02:55 PM Revision e1b2087f (calamari): Wire up cluster name in a couple of places
- 02:41 PM Revision d5f56a8b (calamari): Extend mock diamond stats to include CPU + network
- 01:01 PM Revision 2ba53df9 (calamari): Add some simulated graphite statistics
11/12/2013
- 09:55 PM Revision 108f90ef (calamari): Merge pull request #27 from inktankstorage/wip-venv
- Remove requirement for build to use global /opt
- 06:26 PM Revision ea9015f3 (calamari): Refactor minion_sim into modules
- 05:45 PM Revision 1783941a (calamari): minion-sim: Ceph simulator, salt-minion wrangler
- Creates a simulated ceph cluster, and exposes it
to many salt-minion instances who think they
are different hosts. - 03:09 PM Bug #6757 (Won't Fix): jenkins machines are on VPN, have full source; accessible to non-Inktankers
- The jenkins-calamari instance is authenticated, but the machine it runs on, and its build slaves, are normal VPN mach...
- 02:04 PM Revision b0f57bef (calamari): Get space info from graphite instead of maintaining
- a separate copy just for the REST API.
11/11/2013
- 05:54 PM Revision c8ae6fe4 (calamari): Salt configuration
- This is rough, includes a pip/virtualenv install
of diamond (with debian-specific code) and
some "John's House"-speci... - 05:51 PM Revision 62dfb7b5 (calamari): Add login_required decorator to graphite views
11/10/2013
- 12:38 AM Revision 2d9f0121 (calamari): Enable graphite.dashboard app
- Requires a version of graphite which has
the patch to respect settings.GRAPHITE_API_PREFIX
Also available in: Atom