Actions
BlueStore SMR Support GSOC 2016 Progress Report¶
This page contains documentation of the Progress made for GSOC 2016. For main page, redirect here
TimeLine¶
| Day | Event |
|---|---|
| |
|
| |
|
| 23 May | Students begin coding for their Google Summer of Code projects; Google begins issuing initial student payments provided tax forms are on file and students are in good standing with their communities. |
| |
|
| 20 June 19:00 UTC | |
| 27 June 19:00 UTC | |
| Work Period | Mentors give students a helping hand and guidance on their projects. |
| 15 August - 23 August 19:00 UTC | Final week: Students tidy code, write tests, improve documentation and submit their code sample. Students also submit their final mentor evaluation. |
| 23 August - 29 August 19:00 UTC | Mentors submit final student evaluations. |
| 30 August | Final results of Google Summer of Code 2016 announced |
| Late October | Mentor Summit at Google. Mentors and Organization Administrators from each participating organization are invited to Google for an unconference to collaborate on ideas to make the program better and to make new friends too! |
Weekly Progress Summary¶
| Week | Date | Documentation | Development | Testing | AOB |
|---|---|---|---|---|---|
| Week1 | Apr 25 - May 1 | Created Wiki | Setting Up Ceph Development Environment | Install procedure here | |
| [TODO] Create Architecture diagram | |||||
| [TODO] Document Changes reuired in code | Share with Mentor | ||||
| Week2-Week3 | Apr 1 - May 15 | Exploring SMR Drives | Setting Up Ceph Development Environment |
SMR_Drives¶
The SMR drives that we obtained are Seagate Archive SMR Drives 8TB in size (7.23 GB usable space). It was earlier proposed that these SMR drives are Drive Managed, but now it seems they are host aware SMR drives.
I used libzbc and ran tests on the SMR dirves to see if they work. Since Linux 3.19, each block device in Linux is also indentified as /dev/sg<X>.
I have 2 HDDs in my machine. The first HDD has OS installed. The second HDD is the Host Aware SMR drive.
cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: SAMSUNG SP2504C Rev: 0-52
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST8000AS0022-1WL Rev: SN01
Type: Direct-Access ANSI SCSI revision: 05
The first device is identified as /dev/sg0, second device as /dev/sg1 and so on.
when we run libzbc tests, we get the following results:
sudo ./zbc_test.sh /dev/sg1
Executing command completion tests...
010: REPORT_ZONES command completion... [Passed]
011: REPORT_ZONES (partial bit) command completion... [Passed]
012: REPORT_ZONES (reporting option 0x10) command completion... [Passed]
013: REPORT_ZONES (reporting option 0x11) command completion... [Passed]
014: REPORT_ZONES (reporting option 0x3F) command completion... [Passed]
020: OPEN_ZONE command completion... [Passed]
030: CLOSE_ZONE command completion... [Passed]
040: FINISH_ZONE command completion... [Passed]
050: RESET_WRITE_PTR command completion... [Passed]
060: WRITE command completion... [Passed]
070: READ command completion... [Passed]
Executing sense key, sense code tests...
010: REPORT_ZONES logical block out of range... [Failed]
=> Expected Illegal-request / Logical-block-address-out-of-range
Got /
011: REPORT_ZONES invalid reporting option... [Failed]
=> Expected Illegal-request / Invalid-field-in-cdb
Got Illegal-request / Unaligned-write-command
020: OPEN_ZONE invalid zone start lba... [Passed]
021: OPEN_ZONE insufficient zone resources... [ N/A ]
022: OPEN_ZONE insufficient zone resources (ALL bit set)... [ N/A ]
023: OPEN_ZONE conventional zone... [Passed]
030: CLOSE_ZONE invalid zone start lba... [Passed]
031: CLOSE_ZONE conventional zone... [Passed]
040: FINISH_ZONE invalid zone start lba... [Passed]
041: FINISH_ZONE conventional zone... [Passed]
050: RESET_WRITE_PTR invalid zone start lba... [Passed]
051: RESET_WRITE_PTR conventional zone... [Passed]
060: READ access sequential zone LBAs after write pointer... [Passed]
061: READ sequential zones boundary violation... [Passed]
062: READ conventional/sequential zones boundary violation... [Passed]
070: WRITE unaligned write in sequential zone... [Passed]
071: WRITE sequential zone boundary violation... [Passed]
072: WRITE insufficient zone resources... [Passed]
Executing zone state machine tests...
001: OPEN_ZONE empty to explicit open... [Passed]
002: CLOSE_ZONE empty to empty... [Passed]
003: FINISH_ZONE empty to full... [Passed]
004: RESET_WRITE_PTR empty to empty... [Passed]
005: OPEN_ZONE implicit open to explicit open... [Passed]
006: CLOSE_ZONE implicit open to closed... [Passed]
007: FINISH_ZONE implicit open to full... [Passed]
008: RESET_WRITE_PTR implicit open to empty... [Passed]
009: OPEN_ZONE empty to explicit open to explicit open... [Passed]
010: CLOSE_ZONE empty to explicit open to empty... [Passed]
011: FINISH_ZONE empty to explicit open to full... [Passed]
012: RESET_WRITE_PTR empty to explicit open to empty... [Passed]
013: OPEN_ZONE implicit open to explicit open to explicit open... [Passed]
014: CLOSE_ZONE implicit open to explicit open to closed... [Passed]
015: FINISH_ZONE implicit open to explicit open to full... [Passed]
016: RESET_WRITE_PTR implicit open to explicit open to empty... [Passed]
017: OPEN_ZONE closed to explicit open... [Passed]
018: CLOSE_ZONE closed to closed... [Passed]
019: FINISH_ZONE closed to full... [Passed]
020: RESET_WRITE_PTR closed to empty... [Passed]
021: OPEN_ZONE full to full... [Passed]
022: CLOSE_ZONE full to full... [Passed]
023: FINISH_ZONE full to full... [Passed]
024: FINISH_ZONE full to empty... [Passed]
025: OPEN_ZONE empty to empty (ALL bit set)... [Passed]
026: CLOSE_ZONE empty to empty (ALL bit set)... [Passed]
027: FINISH_ZONE empty to empty (ALL bit set)... [Passed]
028: RESET_WRITE_PTR empty to empty (ALL bit set)... [Passed]
029: OPEN_ZONE implicit open to implicit open (ALL bit set)... [Passed]
030: CLOSE_ZONE implicit open to close (ALL bit set)... [Passed]
031: FINISH_ZONE implicit open to full (ALL bit set)... [Passed]
032: RESET_WRITE_PTR implicit open to empty (ALL bit set)... [Passed]
033: OPEN_ZONE empty to explicit open to explicit open (ALL bit set)... [Passed]
034: CLOSE_ZONE empty to explicit open to empty (ALL bit set)... [Passed]
035: FINISH_ZONE empty to explicit open to full (ALL bit set)... [Passed]
036: RESET_WRITE_PTR empty to explicit open to empty (ALL bit set)... [Passed]
037: OPEN_ZONE implicit open to explicit open to explicit_open (ALL bit set)... [Passed]
038: CLOSE_ZONE implicit open to explicit open to closed (ALL bit set)... [Passed]
039: FINISH_ZONE implicit open to explicit open to full (ALL bit set)... [Passed]
040: RESET_WRITE_PTR implicit open to explicit open to empty (ALL bit set)... [Passed]
041: OPEN_ZONE closed to explicit open (ALL bit set)... [Passed]
042: CLOSE_ZONE closed to closed (ALL bit set)... [Passed]
043: FINISH_ZONE closed to full (ALL bit set)... [Passed]
044: RESET_WRITE_PTR closed to empty test (ALL bit set)... [Passed]
045: OPEN_ZONE full to full (ALL bit set)... [Passed]
046: CLOSE_ZONE full to full (ALL bit set)... [Passed]
047: FINISH_ZONE full to full (ALL bit set)... [Passed]
048: RESET_WRITE_PTR full to empty (ALL bit set)... [Passed]
shehbaz@bluestore:~$ sudo zbc_info /dev/sg1
Device /dev/sg1: ATA ST8000AS0022-1WL SN01
ATA ZAC interface, Host-aware disk model
15628053168 logical blocks of 512 B
1953506646 physical blocks of 4096 B
8001.563 GB capacity
Setting up dev environment.¶
- Partition the drive so that it looks like this:
sdb 8:16 0 7.3T 0 disk
sdb1 8:17 0 15.6G 0 part
sdb2 8:18 0 1G 0 part
sdb3 8:19 0 7.3T 0 part
- run make check
sudo ./run-make-check.sh
- start developer ceph instance
OSD=1 MON=1 MDS=0 ./vstart.sh -d -n -x -l \
-o 'osd objectstore = bluestore' \
-o 'bluestore block = /dev/sdb3' \
-o 'bluestore wal block = /dev/sdb2'
- stop developer ceph instance
./stop.sh
- To run bluestore in gdb, apply following patch
- To run bluestore with a different allocator, run
shehbaz: those are read from the config file (e.g. /etc/ceph/ceph.conf -- add 'bluestore allocator = foo' to the [global] or [osd] section) or the command line args (e.g. --bluestore-allocator foo)
<joshd> space, _, and - are treated equivalently
<shehbaz> joshd, thanks. followup question, I am using vstart.sh script to run the code, could you please elaborate on the command in which we send the --bluestore-allocator option
<joshd> with vstart.sh you can probably use '-o bluestore_allocator=foo' to get it added to the generated ceph.conf, which will get used by all the commands
- To run bluestore without blueFS, use:
OSD=1 MON=1 MDS=0 ./vstart.sh -d -n -x -l \
-o 'osd objectstore = bluestore' \
-o 'bluestore block = /dev/sdb3' \
-o 'bluestore wal block = /dev/sdb2'
-o 'bluestore_bluefs = false'
- Allocation logic
There are two entities associated with allocation - a Freelist Manager and an allocator. The Freelist Manager maintains the On-Disk allocation state of the disk. The Allocator maintains the disk space in memory. The FreeList Allocator state is stored in KVDB. The idea is to maintain the persistent state of disk in KVDB, and the in-memory state in the Allocator.
- Design
Updated by Shehbaz Jaffer almost 8 years ago ยท 16 revisions