Home | 简体中文 | 繁体中文 | 杂文 | 打赏(Donations) | ITEYE 博客 | OSChina 博客 | Facebook | Linkedin | 知乎专栏 | Search | About

第 73 章 Distributed File Systems

目录

73.1. DRBD (Distributed Replicated Block Device)
73.1.1. disk and partition
73.1.2. Installation
73.1.3. configure
73.1.4. Starting
73.1.5. Using
73.2. Network Block Device protocol
73.2.1. nbd-server - Network Block Device protocol - server
73.2.2. nbd-client - Network Block Device protocol - client
73.3. GridFS
73.3.1. nginx-gridfs
73.3.2. lighttpd-gridfs
73.4. Moose File System
73.4.1. Master server installation
73.4.2. Backup server (metalogger) installation
73.4.3. Chunk servers installation
73.4.4. Users’ computers installation
73.4.5. Testing MFS
73.5. LizardFS
73.6. Ceph
73.6.1. Installation on Ubuntu
73.6.2. Installation on CentOS
73.6.2.1. mon
73.6.2.2. mds
73.6.2.3. osd
73.6.2.4. client
73.6.2.5. RADOS Gateway
73.6.3. Block Devices
73.7. GlusterFS
73.7.1. glusterfs-server
73.7.2. glusterfs-client
73.7.3. Testing
73.7.4. RAID
73.7.4.1. Mirror
73.7.4.2. Strip
73.7.5. Filesystem Administration
73.7.6. CentOS 6.3
73.8. Lustre
73.9. MogileFS
73.10. Kosmos distributed file system (KFS)
73.11. Hadoop - HDFS
73.12. BeeGFS - The Parallel Cluster File System
73.13. Coda
73.14. OpenAFS

73.1. DRBD (Distributed Replicated Block Device)

Homepage: http://www.drbd.org/

实验环境需要两台电脑,如果你没有,建议你使用VMware,并且为每一个虚拟机添加两块硬盘。

实验环境

  1. master: 192.168.0.1 DRBD:/dev/sdb

  2. slave: 192.168.0.2 DRBD:/dev/sdb

73.1.1. disk and partition

Each of the following steps must be completed on both nodes

show all of disk and partition

neo@master:~$ sudo sfdisk -s
/dev/sda:   8388608
/dev/sdb:   2097152
total: 10485760 blocks
			

create a new partition on the disk /dev/sdb

$ sudo cfdisk /dev/sdb			
			

you must have extended partition

check partition

neo@master:~$ sudo fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000301bd

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         993     7976241   83  Linux
/dev/sda2             994        1044      409657+   5  Extended
/dev/sda5             994        1044      409626   82  Linux swap / Solaris

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         261     2096451    5  Extended
/dev/sdb5               1         261     2096419+  83  Linux

			

format /dev/sdb1

neo@master:~$ sudo mkfs.ext3 /dev/sdb1
			

you also can using other file system

reiserfs

neo@master:~$ sudo mkfs.reiserfs /dev/sdb1		
			

I suggest you using reiserfs.

73.1.2. Installation

Each of the following steps must be completed on both nodes

search drbd8-utils package

neo@master:~$ apt-cache search drbd
drbd8-utils - RAID 1 over tcp/ip for Linux utilities
drbd0.7-module-source - RAID 1 over tcp/ip for Linux module source
drbd0.7-utils - RAID 1 over tcp/ip for Linux utilities
drbdlinks - Manages symlinks into a shared DRBD partition
			

installation

neo@master:~$ sudo apt-get install drbd8-utils			
			

to add modules from the Linux Kernel

neo@master:~$ sudo modprobe drbd
neo@master:~$ lsmod |grep drbd
drbd                  213000  0
cn                      9632  1 drbd
			
			

73.1.3. configure

Each of the following steps must be completed on both nodes

backup configure file

			
neo@master:~$ sudo cp /etc/drbd.conf /etc/drbd.conf.old
			
			

edit /etc/drbd.conf

global { 
  usage-count yes; 
}
common {
  protocol C;
}
resource r0 {
  on master {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   192.168.0.1:7789;
    meta-disk internal;
  }
  on slave {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   10.1.1.32:7789;
    meta-disk internal;
  }
}
			

73.1.4. Starting

Each of the following steps must be completed on both nodes.

neo@master:~$ sudo drbdadm create-md r0	
neo@master:~$ sudo drbdadm attach r0
neo@master:~$ sudo drbdadm connect r0
neo@master:~$ sudo drbdadm -- --overwrite-data-of-peer primary r0

neo@slave:~$ sudo drbdadm create-md r0		
neo@slave:~$ sudo drbdadm attach r0
neo@slave:~$ sudo drbdadm connect r0

			

master

			
neo@master:~$ sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

slave

			
neo@slave:~# sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

status

neo@master:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:StandAlone st:Primary/Unknown ds:UpToDate/DUnknown   r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0
 1: cs:Connected st:Secondary/Secondary ds:Diskless/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
			
			

73.1.5. Using

master

neo@master:~$ sudo drbdadm primary all		
neo@master:~$ sudo mkfs.reiserfs /dev/drbd0
neo@master:~$ sudo mkdir /mnt/drbd0
neo@master:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@master:~$ sudo touch /mnt/drbd0/helloworld.tmp
neo@master:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  1.3G  6.0G  18% /
varrun                125M  216K  125M   1% /var/run
varlock               125M  8.0K  125M   1% /var/lock
udev                  125M   60K  125M   1% /dev
devshm                125M     0  125M   0% /dev/shm
/dev/drbd0            2.0G   33M  2.0G   2% /mnt/drbd0
neo@master:~$ sudo dd if=/dev/zero of=/mnt/drbd0/tempfile1.tmp bs=104857600 count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 0.564911 s, 186 MB/s
neo@master:~$ sudo umount /mnt/drbd0/
neo@master:~$ sudo drbdadm secondary all
			

slave

neo@slave:~$ sudo drbdadm primary all			
neo@slave:~$ sudo mkdir /mnt/drbd0
neo@slave:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@slave:~$ ls /mnt/drbd0/
helloworld.tmp  tempfile1.tmp