Home | 简体中文 | 繁体中文 | 杂文 | Github | 知乎专栏 | 51CTO学院 | CSDN程序员研修院 | OSChina 博客 | 腾讯云社区 | 阿里云栖社区 | Facebook | Linkedin | Youtube | 打赏(Donations) | About
知乎专栏多维度架构

第 64 章 Distributed File Systems

目录

64.1. DRBD (Distributed Replicated Block Device)
64.1.1. disk and partition
64.1.2. Installation
64.1.3. configure
64.1.4. Starting
64.1.5. Using
64.2. Network Block Device protocol
64.2.1. nbd-server - Network Block Device protocol - server
64.2.2. nbd-client - Network Block Device protocol - client
64.3. GridFS
64.3.1. nginx-gridfs
64.3.2. lighttpd-gridfs
64.4. Moose File System
64.4.1. Master server installation
64.4.2. Backup server (metalogger) installation
64.4.3. Chunk servers installation
64.4.4. Users’ computers installation
64.4.5. Testing MFS
64.5. LizardFS
64.6. Ceph
64.6.1. Installation on Ubuntu
64.6.2. Installation on CentOS
64.6.3. Block Devices
64.7. GlusterFS
64.7.1. glusterfs-server
64.7.2. glusterfs-client
64.7.3. Testing
64.7.4. RAID
64.7.5. Filesystem Administration
64.7.6. CentOS 6.3
64.8. Lustre
64.9. MogileFS
64.10. Kosmos distributed file system (KFS)
64.11. Hadoop - HDFS
64.12. BeeGFS - The Parallel Cluster File System
64.13. Coda
64.14. OpenAFS

64.1. DRBD (Distributed Replicated Block Device)

Homepage: http://www.drbd.org/

实验环境需要两台电脑,如果你没有,建议你使用VMware,并且为每一个虚拟机添加两块硬盘。

实验环境

  1. master: 192.168.0.1 DRBD:/dev/sdb

  2. slave: 192.168.0.2 DRBD:/dev/sdb

64.1.1. disk and partition

Each of the following steps must be completed on both nodes

show all of disk and partition

neo@master:~$ sudo sfdisk -s
/dev/sda:   8388608
/dev/sdb:   2097152
total: 10485760 blocks
			

create a new partition on the disk /dev/sdb

$ sudo cfdisk /dev/sdb			
			

you must have extended partition

check partition

neo@master:~$ sudo fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000301bd

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         993     7976241   83  Linux
/dev/sda2             994        1044      409657+   5  Extended
/dev/sda5             994        1044      409626   82  Linux swap / Solaris

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         261     2096451    5  Extended
/dev/sdb5               1         261     2096419+  83  Linux

			

format /dev/sdb1

neo@master:~$ sudo mkfs.ext3 /dev/sdb1
			

you also can using other file system

reiserfs

neo@master:~$ sudo mkfs.reiserfs /dev/sdb1		
			

I suggest you using reiserfs.

64.1.2. Installation

Each of the following steps must be completed on both nodes

search drbd8-utils package

neo@master:~$ apt-cache search drbd
drbd8-utils - RAID 1 over tcp/ip for Linux utilities
drbd0.7-module-source - RAID 1 over tcp/ip for Linux module source
drbd0.7-utils - RAID 1 over tcp/ip for Linux utilities
drbdlinks - Manages symlinks into a shared DRBD partition
			

installation

neo@master:~$ sudo apt-get install drbd8-utils			
			

to add modules from the Linux Kernel

neo@master:~$ sudo modprobe drbd
neo@master:~$ lsmod |grep drbd
drbd                  213000  0
cn                      9632  1 drbd
			
			

64.1.3. configure

Each of the following steps must be completed on both nodes

backup configure file

			
neo@master:~$ sudo cp /etc/drbd.conf /etc/drbd.conf.old
			
			

edit /etc/drbd.conf

global { 
  usage-count yes; 
}
common {
  protocol C;
}
resource r0 {
  on master {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   192.168.0.1:7789;
    meta-disk internal;
  }
  on slave {
    device    /dev/drbd0;
    disk      /dev/sdb5;
    address   10.1.1.32:7789;
    meta-disk internal;
  }
}
			

64.1.4. Starting

Each of the following steps must be completed on both nodes.

neo@master:~$ sudo drbdadm create-md r0	
neo@master:~$ sudo drbdadm attach r0
neo@master:~$ sudo drbdadm connect r0
neo@master:~$ sudo drbdadm -- --overwrite-data-of-peer primary r0

neo@slave:~$ sudo drbdadm create-md r0		
neo@slave:~$ sudo drbdadm attach r0
neo@slave:~$ sudo drbdadm connect r0

			

master

			
neo@master:~$ sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

slave

			
neo@slave:~# sudo drbdadm create-md r0
v08 Magic number not found
md_offset 2146725888
al_offset 2146693120
bm_offset 2146627584

Found some data
 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block sucessfully created.
success
			
			

status

neo@master:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:StandAlone st:Primary/Unknown ds:UpToDate/DUnknown   r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0
 1: cs:Connected st:Secondary/Secondary ds:Diskless/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
			
			

64.1.5. Using

master

neo@master:~$ sudo drbdadm primary all		
neo@master:~$ sudo mkfs.reiserfs /dev/drbd0
neo@master:~$ sudo mkdir /mnt/drbd0
neo@master:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@master:~$ sudo touch /mnt/drbd0/helloworld.tmp
neo@master:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.6G  1.3G  6.0G  18% /
varrun                125M  216K  125M   1% /var/run
varlock               125M  8.0K  125M   1% /var/lock
udev                  125M   60K  125M   1% /dev
devshm                125M     0  125M   0% /dev/shm
/dev/drbd0            2.0G   33M  2.0G   2% /mnt/drbd0
neo@master:~$ sudo dd if=/dev/zero of=/mnt/drbd0/tempfile1.tmp bs=104857600 count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 0.564911 s, 186 MB/s
neo@master:~$ sudo umount /mnt/drbd0/
neo@master:~$ sudo drbdadm secondary all
			

slave

neo@slave:~$ sudo drbdadm primary all			
neo@slave:~$ sudo mkdir /mnt/drbd0
neo@slave:~$ sudo mount /dev/drbd0 /mnt/drbd0/
neo@slave:~$ ls /mnt/drbd0/
helloworld.tmp  tempfile1.tmp