Ceph介绍
Ceph是统一存储系统,支持三种接口。
Object:有原生的API,而且也兼容Swift和S3的API
Block:支持精简配置、快照、克隆
File:Posix接口,支持快照
Ceph也是分布式存储系统,它的特点是:
高扩展性:使用普通x86服务器,支持10~1000台服务器,支持TB到PB级的扩展。
高可靠性:没有单点故障,多数据副本,自动管理,自动修复。
高性能:数据分布均衡,并行化度高。对于objects storage和block storage,不需要元数据服务器。
Ceph架构
组件
Ceph的底层是RADOS,它的意思是“A reliable, autonomous, distributed object storage”。 RADOS由两个组件组成:
OSD: Object Storage Device,提供存储资源。
Monitor:维护整个Ceph集群的全局状态。
环境:三台装有CentOS 7的 主机,每个主机有三个磁盘(虚拟机磁盘要大于100G)
[root@mon-1 cluster]# cat /etc/RedHat-release
CentOS Linux release 7.2.1511 (Core)
[root@mon-1 cluster]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
fd0 2:0 1 4K 0 disk
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 500M 0 part /boot
└─sda2 8:2 0 19.5G 0 part
├─centos-root 253:0 0 18.5G 0 lvm /
└─centos-swap 253:1 0 1G 0 lvm [SWAP]
sdb 8:16 0 200G 0 disk
sdc 8:32 0 200G 0 disk
sdd 8:48 0 200G 0 disk
sr0 11:0 1 603M 0 rom
修改主机名并添加hosts
vim /etc/hostname #由于7和6.5修改主机名的方法不一样,这里举个例子
mon-1
[root@osd-1 ceph-osd-1]# cat /etc/hosts
192.168.50.123 mon-1
192.168.50.124 osd-1
192.168.50.125 osd-2
配置ssh无密码登陆
[root@localhost ~]# ssh-keygen -t rsa -P ''
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
62:b0:4c:aa:e5:37:92:89:4d:db:c3:38:e2:f1:2a:d6 root@admin-node
The key's randomart image is:
+--[ RSA 2048]----+
| |
| |
| o |
| + o |
| + o o S |
| B B . . |
|+.@ * |
|oooE o |
|oo.. |
+-----------------+
ssh-copy-id mon-1
ssh-copy-id osd-1
ssh-copy-id osd-2
2.集群配置
主机IP功能
3.环境清理
如果之前部署失败了,不必删除ceph客户端,或者重新搭建虚拟机,只需要在每个节点上执行如下指令即可将环境清理至刚安装完ceph客户端时的状态!强烈建议在旧集群上搭建之前清理干净环境,否则会发生各种异常情况。
ps aux|grep ceph |awk '{print $2}'|xargs kill -9
ps -ef|grep ceph
#确保此时所有ceph进程都已经关闭!!!如果没有关闭,多执行几次。
umount /var/lib/ceph/osd/*
rm -rf /var/lib/ceph/osd/*
rm -rf /var/lib/ceph/mon/*
rm -rf /var/lib/ceph/mds/*
rm -rf /var/lib/ceph/bootstrap-mds/*
rm -rf /var/lib/ceph/bootstrap-osd/*
rm -rf /var/lib/ceph/bootstrap-mon/*
rm -rf /var/lib/ceph/tmp/*
rm -rf /etc/ceph/*
rm -rf /var/run/ceph/*
安装部署流程
yum源及ceph的安装
需要在每个主机上执行以下命令
yum clean all
rm -rf /etc/yum.repos.d/*.repo
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
sed -i '/aliyuncs/d' /etc/yum.repos.d/CentOS-Base.repo
sed -i '/aliyuncs/d' /etc/yum.repos.d/epel.repo
sed -i 's/$releasever/7.2.1511/g' /etc/yum.repos.d/CentOS-Base.repo
增加ceph源
vim /etc/yum.repos.d/ceph.repo
[ceph]
name=ceph
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/x86_64/
gpgcheck=0
[ceph-noarch]
name=cephnoarch
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch/
gpgcheck=0
安装ceph客户端
yum -y install ceph ceph-radosgw ntp ntpdate
关闭selinux和firewalld
sed -i 's/SELINUX=.*/SELINUX=disabled/' /etc/selinux/config #重启机器
systemctl stop firewalld
systemctl disable firewalld
同步各个节点时间:
ntpdate -s time-a.nist.gov
echo ntpdate -s time-a.nist.gov >> /etc/rc.d/rc.local
echo "01 01 * * * /usr/sbin/ntpdate -s time-a.nist.gov >> /dev/null 2>&1" >> /etc/crontab
开始部署
在部署节点(mon-1)安装ceph-deploy,下文的部署节点统一指mon-1:
[root@mon-1 ~]# yum -y install ceph-deploy
[root@mon-1 ~]# ceph-deploy --version
1.5.36
[root@mon-1 ~]# ceph -v
ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b)
在部署节点创建部署目录
[root@mon-1 ~]# mkdir cluster
[root@mon-1 ~]# cd cluster
[root@mon-1 cluster]# ceph-deploy new mon-1 osd-1 osd-2
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.36): /usr/bin/ceph-deploy new mon-1 osd-1 osd-2
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] func : <function new at 0x1803230>
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x186d440>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] ssh_copykey : True
[ceph_deploy.cli][INFO ] mon : ['mon-1', 'osd-1', 'osd-2']
[ceph_deploy.cli][INFO ] public_network : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] cluster_network : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] fsid : None
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf... #如果能准确无误的走到这一步就算成功了
更多详情见请继续阅读下一页的精彩内容: http://www.linuxidc.com/Linux/2017-03/141308p2.htm