官方文档地址：http://docs.minio.org.cn/docs/master/distributed-minio-quickstart-guide

前提条件：分布式Minio至少需要4个硬盘，使用分布式Minio自动引入了纠删码功能。

注意:

分布式Minio里所有的节点需要有同样的access秘钥和secret秘钥，这样这些节点才能建立联接。为了实现这个，你需要在执行minio server命令之前，先将access秘钥和secret秘钥export成环境变量。
分布式Minio使用的磁盘里必须是干净的，里面没有数据。
下面示例里的IP仅供示例参考，你需要改成你真实用到的IP和文件夹路径。
分布式Minio里的节点时间差不能超过3秒，你可以使用NTP 来保证时间一致。
在Windows下运行分布式Minio处于实验阶段，请悠着点使用。

这里演示使用一台主机上的4块磁盘，每个磁盘1G

官方步骤：

export MINIO_ACCESS_KEY=<ACCESS_KEY>

export MINIO_SECRET_KEY=<SECRET_KEY>

minio server http://192.168.1.11/export1 http://192.168.1.12/export2 \

               http://192.168.1.13/export3 http://192.168.1.14/export4 \

               http://192.168.1.15/export5 http://192.168.1.16/export6 \

               http://192.168.1.17/export7 http://192.168.1.18/export8

实际操作步骤：

1.在一台主机上创建4个不同的路径模拟4个不同的磁盘来进行

从执行结果来看，这种方式行不通

export MINIO_ACCESS_KEY=admin

export MINIO_SECRET_KEY=12345678

mkdir -p /mnt/{data1,data2,data3,data4}

/usr/local/minio/bin/minio server http://192.168.20.102/mnt/data1 http://192.168.20.102/mnt/data2 http://192.168.20.102/mnt/data3 http://192.168.20.102/mnt/data4 --console-address ":9001"

# 操作完毕后提示信息如下

WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.

         Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD

Formatting 1st pool, 1 set(s), 4 drives per set.

WARNING: Host 192.168.20.102 has more than 2 drives of set. A host failure will result in data becoming unavailable.

API: SYSTEM()

Time: 09:24:00 UTC 03/10/2022

Error: Disk `/mnt/data1` is part of root disk, will not be used (*errors.errorString)

       7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown()

       6: cmd/format-erasure.go:792:cmd.initFormatErasure()

       5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats()

       4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure()

       3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools()

       2: cmd/server-main.go:640:cmd.newObjectLayer()

       1: cmd/server-main.go:491:cmd.serverMain()

API: SYSTEM()

Time: 09:24:00 UTC 03/10/2022

Error: Disk `/mnt/data2` is part of root disk, will not be used (*errors.errorString)

       7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown()

       6: cmd/format-erasure.go:792:cmd.initFormatErasure()

       5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats()

       4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure()

       3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools()

       2: cmd/server-main.go:640:cmd.newObjectLayer()

       1: cmd/server-main.go:491:cmd.serverMain()

API: SYSTEM()

Time: 09:24:00 UTC 03/10/2022

Error: Disk `/mnt/data3` is part of root disk, will not be used (*errors.errorString)

       7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown()

       6: cmd/format-erasure.go:792:cmd.initFormatErasure()

       5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats()

       4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure()

       3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools()

       2: cmd/server-main.go:640:cmd.newObjectLayer()

       1: cmd/server-main.go:491:cmd.serverMain()

API: SYSTEM()

Time: 09:24:00 UTC 03/10/2022

Error: Disk `/mnt/data4` is part of root disk, will not be used (*errors.errorString)

       7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown()

       6: cmd/format-erasure.go:792:cmd.initFormatErasure()

       5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats()

       4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure()

       3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools()

       2: cmd/server-main.go:640:cmd.newObjectLayer()

       1: cmd/server-main.go:491:cmd.serverMain()

Formatting 1st pool, 1 set(s), 4 drives per set.

WARNING: Host 192.168.20.102 has more than 2 drives of set. A host failure will result in data becoming unavailable.

改进步骤：

# 根据上一步的执行提示，更新设置的环境变量名

export MINIO_ROOT_USER=admin

export MINIO_ROOT_PASSWORD=12345678

# 新加四块磁盘，每个磁盘空间1G，不分区，直接格式化后挂载

# 不重启，直接刷新磁盘数据总线，获取新加的磁盘

for host in $(ls /sys/class/scsi_host) ; do echo "- - -" > /sys/class/scsi_host/$host/scan; done

# lsblk

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT

sda               8:0    0   40G  0 disk

├─sda1            8:1    0    1G  0 part /boot

└─sda2            8:2    0   39G  0 part

  ├─centos-root 253:0    0 35.1G  0 lvm  /

  └─centos-swap 253:1    0  3.9G  0 lvm  [SWAP]

sdb               8:16   0    1G  0 disk

sdc               8:32   0    1G  0 disk

sdd               8:48   0    1G  0 disk

sde               8:64   0    1G  0 disk 

[root@myrabbit3 ~]# mkfs.ext4 /dev/sdb

mke2fs 1.42.9 (28-Dec-2013)

/dev/sdb is entire device, not just one partition!

无论如何也要继续? (y,n) y

文件系统标签=

OS type: Linux

块大小=4096 (log=2)

分块大小=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

65536 inodes, 262144 blocks

13107 blocks (5.00%) reserved for the super user

第一个数据块=0

Maximum filesystem blocks=268435456

8 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (8192 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

[root@myrabbit3 ~]# mkfs.ext4 /dev/sdc

mke2fs 1.42.9 (28-Dec-2013)

/dev/sdc is entire device, not just one partition!

无论如何也要继续? (y,n) y

文件系统标签=

OS type: Linux

块大小=4096 (log=2)

分块大小=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

65536 inodes, 262144 blocks

13107 blocks (5.00%) reserved for the super user

第一个数据块=0

Maximum filesystem blocks=268435456

8 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (8192 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

[root@myrabbit3 ~]# mkfs.ext4 /dev/sdd

mke2fs 1.42.9 (28-Dec-2013)

/dev/sdd is entire device, not just one partition!

无论如何也要继续? (y,n) y

文件系统标签=

OS type: Linux

块大小=4096 (log=2)

分块大小=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

65536 inodes, 262144 blocks

13107 blocks (5.00%) reserved for the super user

第一个数据块=0

Maximum filesystem blocks=268435456

8 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (8192 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

[root@myrabbit3 ~]# mkfs.ext4 /dev/sde

mke2fs 1.42.9 (28-Dec-2013)

/dev/sde is entire device, not just one partition!

无论如何也要继续? (y,n) y

文件系统标签=

OS type: Linux

块大小=4096 (log=2)

分块大小=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

65536 inodes, 262144 blocks

13107 blocks (5.00%) reserved for the super user

第一个数据块=0

Maximum filesystem blocks=268435456

8 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (8192 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

[root@myrabbit3 ~]# mount /dev/sdb /mnt/data1/

[root@myrabbit3 ~]# mount /dev/sdc /mnt/data2/

[root@myrabbit3 ~]# mount /dev/sdd /mnt/data3/

[root@myrabbit3 ~]# mount /dev/sde /mnt/data4/

# lsblk

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT

sda               8:0    0   40G  0 disk

├─sda1            8:1    0    1G  0 part /boot

└─sda2            8:2    0   39G  0 part

  ├─centos-root 253:0    0 35.1G  0 lvm  /

  └─centos-swap 253:1    0  3.9G  0 lvm  [SWAP]

sdb               8:16   0    1G  0 disk /mnt/data1

sdc               8:32   0    1G  0 disk /mnt/data2

sdd               8:48   0    1G  0 disk /mnt/data3

sde               8:64   0    1G  0 disk /mnt/data4

# 记得把磁盘挂载信息写道/etc/fstab文件中

# /usr/local/minio/bin/minio server http://192.168.20.102/mnt/data1 http://192.168.20.102/mnt/data2 http://192.168.20.102/mnt/data3 http://192.168.20.102/mnt/data4 --console-address ":9001"

WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.

         Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD

Automatically configured API requests per node based on available memory on the system: 55

Status:         4 Online, 0 Offline.

API: http://192.168.20.102:9000  http://127.0.0.1:9000

RootUser: admin

RootPass: 12345678 

Console: http://192.168.20.102:9001 http://127.0.0.1:9001

RootUser: admin

RootPass: 12345678 

Command-line: https://docs.min.io/docs/minio-client-quickstart-guide

   $ mc alias set myminio http://192.168.20.102:9000 admin 12345678

Documentation: https://docs.min.io

扩展现有的分布式集群

例如我们是通过区的方式启动MinIO集群，命令行如下：

export MINIO_ACCESS_KEY=<ACCESS_KEY>

export MINIO_SECRET_KEY=<SECRET_KEY>

minio server http://host{1...32}/export{1...32}

MinIO支持通过命令，指定新的集群来扩展现有集群（纠删码模式），命令行如下：

export MINIO_ACCESS_KEY=<ACCESS_KEY>

export MINIO_SECRET_KEY=<SECRET_KEY>

minio server http://host{1...32}/export{1...32} http://host{33...64}/export{1...32}

新的对象上传请求会自动分配到最少使用的集群上。通过以上扩展策略，您就可以按需扩展您的集群。重新配置后重启集群，会立即在集群中生效，并对现有集群无影响。如上命令中，我们可以把原来的集群看做一个区，新增集群看做另一个区，新对象按每个区域中的可用空间比例放置在区域中。在每个区域内，基于确定性哈希算法确定位置。

说明: 您添加的每个区域必须具有与原始区域相同的磁盘数量（纠删码集）大小，以便维持相同的数据冗余SLA。例如，第一个区有8个磁盘，您可以将集群扩展为16个、32个或1024个磁盘的区域，您只需确保部署的SLA是原始区域的倍数即可。

多个节点的存储容量和就是分布式Minio的存储容量。

巴特西

分布式安装部署MinIO

扩展现有的分布式集群

最新文章

热门文章