CM的私有仓库搭建
2024-10-21 06:07:37
CM的私有仓库搭建
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
我们有时候会因为网络不稳定的因素，导致安装某些服务直接失败。当安装包比较大的时候，我们就得考虑自建私有仓库了，这样会为我们节省不少时间。而自建私有仓库的确是很简单的一种方式，接下来我们就以CM和CDH的软件包为例，来建设一个局域网的镜像库。
一.安装WEB服务器
要托管内部存储库，必须在Cloudera Manager主机可访问的内部主机上安装或使用现有Web服务器，然后将存储库文件下载到Web服务器主机。
1>.安装Apache HTTP Server
[root@node105 ~]# yum -y install httpd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.huaweicloud.com
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 :2.4.-.el7.centos will be installed
--> Processing Dependency: httpd-tools = 2.4.-.el7.centos for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libaprutil-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libapr-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Running transaction check
---> Package apr.x86_64 :1.4.-.el7_4. will be installed
---> Package apr-util.x86_64 :1.5.-.el7 will be installed
---> Package httpd-tools.x86_64 :2.4.-.el7.centos will be installed
---> Package mailcap.noarch :2.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================================================================================================
Package Arch Version Repository Size
======================================================================================================================================================================================================================================================
Installing:
httpd x86_64 2.4.-.el7.centos base 2.7 M
Installing for dependencies:
apr x86_64 1.4.-.el7_4. base k
apr-util x86_64 1.5.-.el7 base k
httpd-tools x86_64 2.4.-.el7.centos base k
mailcap noarch 2.1.-.el7 base k Transaction Summary
======================================================================================================================================================================================================================================================
Install Package (+ Dependent packages) Total size: 3.0 M
Installed size: M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : apr-1.4.-.el7_4..x86_64 /
Installing : apr-util-1.5.-.el7.x86_64 /
Installing : httpd-tools-2.4.-.el7.centos.x86_64 /
Installing : mailcap-2.1.-.el7.noarch /
Installing : httpd-2.4.-.el7.centos.x86_64 /
Verifying : httpd-tools-2.4.-.el7.centos.x86_64 /
Verifying : apr-1.4.-.el7_4..x86_64 /
Verifying : mailcap-2.1.-.el7.noarch /
Verifying : httpd-2.4.-.el7.centos.x86_64 /
Verifying : apr-util-1.5.-.el7.x86_64 / Installed:
httpd.x86_64 :2.4.-.el7.centos Dependency Installed:
apr.x86_64 :1.4.-.el7_4. apr-util.x86_64 :1.5.-.el7 httpd-tools.x86_64 :2.4.-.el7.centos mailcap.noarch :2.1.-.el7 Complete!
[root@node105 ~]#
[root@node105 ~]# yum -y install httpd
2>.配置Httpd服务
当尝试从本地存储库下载包时，跳过此步骤可能会导致错误消息哈希验证失败，尤其是在Cloudera Manager 6及更高版本中。
[root@node105 ~]# cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.`date +%F`
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz
[root@node105 ~]#
[root@node105 ~]# sed -i s'#.tgz#.tgz .parcel#' /etc/httpd/conf/httpd.conf
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz .parcel #其实我就是在.tgz后面加了一个“.parcel”的后缀，这是CM官方强调要做的一个操作！
[root@node105 ~]#
[root@node105 ~]#
3>.启动httpd服务
[root@node105 ~]# systemctl start httpd
[root@node105 ~]#
[root@node105 ~]# systemctl status httpd
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
Active: active (running) since Thu -- :: CST; 7s ago
Docs: man:httpd()
man:apachectl()
Main PID: (httpd)
Status: "Processing requests..."
CGroup: /system.slice/httpd.service
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
└─ /usr/sbin/httpd -DFOREGROUND Feb :: node105.yinzhengjie.org.cn systemd[]: Starting The Apache HTTP Server...
Feb :: node105.yinzhengjie.org.cn systemd[]: Started The Apache HTTP Server.
[root@node105 ~]#
二.制作CM的私有仓库
1>.创建存放cm安装包的目录
[root@node101 ~]# mkdir /var/www/html/cm-5.16.
2>.将cm相关软件从官方地址下载后上传到上面创建的目录中（https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.1/RPMS/x86_64/）
[root@node101 ~]# ll /var/www/html/cm-5.16.
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm #温馨提示：这个软件包我是从Oracle官方网站download下来在上传到服务器上的！因为官方默认仅仅提供了jDK6和JDK7的rpm包！我强烈推荐使用JDK8版本！
[root@node101 ~]#
[root@node101 ~]#
3>.制作本地源
[root@node101 ~]# yum -y install yum-utils createrepo yum-plugin-priorities
Loaded plugins: fastestmirror
base | 3.6 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
Loading mirror speeds from cached hostfile
* base: mirrors.huaweicloud.com
* extras: mirrors.huaweicloud.com
* updates: mirrors..com
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch :0.9.-.el7 will be installed
--> Processing Dependency: python-deltarpm for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: libxml2-python for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: deltarpm for package: createrepo-0.9.-.el7.noarch
---> Package yum-plugin-priorities.noarch :1.1.-.el7 will be installed
---> Package yum-utils.noarch :1.1.-.el7 will be installed
--> Processing Dependency: yum >= 3.4.- for package: yum-utils-1.1.-.el7.noarch
--> Processing Dependency: python-kitchen for package: yum-utils-1.1.-.el7.noarch
--> Running transaction check
---> Package deltarpm.x86_64 :3.6-.el7 will be installed
---> Package libxml2-python.x86_64 :2.9.-.el7_2. will be installed
--> Processing Dependency: libxml2 = 2.9.-.el7_2. for package: libxml2-python-2.9.-.el7_2..x86_64
---> Package python-deltarpm.x86_64 :3.6-.el7 will be installed
---> Package python-kitchen.noarch :1.1.-.el7 will be installed
--> Processing Dependency: python-chardet for package: python-kitchen-1.1.-.el7.noarch
---> Package yum.noarch :3.4.-.el7.centos.0.1 will be updated
---> Package yum.noarch :3.4.-.el7.centos will be an update
--> Processing Dependency: rpm >= 4.11.- for package: yum-3.4.-.el7.centos.noarch
--> Processing Dependency: python-urlgrabber >= 3.10- for package: yum-3.4.-.el7.centos.noarch
--> Running transaction check
---> Package libxml2.x86_64 :2.9.-.el7_1. will be updated
---> Package libxml2.x86_64 :2.9.-.el7_2. will be an update
---> Package python-chardet.noarch :2.2.-.el7_1 will be installed
---> Package python-urlgrabber.noarch :3.10-.el7 will be updated
---> Package python-urlgrabber.noarch :3.10-.el7 will be an update
---> Package rpm.x86_64 :4.11.-.el7 will be updated
--> Processing Dependency: rpm = 4.11.-.el7 for package: rpm-libs-4.11.-.el7.x86_64
--> Processing Dependency: rpm = 4.11.-.el7 for package: rpm-python-4.11.-.el7.x86_64
---> Package rpm.x86_64 :4.11.-.el7 will be an update
--> Running transaction check
---> Package rpm-libs.x86_64 :4.11.-.el7 will be updated
--> Processing Dependency: rpm-libs(x86-) = 4.11.-.el7 for package: rpm-build-libs-4.11.-.el7.x86_64
---> Package rpm-libs.x86_64 :4.11.-.el7 will be an update
---> Package rpm-python.x86_64 :4.11.-.el7 will be updated
---> Package rpm-python.x86_64 :4.11.-.el7 will be an update
--> Running transaction check
---> Package rpm-build-libs.x86_64 :4.11.-.el7 will be updated
---> Package rpm-build-libs.x86_64 :4.11.-.el7 will be an update
--> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================================
Installing:
createrepo noarch 0.9.-.el7 base k
yum-plugin-priorities noarch 1.1.-.el7 base k
yum-utils noarch 1.1.-.el7 base k
Installing for dependencies:
deltarpm x86_64 3.6-.el7 base k
libxml2-python x86_64 2.9.-.el7_2. base k
python-chardet noarch 2.2.-.el7_1 base k
python-deltarpm x86_64 3.6-.el7 base k
python-kitchen noarch 1.1.-.el7 base k
Updating for dependencies:
libxml2 x86_64 2.9.-.el7_2. base k
python-urlgrabber noarch 3.10-.el7 base k
rpm x86_64 4.11.-.el7 base 1.2 M
rpm-build-libs x86_64 4.11.-.el7 base k
rpm-libs x86_64 4.11.-.el7 base k
rpm-python x86_64 4.11.-.el7 base k
yum noarch 3.4.-.el7.centos base 1.2 M Transaction Summary
===================================================================================================================================================================================
Install Packages (+ Dependent packages)
Upgrade ( Dependent packages) Total download size: 4.7 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(/): libxml2-2.9.-.el7_2..x86_64.rpm | kB ::
(/): python-deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): python-kitchen-1.1.-.el7.noarch.rpm | kB ::
(/): python-chardet-2.2.-.el7_1.noarch.rpm | kB ::
(/): python-urlgrabber-3.10-.el7.noarch.rpm | kB ::
(/): rpm-build-libs-4.11.-.el7.x86_64.rpm | kB ::
(/): rpm-4.11.-.el7.x86_64.rpm | 1.2 MB ::
(/): yum-3.4.-.el7.centos.noarch.rpm | 1.2 MB ::
(/): yum-plugin-priorities-1.1.-.el7.noarch.rpm | kB ::
(/): yum-utils-1.1.-.el7.noarch.rpm | kB ::
(/): rpm-libs-4.11.-.el7.x86_64.rpm | kB ::
(/): libxml2-python-2.9.-.el7_2..x86_64.rpm | kB ::
(/): rpm-python-4.11.-.el7.x86_64.rpm | kB ::
createrepo-0.9.-.el7.noarch FAILED
http://centos.ustc.edu.cn/centos/7.6.1810/os/x86_64/Packages/createrepo-0.9.9-28.el7.noarch.rpm: [Errno 12] Timeout on http://centos.ustc.edu.cn/centos/7.6.1810/os/x86_64/Packages/createrepo-0.9.9-28.el7.noarch.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
Trying other mirror.
(/): createrepo-0.9.-.el7.noarch.rpm | kB ::
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 4.7 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : rpm-4.11.-.el7.x86_64 /
Updating : rpm-libs-4.11.-.el7.x86_64 /
Installing : deltarpm-3.6-.el7.x86_64 /
Installing : python-deltarpm-3.6-.el7.x86_64 /
Updating : rpm-build-libs-4.11.-.el7.x86_64 /
Updating : rpm-python-4.11.-.el7.x86_64 /
Installing : python-chardet-2.2.-.el7_1.noarch /
Installing : python-kitchen-1.1.-.el7.noarch /
Updating : libxml2-2.9.-.el7_2..x86_64 /
Installing : libxml2-python-2.9.-.el7_2..x86_64 /
Updating : python-urlgrabber-3.10-.el7.noarch /
Updating : yum-3.4.-.el7.centos.noarch /
Installing : yum-plugin-priorities-1.1.-.el7.noarch /
Installing : createrepo-0.9.-.el7.noarch /
Installing : yum-utils-1.1.-.el7.noarch /
Cleanup : yum-3.4.-.el7.centos.0.1.noarch /
Cleanup : rpm-python-4.11.-.el7.x86_64 /
Cleanup : python-urlgrabber-3.10-.el7.noarch /
Cleanup : rpm-build-libs-4.11.-.el7.x86_64 /
Cleanup : rpm-libs-4.11.-.el7.x86_64 /
Cleanup : rpm-4.11.-.el7.x86_64 /
Cleanup : libxml2-2.9.-.el7_1..x86_64 /
Verifying : rpm-python-4.11.-.el7.x86_64 /
Verifying : rpm-libs-4.11.-.el7.x86_64 /
Verifying : python-urlgrabber-3.10-.el7.noarch /
Verifying : libxml2-2.9.-.el7_2..x86_64 /
Verifying : yum-plugin-priorities-1.1.-.el7.noarch /
Verifying : rpm-4.11.-.el7.x86_64 /
Verifying : rpm-build-libs-4.11.-.el7.x86_64 /
Verifying : libxml2-python-2.9.-.el7_2..x86_64 /
Verifying : deltarpm-3.6-.el7.x86_64 /
Verifying : createrepo-0.9.-.el7.noarch /
Verifying : python-deltarpm-3.6-.el7.x86_64 /
Verifying : yum-utils-1.1.-.el7.noarch /
Verifying : python-chardet-2.2.-.el7_1.noarch /
Verifying : python-kitchen-1.1.-.el7.noarch /
Verifying : yum-3.4.-.el7.centos.noarch /
Verifying : rpm-4.11.-.el7.x86_64 /
Verifying : libxml2-2.9.-.el7_1..x86_64 /
Verifying : rpm-libs-4.11.-.el7.x86_64 /
Verifying : rpm-build-libs-4.11.-.el7.x86_64 /
Verifying : yum-3.4.-.el7.centos.0.1.noarch /
Verifying : rpm-python-4.11.-.el7.x86_64 /
Verifying : python-urlgrabber-3.10-.el7.noarch / Installed:
createrepo.noarch :0.9.-.el7 yum-plugin-priorities.noarch :1.1.-.el7 yum-utils.noarch :1.1.-.el7 Dependency Installed:
deltarpm.x86_64 :3.6-.el7 libxml2-python.x86_64 :2.9.-.el7_2. python-chardet.noarch :2.2.-.el7_1 python-deltarpm.x86_64 :3.6-.el7 python-kitchen.noarch :1.1.-.el7 Dependency Updated:
libxml2.x86_64 :2.9.-.el7_2. python-urlgrabber.noarch :3.10-.el7 rpm.x86_64 :4.11.-.el7 rpm-build-libs.x86_64 :4.11.-.el7 rpm-libs.x86_64 :4.11.-.el7
rpm-python.x86_64 :4.11.-.el7 yum.noarch :3.4.-.el7.centos Complete!
[root@node101 ~]#
[root@node101 ~]# yum -y install yum-utils createrepo yum-plugin-priorities #安装本地源制作相关工具
[root@node101 ~]# cd /var/www/html/cm-5.16.
[root@node101 cm-5.16.]# ll
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm
[root@node101 cm-5.16.]#
[root@node101 cm-5.16.]# createrepo .
Spawning worker with pkgs
Spawning worker with pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@node101 cm-5.16.]#
三.选定节点测试
1>.备份主机之前的CM
[root@node110 ~]# mkdir /etc/yum.repos.d/repo-bak
[root@node110 ~]#
[root@node110 ~]# mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/repo-bak/
[root@node110 ~]#
2>.编写yum源文件内容
[root@node110 ~]# cat /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
# Packages for Cloudera Manager, Version , on RedHat or CentOS x86_64
name=Cloudera Manager
baseurl=http://node101.yinzhengjie.org.cn/cm-5.16.1/
gpgkey =https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
enabled =
gpgcheck = [root@node110 ~]#
3>.查看yum源的列表
[root@node110 ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
repo id repo name status
cloudera-manager Cloudera Manager
repolist:
[root@node110 ~]#
4>.测试是否可以正常安装软件
[root@node110 ～]# yum -y install cloudera-manager-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 will be installed
--> Processing Dependency: cloudera-manager-daemons = 5.16. for package: cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64
--> Running transaction check
---> Package cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
cloudera-manager-server x86_64 5.16.-.cm5161.p0..el7 cloudera-manager 8.5 k
Installing for dependencies:
cloudera-manager-daemons x86_64 5.16.-.cm5161.p0..el7 cloudera-manager M Transaction Summary
========================================================================================================================================================================
Install Package (+ Dependent package) Total download size: M
Installed size: M
Downloading packages:
(/): cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm | 8.5 kB ::
(/): cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm | MB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total MB/s | MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 /
Installing : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 / Installed:
cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 Dependency Installed:
cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 Complete!
[root@node110 ～]#
[root@node110 ～]# yum -y install cloudera-manager-server
[root@node110 ~]# yum -y install java
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package jdk1..x86_64 :1.8.0_201-fcs will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
jdk1. x86_64 :1.8.0_201-fcs cloudera-manager M Transaction Summary
========================================================================================================================================================================
Install Package Total download size: M
Installed size: M
Downloading packages:
jdk-8u201-linux-x64.rpm | MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : :jdk1.-1.8.0_201-fcs.x86_64 /
Unpacking JAR files...
tools.jar...
plugin.jar...
javaws.jar...
deploy.jar...
rt.jar...
jsse.jar...
charsets.jar...
localedata.jar...
Verifying : :jdk1.-1.8.0_201-fcs.x86_64 / Installed:
jdk1..x86_64 :1.8.0_201-fcs Complete!
[root@node110 ~]#
[root@node110 ~]# yum -y install java
[root@node110 ~]# java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) -Bit Server VM (build 25.201-b09, mixed mode)
[root@node110 ~]#
博主推荐阅读文档：
最新文章
- SQL Server-聚焦IN VS EXISTS VS JOIN性能分析（十九）
- bzoj2330: [SCOI2011]糖果
- AD帐户操作C#示例代码（一）——导入用户信息
- BZOJ 4144: [AMPPZ2014]Petrol
- apache开启虚拟主机localhost无法访问
- Minimum Size Subarray Sum —— LeetCode
- C 本地文件夸网文件Cp操作
- yii2 去掉index.php的方法
- HDU 2149-Public Sale(巴什博奕)
- SSH框架基础
- C# 合并、拆分PPT幻灯片
- 《Java》第九周学习总结
- 生成器(generator,yield),next,send
- 将模型.pb文件在tensorboard中展示结构
- 13、SpringBoot-CRUD员工修改操作/删除
- asp.net core mvc中如何把二级域名绑定到特定的控制器上
- unity制作简单血条
- rem怎么计算
- [转载]np.where()使用说明
- 《JAVA核心卷I》之Java基本程序结构