0%

ambari-2.4.1

2016-10-24-ambari-2.4.1

yum 源安装

https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+for+New+VM+Users

依赖

rpm
curl
wget
pdsh

添加源

1
2
rpm -ivh https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install -y pdsh

centos6.7-mini

系统版本 CentOS-6.7-x86_64-minimal.iso 2 核 4G 100G 硬盘

1
wget -nv https://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
1
yum install ambari-server -y

rpm 包

1
https://162.209.124.204/hortonworks/ambari/centos6/2.x/updates/2.1.1/Updates-ambari-2.1.1/ambari/

准备机器

192.168.100.37
192.168.100.21
192.168.100.22
192.168.100.23
192.168.100.24

192.168.100.37 安装 ambari-server dnsmasq

各机器配置(同步时间等)

1
2
yum install -y ntp
ntpdate 192.168.100.10

修改 hostname 及 IP

1
2
vi /etc/sysconfig/network
vi /etc/sysconfig/network-scripts/ifcfg-eth2

修改 dns 解析

1
2
vi /etc/resolv.conf
nameserver 192.168.100.37
192.168.100.37
1
rpm -ivh https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
1
2
3
yum install -y wget rpm curl wget pdsh

wget -nv https://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
1
2
yum install -y dnsmasq
yum install -y ambari-server

配置 hosts 及 ssh 免密登陆

1
2
3
4
5
6
7
8
9
10
11
12
13
vi /etc/hosts

192.168.100.37 ambari37
192.168.100.36 ambari36
192.168.100.35 kafka35
192.168.100.34 kafka34
192.168.100.33 storm33
192.168.100.32 storm32
192.168.100.31 storm31
192.168.100.21 node21
192.168.100.22 node22
192.168.100.23 node23
192.168.100.24 node24

ssh 免密登陆

1
2
3
4
5
ssh-keygen
ssh-copy-id root@node21
ssh-copy-id root@node22
ssh-copy-id root@node23
ssh-copy-id root@node24

启动 dnsmasq

1
service dnsmasq start
192.168.100.37
1
ambari-server setup -s

自动检查环境 自动下载 jdk-8u77-linux-x64.tar.gz

执行过程

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
[root@ambari37 ~]# ambari-server setup -s
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'enforcing'
Temporarily disabling SELinux
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)?
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1):
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at https://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?
Downloading JDK from https://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u77-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u77-linux-x64.tar.gz
jdk-8u77-linux-x64.tar.gz... 100% (173.0 MB of 173.0 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u77-linux-x64.tar.gz
Installing JDK to /usr/jdk64/
Successfully installed JDK to /usr/jdk64/
Downloading JCE Policy archive from https://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to /var/lib/ambari-server/resources/jce_policy-8.zip

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip
Installing JCE policy...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database: [ OK ]

About to start PostgreSQL
Configuring local database...
Connecting to local database...done.
Configuring PostgreSQL...
Restarting PostgreSQL
Extracting system views...
.........ambari-admin-2.4.1.0.22.jar
..
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@ambari37 ~]#

启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
ambari-server start
[root@ambari37 ~]# ambari-server start
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
Ambari database consistency check finished
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.
[root@ambari37 ~]#

检查

1
2
3
4
5
6
[root@ambari37 ~]# ambari-server status
Using python /usr/bin/python
Ambari-server status
Ambari Server running
Found Ambari Server PID: 11599 at: /var/run/ambari-server/ambari-server.pid
[root@ambari37 ~]#

192.168.100.37:8080

离线 yum 安装

HDP 2.4.2

HDP 2.5.0

安装本地源

https://my.oschina.net/u/1461927/blog/372147

1
2
wget -nv https://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
wget -nv https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.5.0.0/hdp.repo -O /etc/yum.repos.d/hdp.repo

下载链接

1
2
3
4
5
6
https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.5.0.0/HDP-2.5.0.0-centos6-rpm.tar.gz
https://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0/ambari-2.4.1.0-centos6.tar.gz
https://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6/HDP-UTILS-1.1.0.21-centos6.tar.gz

下面链接可以不用下载
https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.5.0.0/HDP-2.5.0.0-centos6-tars-tarball.tar.gz

将 ambari-2.4.1.0-centos6\AMBARI-2.4.1.0\centos6\2.4.1.0-22 目录中的文件复制到 /var/www/html/ambari/centos6/Updates-ambari-2.4.1.0

将 HDP-2.5.0.0-centos6-rpm\HDP\centos6 目录中的文件复制到 /var/www/html/hdp/centos6/HDP-2.5.0.0

将 HDP-UTILS-1.1.0.21-centos6\HDP-UTILS-1.1.0.21\repos\centos6 目录中的文件复制到 /var/www/html/hdp/centos6/HDP-UTILS-1.1.0.21

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
yum install httpd
yum install yum-utils createrepo
mkdir -p /var/www/html
cd /var/www/html/
mkdir -p ambari/centos6
cd ambari/centos6/
reposync -r Updates-ambari-2.4.1.0
cd /var/www/html/
mkdir -p hdp/centos6
cd hdp/centos6
reposync -r HDP-2.5.0.0
reposync -r HDP-UTILS-1.1.0.21

createrepo /var/www/html/ambari/centos6/Updates-ambari-2.4.1.0
createrepo /var/www/html/hdp/centos6/HDP-2.5.0.0
createrepo /var/www/html/hdp/centos6/HDP-UTILS-1.1.0.21

搭建完成本地 repo 源后修改/etc/yum.repo.d/ambari.repo 和/etc/yum.repo.d/hdp.repo,将链接地址修改

HDP-2.4.2.0 链接

1
2
3
https://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari-2.2.2.0-centos6.tar.gz
https://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz
https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/HDP-2.4.2.0-centos6-rpm.tar.gz

删除 ambari

https://community.hortonworks.com/questions/1110/how-to-completely-remove-uninstall-ambari-and-hdp.html

服务端重置

1
2
3
4
ambari-server stop
ambari-server reset

ambari-server setup (重新设置server,可以不做)

客户端删除

清除各目录

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
service ambari-agent stop
python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent

yum erase -y ambari-agent hdp-select
rpm -qa|grep ambari |xargs yum erase -y

rm -rf /var/lib/ambari-agent

rm -rf /var/lib/ambari-server

rm -rf /var/run/ambari-agent

rm -rf /usr/lib/amrbari-agent

rm -rf /etc/ambari-agent

rm -rf /var/log/ambari-agent

rm -rf /usr/lib/python2.6/site-packages/ambari*

rm -rf /etc/hadoop/

rm -rf /etc/storm/

rm -rf /usr/hdp/

重启机器

1
reboot

说明

1
2
3
4
5
6
python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent --skip=users

Note:
To clean up in interactive mode, remove --silent option.
To clean up all resources, including users, remove --skip=users option.
Use --help for a list of available options.

删除 rpm 包

1
2
rpm -qa|grep ambari |xargs yum erase -y
yum -y erase hdp-select

编译安装

document

https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.4.1

https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development

环境

jdk 1.7.0_80

python 2.6.6

maven 3.3.9

1
2
3
4
5
6
7
[root@ambari36 ~]# mvn -version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /usr/local/maven
Java version: 1.7.0_80, vendor: Oracle Corporation
Java home: /usr/local/java/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64", family: "unix"

安装 g++ gcc

1
yum -y install gcc gcc-c++ autoconf automake libtool make

安装 rpm-build

1
yum install rpm-build

安装 Node.js npm 0.10.44

https://nodejs.org/download/release/v0.10.44/

1
wget https://nodejs.org/dist/v0.10.44/node-v0.10.44-linux-x64.tar.gz
1
2
3
4
5
6
vi /etc/profile
export JAVA_HOME=/usr/local/java
export MAVEN_HOME=/usr/local/maven
export NODE_HOME=/usr/local/node
export PATH=$JAVA_HOME/bin:$NODE_HOME/bin:$PATH:$MAVEN_HOME/bin
source /etc/profile

安装 brunch

1
npm install -g [email protected]

安装 git

1
yum install -y git

下载编译

1
2
3
4
5
6
7
8
wget https://www.apache.org/dist/ambari/ambari-2.4.1/apache-ambari-2.4.1-src.tar.gz
tar xfvz apache-ambari-2.4.1-src.tar.gz
cd apache-ambari-2.4.1-src
mvn versions:set -DneVersion=2.4.1.0.0

pushd ambari-metrics
mvn versions:set -DnewVersion=2.4.1.0.0
popd
1
mvn -B clean install package rpm:rpm -DnewVersion=2.4.1.0.0 -DskipTests -Dpython.ver="python >= 2.6"

error

编译中由于网络原因 总是失败

1

1
[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.0.1:rpm (default-cli) on project ambari: RPM build execution returned: '127' executing '/bin/sh -c cd /root/apache-ambari-2.4.1-src/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /root/apache-ambari-2.4.1-src/target/rpm/ambari/buildroot --define '_topdir /root/apache-ambari-2.4.1-src/target/rpm/ambari' --target noarch-redhat-linux ambari.spec' -> [Help 1]

rpmbuild 命令未找到

1
yum install rpm-build

2

1
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (compile-npm) on project ambari-web: Command execution failed. Cannot run program "npm" (in directory "/root/apache-ambari-2.4.1-src/ambari-web"): error=2, 没有那个文件或目录 -> [Help 1]

npm 命令不存在

安装 node.js npm

3

1
wget https://nodejs.org/dist/v0.10.44/node-v0.10.44-linux-x64.tar.gz
1
wget https://github.com/Medium/phantomjs/releases/download/v1.9.19/phantomjs-1.9.8-linux-x86_64.tar.bz2

phantomjs error

1
2
3
4
5
6
7
8
9
[root@ambari36 phantomjs-1.9.8-linux-x86_64]# bin/phantomjs
bin/phantomjs: error while loading shared libraries: libfreetype.so.6: cannot open shared object file: No such file or directory
解决
yum install freetype

[root@ambari36 phantomjs-1.9.8-linux-x86_64]# bin/phantomjs
bin/phantomjs: error while loading shared libraries: libfontconfig.so.1: cannot open shared object file: No such file or directory
解决
yum install fontconfig freetype libfreetype.so.6 libfontconfig.so.1 libstdc++.so.6