Executing YUM Commands Against 3rd Party YUM Repository on Oracle Big Data Appliance

Asset ID:	1-79-1618420.1
Update Date:	2016-10-11
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 1618420.1 : Executing YUM Commands Against 3rd Party YUM Repository on Oracle Big Data Appliance

Applies to:

Big Data Appliance Integrated Software - Version 2.1.1 and later
Big Data Appliance Hardware - Version All Versions and later
Linux x86-64

Purpose

This document provides steps on how to execute YUM (Yellowdog Updater Modified) commands using dcli against 3rd party yum repository on Oracle Big Data Appliance (BDA).

Scope

Oracle Big Data Appliance Administrators, Oracle Advanced Customer Support (ACS), etc...

Details

Overview

Execute below steps steps on node01 of the BDA cluster as the root user.

Ensure that passwordless-ssh for root is setup on node1 of cluster by running:

# dcli -C hostname

If it is not setup, set it up by executing the following command:

# setup-root-ssh -C

Setup Steps

1) By default httpd service on node 1 is stopped after Mammoth install/upgrade is completed. Start httpd service on node 1

a) Check the status of the httpd service on node01

# service httpd status

httpd is stopped

b) Start httpd service on node01

# service httpd start

Starting httpd: [ OK ]

# service httpd status

httpd (pid 13543) is running...

2) Copy the needed repo files into /tmp directory of node01.

Sample repo files used in this document are:

a) Fedora EPEL (Extra Packages for Enterprise Linux) epel.repo file which can be generated using http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm

[epel]
name=Extra Packages for Enterprise Linux 5 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/5/$basearch
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-5&arch=$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL

b) Cloudera cloudera-gplextras4.repo which has been taken from http://archive.cloudera.com/gplextras/redhat/5/x86_64/gplextras/cloudera-gplextras4.repo

[cloudera-gplextras4]
# Packages for Cloudera's Distribution for Hadoop, Version 4, on RedHat or CentOS 5 x86_64
name=Cloudera's Distribution for Hadoop, Version 4
baseurl=http://archive.cloudera.com/gplextras/redhat/5/x86_64/gplextras/4/
gpgkey = http://archive.cloudera.com/gplextras/redhat/5/x86_64/gplextras/RPM-GPG-KEY-cloudera
gpgcheck = 1

Note:- The sample repo files provided in this document are for OEL 5 OS. If OS on BDA is OEL 6 then appropriate repo file needs to be downloaded.

For example:

Cloudera OEL6 gplextras4 repo file would be at http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/cloudera-gplextras4.repo and

Fedora repo file for OEL6 can be generated using http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

c)Get the public-yum-ol6.repo or public-yum-el5.repo

In order to access external site / url from BDA, http_proxy/https_proxy environment variables need to be set.

export http_proxy=http://<proxy-server>:<port>
export https_proxy=http://<proxy-server>:<port>

For OL6

cd /etc/yum.repos.d
wget http://public-yum.oracle.com/public-yum-ol6.repo

For OL5

cd /etc/yum.repos.d
wget http://public-yum.oracle.com/public-yum-el5.repo

3) Use dcli to copy the repo file to all nodes on the BDA cluster into /etc/yum.repos.d directory.

a) Check what repo files are currently in /etc/yum.repos.d directory. By default there should only be bda.repo file:

dcli -C ls -l /etc/yum.repos.d

Sample output:

<node-private-ip>: total 4
<node-private-ip>: -rw-r--r-- 1 root root 85 Jan 15 14:44 bda.repo

b) Copy repo files to all nodes in the BDA cluster using dcli with -f and -d options.

For example:

dcli -C -f /tmp/cloudera-gplextras4.repo -d /etc/yum.repos.d/cloudera-gplextras4.repo
dcli -C -f /tmp/epel.repo -d /etc/yum.repos.d/epel.repo
dcli -C -f /etc/yum.repos.d/public-yum-ol6.repo -d /etc/yum.repos.d/public-yum-ol6.repo

c) Check /etc/yum.repos.d directory to ensure cloudera-gplextras4.repo file is copied to all nodes

# dcli -C ls -l /etc/yum.repos.d

<node1-private-ip>: total 16
<node1-private-ip>: -rw-r--r-- 1 root root 85 Jan 9 10:42 bda.repo
<node1-private-ip>: -rw-r--r-- 1 root root 347 Jan 30 11:53 cloudera-gplextras4.repo
<node1-private-ip>: -rw-r--r-- 1 root root 302 Jan 30 11:53 epel.repo
<node1-private-ip>: -rw-r--r-- 1 root root 6373 Mar 3 12:55 public-yum-ol6.repo
.............
<lastnode-private-ip>: total 16
<lastnode-private-ip>: -rw-r--r-- 1 root root 85 Jan 9 10:42 bda.repo
<lastnode-private-ip>: -rw-r--r-- 1 root root 347 Jan 30 11:53 cloudera-gplextras4.repo
<lastnode-private-ip>: -rw-r--r-- 1 root root 302 Jan 30 11:53 epel.repo
<lastnode-private-ip>: -rw-r--r-- 1 root root 6373 Mar 3 12:55 public-yum-ol6.repo

d) Clear yum cache on all nodes

# dcli -C yum clean all

Repository 'bda' is missing name in configuration, using id
.........
<node-private-ip>: Loaded plugins: rhnplugin, security
<node-private-ip>: Cleaning up Everything
..............

4) Execute yum command with dcli to install/upgrade/remove rpm's on all nodes in the BDA cluster.

a) In order to access external site / url from BDA, http_proxy/https_proxy environment variables need to be set.

export http_proxy=http://<proxy-server>:<port>
export https_proxy=http://<proxy-server>:<port>

b) When there are multiple repo files in /etc/yum.repos.d directory then --disablerepo and --enablerepo options can be used with the yum command to ensure the rpms are downloaded from the needed / correct repository.

--enablerepo=<Comma separated needed repo names> enable one or more repositories (wildcards allowed)
--disablerepo=<Comma separated NOT needed repo names> disable one or more repositories (wildcards allowed)

The 'yum --help' command can be used to explore available options.

The repolist command can be used to check if the correct repository is being accessed.

For example if the rpm need to be downloaded from Fedora EPEL then yum command would be:

export http_proxy=http://<proxy-server>:<port>
yum --disablerepo=cloudera-gplextras4,bda --enablerepo=epel repolist

Sample output

Loaded plugins: rhnplugin, security
Repository 'bda' is missing name in configuration, using id
This system is not registered with ULN.
ULN support will be disabled.
epel                                                                                                             | 3.6 kB     00:00
epel/primary_db                                                                                                | 3.9 MB     00:02
repo id                                     repo name                                                                           status
epel                                        Extra Packages for Enterprise Linux 5 - x86_64                          7,594
repolist: 7,594

Below dcli command can be used to check on all nodes in the cluster:

dcli -C "export http_proxy=http://www-proxy.us.oracle.com:80; yum -y --disablerepo=cloudera-gplextras4,bda --enablerepo=epel repolist"

c) yum command to install, upgrade or remove an rpm

With yum command pass -y option so no user interaction is needed during execution of yum command using dcli. More details about executing yum command using dcli can be found in Doc ID 1615029.1

Syntax

dcli -C "export http_proxy=http://<proxy-server>:<port>; yum -y --disablerepo=<comma separated NOT needed repo names> --enablerepo=<3rd party repo > install <rpm/package name>"

dcli -C "export http_proxy=http://<proxy-server>:<port>; yum -y --disablerepo=<comma separated NOT needed repo names> --enablerepo=<3rd party repo > upgrade <rpm/package name>"

dcli -C "export http_proxy=http://<proxy-server>:<port>; yum -y --disablerepo=<comma separated NOT needed repo names> --enablerepo=<3rd party repo > remove <rpm/package name>"

Sample Commands:

dcli -C "export http_proxy=http://www-proxy.us.oracle.com:80; yum -y --disablerepo=cloudera-gplextras4,bda --enablerepo=epel install lzo-2.02-2.el5.1.x86_64"

dcli -C "export http_proxy=http://www-proxy.us.oracle.com:80; yum -y --disablerepo=bda,epel --enablerepo=cloudera-gplextras4 install hadoop-lzo-cdh4"

dcli -C "export http_proxy=http://www-proxy.us.oracle.com:80; yum -y --disablerepo=bda,epel --enablerepo=cloudera-gplextras4 install impala-lzo"

Sample Output of installing rpm from Cloudera repository:

Repository 'bda' is missing name in configuration, using id
..........
warning: rpmts_HdrFromFdno: Header V4 DSA/SHA1 Signature, key ID e8f86acd: NOKEY
Importing GPG key 0xE8F86ACD:
Userid: "Yum Maintainer <webmaster@cloudera.com>"
From : http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/RPM-GPG-KEY-cloudera
..................
<node-private-ip>: Loaded plugins: aliases, changelog, downloadonly, filter-data, keys, list-data,
<node-private-ip>: : ps, security, verify
<node-private-ip>: Skipping filters plugin, no data
<node-private-ip>: Setting up Install Process
<node-private-ip>: Resolving Dependencies
<node-private-ip>: Skipping filters plugin, no data
<node-private-ip>: --> Running transaction check
<node-private-ip>: ---> Package hadoop-lzo-cdh4.x86_64 0:0.4.15-1.gplextras.p0.152.el6 will be installed
<node-private-ip>: --> Finished Dependency Resolution
<node-private-ip>:
<node-private-ip>: Dependencies Resolved
<node-private-ip>:
<node-private-ip>: ================================================================================
<node-private-ip>: Package         Arch   Version                       Repository           Size
<node-private-ip>: ================================================================================
<node-private-ip>: Installing:
<node-private-ip>: hadoop-lzo-cdh4 x86_64 0.4.15-1.gplextras.p0.152.el6 cloudera-gplextras4 80 k
<node-private-ip>:
<node-private-ip>: Transaction Summary
<node-private-ip>: ================================================================================
<node-private-ip>: Install       1 Package(s)
<node-private-ip>:
<node-private-ip>: Total download size: 80 k
<node-private-ip>: Installed size: 173 k
<node-private-ip>: Downloading Packages:
<node-private-ip>: Retrieving key from http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/RPM-GPG-KEY-cloudera
<node-private-ip>: Running rpm_check_debug
<node-private-ip>: Running Transaction Test
<node-private-ip>: Transaction Test Succeeded
<node-private-ip>: Running Transaction
<node-private-ip>: Installing : hadoop-lzo-cdh4-0.4.15-1.gplextras.p0.152.el6.x86_64         1/1
<node-private-ip>: Verifying : hadoop-lzo-cdh4-0.4.15-1.gplextras.p0.152.el6.x86_64         1/1
<node-private-ip>:
<node-private-ip>: Installed:
<node-private-ip>: hadoop-lzo-cdh4.x86_64 0:0.4.15-1.gplextras.p0.152.el6
<node-private-ip>:
<node-private-ip>: Complete!
..........

Sample output of installing rpm from Fedora EPEL repository:

Repository 'bda' is missing name in configuration, using id
This system is not registered with ULN.
ULN support will be disabled.
..........
<node-private-ip>: Loaded plugins: rhnplugin, security
<node-private-ip>: Setting up Install Process
<node-private-ip>: Resolving Dependencies
<node-private-ip>: --> Running transaction check
<node-private-ip>: ---> Package lzo.x86_64 0:2.02-2.el5.1 set to be updated
<node-private-ip>: --> Finished Dependency Resolution
<node-private-ip>:
<node-private-ip>: Dependencies Resolved
<node-private-ip>:
<node-private-ip>: ================================================================================
<node-private-ip>: Package        Arch              Version                 Repository       Size
<node-private-ip>: ================================================================================
<node-private-ip>: Installing:
<node-private-ip>: lzo            x86_64            2.02-2.el5.1            epel             57 k
<node-private-ip>:
<node-private-ip>: Transaction Summary
<node-private-ip>: ================================================================================
<node-private-ip>: Install       1 Package(s)
<node-private-ip>: Upgrade       0 Package(s)
<node-private-ip>:
<node-private-ip>: Total download size: 57 k
<node-private-ip>: Downloading Packages:
<node-private-ip>: Running rpm_check_debug
<node-private-ip>: Running Transaction Test
<node-private-ip>: Finished Transaction Test
<node-private-ip>: Transaction Test Succeeded
<node-private-ip>: Running Transaction
<node-private-ip>: Installing     : lzo                                                      1/1
<node-private-ip>:
<node-private-ip>: Installed:
<node-private-ip>: lzo.x86_64 0:2.02-2.el5.1
<node-private-ip>:
<node-private-ip>: Complete!
......................

5) Please keep in mind during upgrade of BDA only bda.repo file need to exist in /etc/yum.repos.d directory. So once the needed rpms/packages are installed delete/move the 3rd party rpms from /etc/yum.repos.d directory.

For example below steps can be followed to move 3rd party repo files into /etc/yum.repos.d/myrepo directory:

dcli -C mkdir /etc/yum.repos.d/myrepo
dcli -C ls -ld /etc/yum.repos.d/myrepo
-- Change the repo file names to match yours
dcli -C mv /etc/yum.repos.d/cloudera-gplextras4.repo /etc/yum.repos.d/myrepo/cloudera-gplextras4.repo
dcli -C mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/myrepo/epel.repo
dcli -C ls -l /etc/yum.repos.d

6) Clear yum cache to reflect the removed repository.

# dcli -C yum clean all

Repository 'bda' is missing name in configuration, using id
.........
<node-private-ip>: Loaded plugins: rhnplugin, security
<node-private-ip>: Cleaning up Everything
..............

7) Check if bda is the only repository listed on all the nodes

# dcli -C yum repolist

Repository 'bda' is missing name in configuration, using id
This system is not registered with ULN.
ULN support will be disabled.
...........
<node-private-ip>: Loaded plugins: rhnplugin, security
<node-private-ip>: repo id repo name status
<node-private-ip>: bda bda 1,288
<node-private-ip>: repolist: 1,288 <-- This number might vary between releases
...........

8) Stop httpd service on node01 once the yum operations are completed.

# service httpd stop

Stopping httpd: [ OK ]

# service httpd status

httpd is stopped

Attachments

This solution has no attachment