Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-79-1476008.1
Update Date:2017-12-21
Keywords:

Solution Type  Predictive Self-Healing Sure

Solution  1476008.1 :   Using DCLI on Oracle Big Data Appliance  


Related Items
  • Big Data Appliance X3-2 In-Rack Expansion
  •  
  • Big Data Appliance Hardware
  •  
  • Big Data Appliance X3-2 Full Rack
  •  
  • Big Data Appliance X3-2 Hardware
  •  
  • Big Data Appliance X3-2 Starter Rack
  •  
Related Categories
  • PLA-Support>Eng Systems>BDA>Big Data Appliance>DB: BDA_EST
  •  




In this Document
Purpose
Scope
Details


Applies to:

Big Data Appliance Hardware - Version All Versions to All Versions [Release All Releases]
Big Data Appliance X3-2 Hardware - Version All Versions to All Versions [Release All Releases]
Big Data Appliance X3-2 Full Rack - Version All Versions to All Versions [Release All Releases]
Big Data Appliance X3-2 Starter Rack - Version All Versions to All Versions [Release All Releases]
Big Data Appliance X3-2 In-Rack Expansion - Version All Versions to All Versions [Release All Releases]
Linux x86-64

Purpose

 This document will describe the dcli utility and how it is used with Oracle Big Data Appliance.

Scope

This document is intended for System administrators of the Oracle Big Data Appliance.

Details

DCLI is a distibuted command line interface utility. The dcli utility allows you to execute commands across a group of servers on Oracle Big Data Appliance and returns the output. The dcli utility allows you to run commands across multiple Oracle Big Data Appliance Servers in parallel using the Infiniband(bondib0) interface to make the connections. The dcli utility may be run from any server.

Prerequisites:

The dcli utility requires a passwordless Secure Shell (SSH) between the local server and all target servers. The dcli utility is run on the local server, and the commands specified in dcli execute on the target servers.

Two scripts facilitate the use of SSH on Oracle Big Data Appliance: setup-root-ssh and remove-root-ssh. These scripts accept two options that are also used by dcli:

    -C: Targets all the servers in a Hadoop cluster

    -g: Targets a user-defined set of servers  
NOTE: By default if -C or -g or -c options are not specified, the command will run on all servers in the rack (as specified in /opt/oracle/bda/rack-hosts-infiniband). 

To set up passwordless SSH for root simply follow these steps:

1.  Connect to an Oracle Big Data Appliance server using PuTTY or a similar utility. Select an SSH connection type.

2.  Log in as root.

3.  Set up passwordless SSH for root across the rack:

    setup-root-ssh

    Or, to set up passwordless SSH across a Hadoop cluster of multiple racks:

    setup-root-ssh -C

    The message "ssh key added" from each server will be seen.

4.  Any ssh command command can now be run on any server in the rack without entering a password. In addition to dcli commands, scp can be used to copy files between servers.

5.  To remove passwordless SSH from root simply issue the following:

     remove-root-ssh

The command setup-root-ssh -j eth0_ips is the default and equal to setup-root-ssh.

 

More information on the options which can be used with the dcli utility can be obtained by issuing the help command by the -h option or --help option. Here is the output from issuing the dcli -h option:

dcli -h

Distributed command line interface

This script executes commands on multiple nodes in parallel threads.
The nodes are referenced by their domain name or ip address.
Local files can be copied to nodes and executed on nodes.
This tool does not support interactive sessions with host applications.
Use of this tool assumes ssh is running on local host and nodes.
The -k option should be used initially to perform key exchange with nodes.  User may be prompted to acknowledge node authenticity, and may be prompted for the remote user password.  This -k step is serialized
to prevent overlayed prompts.  After -k option is used once, then subsequent commands to the same nodes do not require -k and will not require passwords for that user from the host.
Command output (stdout and stderr) is collected and displayed after the copy and command execution has finished on all nodes.
Options allow this command output to be abbreviated.

Return values:
 0 -- file or command was copied and executed successfully on all nodes
 1 -- one or more nodes could not be reached or remote execution returned non-zero status.
 2 -- An error prevented any command execution

Examples:
 dcli -g nodes -k
 dcli -c stsd2s2,stsd2s3 vmstat
 dcli -g nodes "ls -l"
 dcli -g nodes -x myscript.sh

usage: dcli [options] [command]

OptionDescription
--version Show program's version number and exit
-C Use the cluster hosts file /opt/oracle/bda/cluster-hosts-infiniband
-c NODES Comma-separated list of nodes
-j JSONEXPR expr in json-select --jxp=expr --bash --deploy
-d DESTFILE Destination directory or file
-f FILE File to be copied
-g GROUPFILE File containing list of nodes
-h, --help Show help message and exit
-k Push ssh key to node's authorized_keys file
-l USERID User to login as on remote nodes (default: root)
--maxlines=MAXLINES Limit output lines from a node when in parallel execution over multiple nodes (default: 100000)
-n Abbreviate non-error output
-r REGEXP Abbreviate output lines matching a regular expression
-s SSHOPTIONS String of options passed through to ssh
--scp=SCPOPTIONS String of options passed through to scp if different from sshoptions
--serial Serialize execution over the nodes
-t List target nodes
--unkey Drop keys from target nodes' authorized_keys file
-v Print extra messages to stdout
--vmstat=VMSTATOPS vmstat command options
-x EXECFILE File to be copied and executed

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Examples of Using the DCLI Utility to Issue Commands

Display the dcli version:

# dcli --version
version 1.0

Display the date on the servers:

# dcli date

1xx.xxx.x.1: Tue Jul 17 12:06:39 PDT 2012
1xx.xxx.x.2: Tue Jul 17 12:06:39 PDT 2012
1xx.xxx.x.3: Tue Jul 17 12:06:39 PDT 2012

...

...

1xx.xxx.x.18: Tue July 17 12:06:39 PDT 2012

List the bondib0_ips and hostnames for the nodes in the rack:

#  dcli -j bondib0_ips hostname
192.168.10.1: bda1node01.<domain>
192.168.10.2: bda1node02.<domain>
192.168.10.3: bda1node03.<domain>
192.168.10.4: bda1node04.<domain>
192.168.10.5: bda1node05.<domain>
192.168.10.6: bda1node06.<domain>
192.168.10.7: bda1node07.<domain>
192.168.10.8: bda1node08.<domain>
192.168.10.9: bda1node09.<domain>
192.168.10.10: bda1node10.<domain>
192.168.10.11: bda1node11.<domain>
192.168.10.12: bda1node12.<domain>
192.168.10.13: bda1node13.<domain>
192.168.10.14: bda1node14.<domain>
192.168.10.15: bda1node15.<domain>
192.168.10.16: bda1node16.<domain>
192.168.10.17: bda1node17.<domain>
192.168.10.18: bda1node18.<domain>

The dcli command uses eth0_ips (admin network, not the iloms) by default, thus dcli is equal to dcli -j eth0_ips. Mammoth sets up a node file consisting of the bondib0_ip's of the cluster that will be used when using dcli -C.
There is a difference here that -C is  cluster wide, whereas dcli -j bondib0_ips is rack wide. Note that it is also possible to use sub scripting to target a subset of nodes like in dcli -j "bondib0_ips[1,2,3]" and  dcli -j "bondib0_ips[1:5,7,9,11:18]"

List the target nodes:

# dcli -t
Target nodes: [1xx.xxx.x.1, 1xx.xxx.x.2, 1xx.xxx.x.3, ..., 1xx.xxx.x.18]

Display the hostname for all the servers in the cluster:

# dcli -C hostname
1xx.xxx.x.1: bda1node01.example.com
1xx.xxx.x.2: bda1node02.example.com
1xx.xxx.x.3: bda1node03.example.com

...

...

1xx.xxx.x.18: bda1node18.example.com

Display the Big Data Appliance Image Info for all the servers on the cluster:

# dcli imageinfo
1xx.xxx.x.1: Big Data Appliance Image Info
1xx.xxx.x.1:
1xx.xxx.x.1: IMAGE_VERSION             : 1.0.3
1xx.xxx.x.1: IMAGE_CREATION_DATE       : Thu Jun 21 14:55:53 PDT 2012
1xx.xxx.x.1: IMAGE_LABEL               : BDA_1.0.3_LINUX.X64_RELEASE
1xx.xxx.x.1: KERNEL_VERSION            : 2.6.32-200.21.1.el5uek
1xx.xxx.x.1: BDA_RPM_VERSION           : bda-1.0.3-1
1xx.xxx.x.1: OFA_RPM_VERSION           : ofa-2.6.32-200.21.1.el5uek-1.5.5-4.0.55.4
1xx.xxx.x.1: JDK_VERSION               : jdk-1.6.0_29-fcs
1xx.xxx.x.2: Big Data Appliance Image Info
1xx.xxx.x.2:
1xx.xxx.x.2: IMAGE_VERSION             : 1.0.3
1xx.xxx.x.2: IMAGE_CREATION_DATE       : Thu Jun 21 14:55:53 PDT 2012
1xx.xxx.x.2: IMAGE_LABEL               : BDA_1.0.3_LINUX.X64_RELEASE
1xx.xxx.x.2: KERNEL_VERSION            : 2.6.32-200.21.1.el5uek
1xx.xxx.x.2: BDA_RPM_VERSION           : bda-1.0.3-1
1xx.xxx.x.2: OFA_RPM_VERSION           : ofa-2.6.32-200.21.1.el5uek-1.5.5-4.0.55.4
1xx.xxx.x.2: JDK_VERSION               : jdk-1.6.0_29-fcs

...

...

1xx.xxx.x.18: IMAGE_VERSION             : 1.0.3
1xx.xxx.x.18: IMAGE_CREATION_DATE       : Thu Jun 21 14:55:53 PDT 2012
1xx.xxx.x.18: IMAGE_LABEL               : BDA_1.0.3_LINUX.X64_RELEASE
1xx.xxx.x.18: KERNEL_VERSION            : 2.6.32-200.21.1.el5uek
1xx.xxx.x.18: BDA_RPM_VERSION           : bda-1.0.3-1
1xx.xxx.x.18: OFA_RPM_VERSION           : ofa-2.6.32-200.21.1.el5uek-1.5.5-4.0.55.4
1xx.xxx.x.18: JDK_VERSION               : jdk-1.6.0_29-fcs

Verify the Infininband ports are up, two on each server (36 total):

# dcli ibstatus | grep phys
1xx.xxx.x.1: phys state:        5: LinkUp
1xx.xxx.x.1: phys state:        5: LinkUp
1xx.xxx.x.2: phys state:        5: LinkUp
1xx.xxx.x.2: phys state:        5: LinkUp
1xx.xxx.x.3: phys state:        5: LinkUp
1xx.xxx.x.3: phys state:        5: LinkUp

...

...

1xx.xxx.x.18: phys state:        5: LinkUp
1xx.xxx.x.18: phys state:        5: LinkUp

Verify that the InfiniBand ports are running at 40 Gbps (4X QDR):

# dcli ibstatus | grep rate
1xx.xxx.x.1: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.1: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.2: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.2: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.3: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.3: rate:              40 Gb/sec (4X QDR)

...

...

1xx.xxx.x.18: rate:              40 Gb/sec (4X QDR)
1xx.xxx.x.18: rate:              40 Gb/sec (4X QDR)

Verify that Oracle ILOM does not detect any faults:

# dcli 'ipmitool sunoem cli "show faulty"'

The output should appear as follows for each server:

bda1node02.example.com: Connected. Use ^D to exit.
bda1node02.example.com: -> show faulty
bda1node02.example.com: Target      | Property        | Value
bda1node02.example.com:-------------+---------------------+-----------
bda1node02.example.com:
bda1node02.example.com: -> Session closed
bda1node02.example.com: Disconnected

Copy a file to all target servers:

# dcli -f /root/package.rpm -d /tmp/package.rpm

Copy and execute a local shell script on all target servers:

# dcli -x myscript.sh

 


Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback