| Asset ID: |
1-71-1909282.1 |
| Update Date: | 2017-03-24 |
| Keywords: | |
Solution Type
Technical Instruction Sure
Solution
1909282.1
:
Oracle Big Data Appliance Port Numbers for BDA V3.*/CDH 5.* and Higher Client/Edge Servers
| Related Items |
- Big Data Appliance Integrated Software
- Big Data Appliance Hardware
|
| Related Categories |
- PLA-Support>Eng Systems>BDA>Big Data Appliance>DB: BDA_EST
|
In this Document
Created from <SR 3-9173518464>
Applies to:
Big Data Appliance Hardware - Version All Versions and later
Big Data Appliance Integrated Software - Version 3.0 and later
x86_64
Goal
The goal is to identify the port numbers you will need to be aware of for configuring a BDA V3.*/CDH 5.* client/edge server.
The port numbers you will need to be aware of for CDH 5 can be found in the Cloudera documentation: "Configuring Ports for Cloudera Manager".
A list is also included in the Oracle Big Data Appliance Software User's Guide Release 3 (3.0) Section on: Port Numbers Used on Oracle Big Data Appliance.
Two Cloudera blogs with additional information are:
Migrating to MapReduce 2 on YARN (For Operators) and Migrating to MapReduce 2 on YARN (For Users).
From the blogs additional internal CDH 5.* ports are:
8030 internal ResourceManager Scheduler RPC (for ApplicationMasters)
8033 internal ResourceManager Admin RPC
8031 internal ResourceManager Resource Tracker RPC (for NodeManagers)
8040 internal NodeManager Localizer RPC
10020 internal Job History RPC
13562 internal Shuffle HTTP
The list of ports included in the above documentation includes those for Cloudera Manager, CDH components, managed servcies and other components included with BDA V3.*. Note that generally the same hold for BDA V3.* and higher.
When setting up a CDH client/edge server:
1. The ports for the components you are using must be open.
2. In the case of a firewall, like iptables, the firewall will need to be disabled as needed for the required ports.
Note: These ports are open by default on BDA servers.
Solution
Ports
In the output below an "External" Access Requirement indicates that the port "needs end user access". Otherwise the port is internal to the cluster.
A potential list of ports new to BDA V3.* / CDH 5.* to check when setting up a client/edge server can be found in the table here:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| YARN |
ResourceManager Node 2 |
|
8032 |
TCP |
External |
yarn.
resourcemanager.
address
|
|
| YARN |
ResourceManager Node 3
|
|
8032 |
TCP |
External |
yarn.
resourcemanager.
address
|
|
| YARN |
ResourceManager Node 2 |
|
8088 |
TCP |
External |
yarn.
resourcemanager.
webapp.address
|
|
| YARN |
ResourceManager Node 3 |
|
8088 |
TCP |
External |
yarn.
resourcemanager.
webapp.address
|
|
| YARN |
ResourceManager Node 2 - Secure |
|
8090 |
TCP |
|
|
|
| YARN |
ResourceManager Node 3 - Secure |
|
8090 |
TCP |
|
|
|
| YARN |
JobHistory Server Node 3 |
|
19888 |
TCP |
External |
mapreduce.
jobhistory.
webapp.address
|
|
| YARN |
NodeManager -on Nodes as appropriate |
|
8042 |
TCP |
External |
yarn.
nodemanager.
webapp.address
|
|
| YARN |
NodeManager |
|
8042 |
TCP |
|
yarn.
nodemanager.
webapp.address
|
|
| YARN |
NodeManager |
|
8041 |
TCP |
|
yarn.
nodemanager.
address
|
|
Other ports of potential interest for CDH 5.* client/edge servers are:
Spark related ports are:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| Spark |
Default Master RPC port |
|
7077 |
TCP |
External |
|
|
| Spark |
Default Master web UI port |
|
18080 |
TCP |
External |
|
|
| Spark |
Default Worker web UI port |
|
18081 |
TCP |
|
|
|
HBase related ports are:
| Component | Service | Qualifer | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| HBase |
Master |
|
60010 |
TCP |
External |
hbase.master.
info.port
|
HTTP |
| HBase |
RegionServer |
|
60030 |
TCP |
External |
hbase.
regionserver.
info.port
|
HTTP |
| HBase |
REST UI |
|
8085 |
TCP |
External |
|
|
| HBase |
ThriftServer |
|
9095 |
TCP |
External |
|
|
Ports for Solr:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| Cloudera Search |
Solr search/update |
|
8983 |
HTTP |
External |
|
All Solr-specific actions, update/query. |
Ports for Oozie:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| Oozie |
Oozie Server |
|
11000 |
TCP |
External |
OOZIE_HTTP_
PORT in oozie-env.sh
|
HTTP |
Ports for Impala:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| Impala StateStore Daemon |
StateStore HTTP Server Port |
External |
25010 |
|
External |
|
StateStore web interface for administrators to monitor and troubleshoot. |
| Impala Catalog Daemon |
Catalog HTTP Server Port |
External |
25020 |
|
External |
|
Catalog service web interface for administrators to monitor and troubleshoot. New in Impala 1.2 and higher. |
Other HDFS ports:
| Component | Service | Qualifier | Port | Protocol | Access Requirement | Configuration | Comment |
|---|
| HDFS |
NameNode |
Secure |
50470 |
TCP |
External |
dfs.https.
address or
dfs.namenode.
https-address
|
dfs.https.
address
is deprecated (but still works) |
| HDFS |
DataNode |
Secure |
1004 |
TCP |
External |
|
|
How to check if ports are open
You can check if a port is open using lsof or netstat as follows:
1. With 'lsof -i :<port#>. For example:
# lsof -i :<port#>
# lsof -i | grep 8040
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 13176 yarn 488u IPv4 51306442 0t0 TCP bdanode04.example.com:8040 (LISTEN)
2. With '# netstat -anp | grep <port#>'. For example:
# netstat -anp | grep <port#>
# netstat -anp | grep 8040
tcp 0 0 192.168.42.104:8040 0.0.0.0:* LISTEN 13176/java
Attachments
This solution has no attachment
