Oracle Database - Enterprise Edition
 
Oracle Database - Enterprise Edition
 
Exadata X3-2 Full Rack
 
Exadata Database Machine X2-2 Full Rack
 
Oracle Net Services
 
Exadata Database Machine X2-8
 

PLA-Support>Database>DB Foundation>DB Networking>DB: NET Core
 
_Old GCS Categories>ST>Network>ConnectivityConcepts>Architecture
 

How to configure the Database, RAC, and Exadata to work from behind a Network Address Translator (NAT)

In this Document

Purpose

Scope

Details

The Transparent Network Substrate (TNS) Connection and NAT

Telling the Listener what to send

Default TNS Redirect

For a default TNS Redirect packet (NSPTRD) the IP address is used to redirect the Client.  To see this, client-side Oracle Net tracing was enabled to level SUPPORT.  The portion of the trace which shows the contents of the TNS Redirect packet is this:

Setting LOCAL_LISTENER to FQDN

SCAN and NAT

Other Solutions

Use NAT/Firewall SQL*Net module (when NAT is employed)

Use Oracle Connection Manager (CMAN)

References

Applies to:

Purpose

In order to expand the IPv4 address space, a few techniques were employed, one of which was the Network Address Translator.  These devices would present to the Internet a specific set if Internet Protocol (IP) addresses that would represent a larger group of addresses.  The addresses used on the non-Internet side of a NAT come from the Private Network address ranges:

Since these address ranges can be used in multiple locations, to properly route between subnets with identical subnet numbers, the NAT devices would provide the mapping necessary to assure separation.

In its default configuration, how the Oracle Listener responds to a connection request may not be compatible with NAT.  This note addresses ways to configure the Database, Real Application Cluster (RAC) and ExaData systems so they can properly message the client to compensate for NAT address translations.  It is assumed that the reader is familiar with NAT devices and IP address translations.  It also is assumed that the NAT device works only on the IP layer and does not perform deep packet modifications.

Scope

 Database Administrators, Network Administrators.

Details

The Transparent Network Substrate (TNS) Connection and NAT

The TNS connection is a part of the Session Layer TNS protocol.  In the steps of creating a server process that handles the login of the client, the connection setup follows the Open Systems Interconnection (OSI) layers.  The client first connects to the Listener using the TCP three-way handshake.  When a NAT device is used, it stands in the middle of this handshake and translates the destination IP address the client used to the actual destination IP address of the server inside the NAT controlled network. The NAT plays a very active role in the communication as each and every packet needs to have both source and destination IP addresses changed according to the address mapping configured in the NAT.

Once the TCP connection has been established, the client passes a TNS Connect packet, through the NAT, to the Listener process.  This Listener will either send a TNS Resend packet or a TNS Redirect packet back to the client through the NAT.  It is the TNS Redirect packet that creates the problem when using a NAT.  The Listener will send the TNS Redirect packet if the Database is configured for Shared Server or if this is a RAC based system (which includes ExaData).

By default, the TNS Redirect packet contains the IP address of the host along with the port number of the process the client is to be redirected to.  This process can either be a Dispatcher or another Listener.  In the case of RAC the first Listener the client connects to is the Single Client Access Name (SCAN) Listener and it redirects the client to a VIP Listener on the least loaded node.  This IP address is contained inside the TCP data section of the packet and is not part of the IP header.  The NAT will not know what is in the TNS Redirect packet so it won't modify that part of the packet.

The client receives the TNS Redirect packet and closes down the TCP connection, so to the NAT this connection is closed.  The client now takes the IP address and the port number that was contained in the TNS Redirect and initiates a TCP connection using that IP address.  If the database host's IP address is returned, the client will not connect as it does not have a route to that IP address (or the route leads to a different host), so the connection will fail with a TNS connect error.  The Listener must be told to return something that the client can correctly resolve to the NAT address assigned to the Database.

Telling the Listener what to send

The Listener gets the redirect information from the instance’s Process Monitor (PMON).  PMON periodically updates the various listeners through the Automatic Registration process.  Automatic Registration is controlled by three parameters inside each instance: LOCAL_LISTENER, REMOTE_LISTENER, and LISTENER_NETWORKS.  The parameter to configure to resolve NAT issues is the LOCAL_LISTENER parameter.  The LOCAL_LISTENER parameter is used to tell PMON which Listeners are running on that host and are responsible to create the final connection.  The REMOTE_LISTENER parameter is used by PMON so it knows which Listeners are responsible to sending the redirect packet.  PMON tells all the Remote Listeners about the Local Listener by passing the information in the LOCAL_LISTENER parameter.

It will not be possible to have PMON "lie" using the LOCAL_LISTENER parameter by using the NAT address.  This parameter must be configured so PMON can resolve the host and locally connect to update the Local Listener as to the status of the instance.  This parameter must also be set to something that the client can resolve to the NAT address assigned.

The default setting of the LOCAL_LISTENER parameter is to use the IP address of the VIP or host.  If the client was given a host name instead, it could then resolve that name into an IP address.  When a client resolves a just a host name, it assumes that host is in the same domain as it is.  But if the DNS naming is different, it would be more useful to have the client be redirected to the Fully Qualified Domain Name (FQDN).  Once the FQDN of the Database host is properly defined in the client's DNS to point to the NAT, it will be able to connect.  Understanding exactly how the NAT is configured will choose what the LOCAL_LISTENER parameter should be set to.

In a RAC system, there is the SCAN, the VIP names, and the nodes' unique names.  The SCAN will always be a Remote Listener, so it needs to be properly defined in the NAT as it is the initial connect the client will make.  Let's use FQDN and assume that the domain name of the network behind the NAT is "nat.mydomain.com" and the client's domain is "client.mydomain.com".
 
The names and IP addresses defined both in the NAT and on the database network would be:

In this case, the client will want to connect to scan.nat.mydomain.com and its DNS will say to connect to 10.0.0.1, 10.0.0.2, or 10.0.0.3 (a little more about SCAN and NAT is covered later in this document).  The address of 10.0.0.1 is actually the NAT device which has the mapping of forwarding all requests to 10.0.0.1 to 192.168.1.30.  The SCAN Listener will be listening on 192.168.1.30 and will accept the TCP connect and the TNS Connect packet.  It will send a TNS Redirect packet back to the client with the FQDN of the VIP for node2, or node2-vip.nat.mydomain.com and the client disconnects.  The client will query its DNS for the IP address of node2-vip.nat.mydomain.com and get back 10.0.0.200.  The address of 10.0.0.200 is again the NAT device but it has this address mapped to the 192.168.1.25 address.  So when the client connects to 10.0.0.200, it is forwarded to 192.168.1.25, or to the VIP Listener for node2.

Default TNS Redirect

For a default TNS Redirect packet (NSPTRD) the IP address is used to redirect the Client.  To see this, client-side Oracle Net tracing was enabled to level SUPPORT.  The portion of the trace which shows the contents of the TNS Redirect packet is this:

In the example above, the parameter sent to the client was (HOST=192.168.1.25).  If the client is not in the same network as the database server, the client may get connected to some other host with that IP address.  What it needs to get is the host’s name so a DNS can give it an IP address the NAT will understand.

Setting LOCAL_LISTENER to FQDN

The next example is to show how to get the Remote Listener to send the FQDN of the Local Listener.  The same will hold if just the host name is used as the Listener will just send the string set in the LOCAL_LISTENER parameter.
By default the LOCAL_LISTENER parameter is not set, but PMON will discover the Listener by resolving the host name (or VIP name) and assuming the Listener is on port 1521.  If just the host name of VIP name used, the client will assume the host is in the same domain as it is and query DNS for that name.  In this example it is assumed that the DNS is configured so the client is on one Domain and the Server is in the Domain behind the NAT.  If just the host name is used in this DNS configuration the client connection would fail as it would get the host or VIP name (node2-vip) and assume it was in its domain:

node2-vip.client.mydomain.com

But if it got the FQDN the client will fail to connect because it asks the DNS for:

node2-vip.nat.mydomain.com

To configure the Database for this NAT environment, the LOCAL_LISTENER parameter needs to be set by using the FQDN.  The parameter can be set using the full string with an alter system command, but it is best to use the TNSNAMES.ORA file due to some size limits of parameters.  For RAC, make sure you set a unique TNS Alias for each instance's LOCAL_LISTENER parameter so a single TNSNAMES.ORA file can be transported to each node in the Cluster.  Here is an example of the TNS Alias for the Local Listener:

The next step is to change the LOCAL_LISTENER parameter for each Instance (on all nodes) that is to be accessed by clients not on the NAT network.  This is done using the "ALTER SYSTEM" command:

Now tell that instance to register with all the REMOTE_LISTENER and LOCAL_LISTENER.

To validate the setting is working, enable Oracle Net tracing on the client to level 16 and examine the contents of the file for the TNS Redirect (NSPTRD packet).  Once this works, the initial Listener contacted will send back the FQDN in the TNS Redirect packet:

 
In this example, the Listener sent back the FQDN of cnode2-vip.coe-us.oracle.com.  Had this set of test hosts been in the nat.mydomain.com, they would have sent back node2-vip.nat.mydomain.com.

SCAN and NAT

When setting up the NAT for the SCAN one needs to understand what happens when the SCAN starts and how the client attempts to connect to it.  When the SCAN is first created, it queries the DNS using the FQDN for the SCAN to get the IP addresses associated with the SCAN.  The IP addresses returned are then stored in the Cluster Registry.  Every time the SCAN is started the Cluster will refer to the Cluster Registry for the IP addresses and will bring up the SCAN using these addresses.  For example, the Cluster will ask the DNS it uses for the SCAN name cnode-cluster-scan.nat.mydomain.com and the DNS will respond with the addresses 192.168.1.30, 192.168.1.31, and 192.168.1.32.

The client’s DNS should not associate these IP addresses with the SCAN that is defined on the NAT.  The client’s DNS should have knowledge of the SCAN under the name cnode-cluster-scan.nat.mydomain.com and will send to the client the IP addresses 10.0.0.1, 10.0.0.2, and 10.0.0.3.

Now configure the NAT so that the client-side IP address of 10.0.0.1 maps to 192.168.1.30, 10.0.0.2 maps to 192.168.1.31, and 10.0.0.3 maps to 192.168.1.32.  With this configuration when the NAT gets a TCP connect request to 10.0.0.2 it will convert that connect request to one toward 192.168.1.31.  So configuring the client to connect to the host (HOST= cnode-cluster-scan.nat.mydomain.com) the DNS should return the three NAT defined addresses (10.0.0.1, 10.0.0.2, 10.0.0.3) so when the client connects to one, say 10.0.0.2, the NAT will translate and forward the connection to the SCAN address of 192.168.1.31.

Other Solutions

Use NAT/Firewall SQL*Net module (when NAT is employed)

Many commercial firewall/security vendors have implemented content-based module for the Oracle SQL*Net protocol and will pass through Oracle database connections; some of them also offer support for "fixing up" the NAT addresses in packets at both IP level (layer 3) but also in the SQL*Net payload.  To implement this solution contact your firewall/network administrator and/or your firewall vendor and ask about SQL*Net NAT features.  For more details see and Note 125021.1

Use Oracle Connection Manager (CMAN)

Connection Manager will “absorb” the TNS Redirect packet so the client will never see it.  This is because the client will connect to CMAN instead of to the Database.   CMAN should be on the inside the NAT perimeter.  CMAN is a proxy service between the client and the Database.  For details on how to implement this solution please see Note 2077721.6
Offer the client direct IP level (layer 3) access to the database node(s)

A VPN or other tunneling method can be deploied to have the database nodes directly accessible the database client at network level so listener(s) will REDIRECT to an accessible IP.  Please consult with your firewall/network administrator for security assessment and implementation.

References

This solution has no attachment