Private Cloud Appliance
 
Private Cloud Appliance X5-2 Hardware
 

PLA-Support>Eng Systems>Exalogic/OVCA>Oracle Virtual Compute Appliance>DB: OVCA_EST
 

This note describes how to add additional Oracle VM repository capacity in the Oracle Private Cloud Appliance (formerly called Oracle Virtual Compute Appliance)

In this Document

Goal

Solution

Applies to:

Goal

 This note shows how to expand repository capacity in the Private Cloud Appliance (PCA)  beyond the default repository allocation.

Solution

Several methods to increase storage capacity are available if the default 300GB capacity for the Rack1-Repository isn't sufficient. IMPORTANT NOTE: if the PCA will be configured to use multiple tenant groups, then disk space must be left available for the tenant groups' repositories and pool file systems. If Rack1-Repository uses up too much of the internal ZFS storage capacity then it will not be possible to create new tenant groups.

1. Increase the size of the repository's iSCSI LUN. This should be done at initial install time before any compute nodes have been provisioned. It can be done afterwards with additional steps, but the process is more complicated and not recommended. The simpler LUN expansion method, available before compute nodes are provisioned, will be illustrated here. The repository can be expanded up to a total of 8TB of space but it is recommended to not go above 2TB in order to reserve disk capacity for future functionality.
2. Add an NFS-based repository on the internal ZFS appliance using the pre-defined NFS mountpoint. It can provide up to a total of 8TB of space but it is recommended to not use more than 2TB. This is the recommended method if storage should be expanded on the internal ZS3 after compute nodes have been provisioned, since it is easier to implement than extending the iSCSI-based Rack1-Repository.
3. Add an iSCSI-based repository on the internal ZFS appliance. This is an alternative to the above method for customers who prefer using iSCSI for consistency with the default repository. Available storage would be the same as with NFS. As with method 1, it is recommended to not go above 2TB of storage.
4. Add an external Oracle ZFS Storage appliance on the same InfiniBand network as the internal ZS3. Alternatively, connect Fibre Channel external storage to the Private Cloud Appliance. This is the recommended method for adding capacity, and can provide arbitrarily large amounts of disk capacity and I/O performance.

Method 1 - increase the size of the Rack1-Repository iSCSI LUN

This method can be used when the Oracle Private Cloud Appliance is initially installed before any compute nodes have been provisioned, using either the ZFS Storage appliance GUI or a command line sequence. While connected to the Private Cloud Appliance using a laptop connected to the internal 192.168.4.0 network,

• point your browser to the ZFS storage appliance user interface and login as root using the default PCA system password. Up to PCA version 2.0.5 use URL https://192.168.4.1:215/ and starting with PCA version 2.1.1, use https://192.168.4.100:215/
• Navigate to Shares -> LUNs
• Highlight OVM/iscsi_repository1
• Click the Edit pencil
• Change 300G to the desired size, such as 2T
• Click Apply
• logoff from the ZFS appliance user interface

Alternatively, you can use a command line sequence for the same effect. This example shows the LUN being increased to 2TB.

• Login to the ZFS appliance: "ssh -l root 192.168.4.1" up to PCA 2.0.5, or "ssh -l root 192.168.4.100" starting with PCA 2.1.1.
• Follow the prompts and enter the highlighted text
  • ovcasn01:> shares
  • ovcasn01:shares> list
  • ovcasn01:shares> select OVM
  • ovcasn01:shares OVM> list
  • ovcasn01:shares OVM> select iscsi_repository1
  • ovcasn01:shares OVM/iscsi_repository1> set volsize=2T
  • ovcasn01:shares OVM/iscsi_repository1> commit
  • ovcasn01:shares OVM/iscsi_repository1> exit

When the first compute node finishes being provisioned with Oracle VM Server, it will create an ocfs2 file system on the iSCSI LUN, and then create an Oracle VM Repository on it.

Method 2 Add an NFS-based repository on the internal ZFS appliance

This method is recommended for adding repository capacity on the internal ZS3 appliance after compute nodes have been provisioned. An NFS mount is pre-defined on the ZS3 appliance and can be used for a repository. This is the procedure documented for discovering a file server (http://docs.oracle.com/cd/E35328_01/E35332/html/vmusg-storage-manage.html#vmusg-storage-fs-discover) and then creating a repository (http://docs.oracle.com/cd/E35328_01/E35332/html/vmusg-repo-manage.html)

• Login to the Oracle Private Cloud Appliance using the Oracle VM Manager browser interface: https://IPaddress:7002/ovm/console
• Select the Storage tab.
• Click on the Discover File Server icon.
• In the popup box, enter a descriptive name such as "Private Cloud Appliance NFS", and enter the IP address 192.168.40.1
• Click next, and in the following screens select all servers to be added as Admin Servers and Refresh servers, clicking Next at each step. Refresh can take some time to complete.
• On the next screen you will see a list of NFS shares - select the one labeled nfs:/export/nfs_repository1, and then click Finish.
• Go to the Repositories tab. Click on the "+" icon to create a repository.
• Enter a repository name, such as "Rack1-Repository-NFS". Leave the "NFS" button selected, and click on the magnifying glass icon.
• Select the share you previously created and click OK. Click Next.
• In the "Present to Server" screen, select all the servers and click Finish.

At this point there is a new repository that can be used to host Oracle VM Assemblies, Templates, ISO Images, virtual disk and VM information.

Care should be taken to ensure that the NFS repository contents do not consume all of the ZFS appliance disk capacity, which would have severe impact on system availability. Note also that compute node vmcore dumps are also stored on the ZFS appliance, which further consumes storage.

Storage consumption can be monitored by logging into the storage appliance and seeing how much disk space is available:

• Login to the Oracle Private Cloud Appliance on either management node: ssh -l root IPaddress
• Issue the command df -h /nfs/shared_storage
• Look at the value shown under the "Avail" column, and make sure there is plenty of residual space - at least several hundred GB available.

While logged onto management node, it is possible to log into the ZFS storage appliance and obtain its view of storage capacity, and optionally set a quota on the repository share to prevent it from using all the ZFS storage.

• While logged into a management node, as in the previous step, issue: 'ssh -l root 192.168.4.1' or 'ssh -l root 192.168.4.100'
• Issue the highlighted part of the following commands:
  • ovcasn01r1:> shares
  • ovcasn01r1:shares> select OVM
  • ovcasn01r1:shares OVM> select nfs_repository1
  • ovcasn01r1:shares OVM/nfs_repository1> list
• You should see lines showing space_available and space_total.
• If space_available is too small, you will need to remove data objects from the repository: log into Oracle VM Manager, and select objects to remove from Rack1-Repository
• To add a quota to protect against consuming excessive disk space from the ZFS command line as in the above step, issue a command like the following
  
  • ovcasn01r1:shares OVM/nfs_repository1> set quota=2TB
  • ovcasn01r1:shares OVM/nfs_repository1> commit

An Oracle VM Manager operation that fills the space quota will cause an error visible in Oracle VM Manager, but this will protect the overall PCA environment from running out of disk space for other purposes.

Method 3: Add an iSCSI-based repository on the internal ZFS appliance

Log into the ZFS storage appliance as shown in the preceding method and create a new iSCSI LUN. In this example we name it 'auxlun' and give it a size of 2TB

• ovcasn01r1:> shares select OVM
• ovcasn01r1:shares OVM> lun auxlun
• ovcasn01r1:shares OVM/auxlun (uncommitted)> set volsize=2t
•                                       volsize = 4T (uncommitted)
• ovcasn01r1:shares OVM/auxlun (uncommitted)> set targetgroup=OVM
•                                      targetgroup = OVM (uncommitted)
• ovcasn01r1:shares OVM/auxlun (uncommitted)> set initiatorgroup=OVM-igroup
•                                      initiatorgroup = OVM-igroup (uncommitted)
• ovcasn01r1:shares OVM/auxlun (uncommitted)> commit

 When this is done you have a new iSCSI LUN of the desired size. Now log into the PCA's Oracle VM Manager user interface, select the "Servers and VMs tab", choose a compute node and the perspective "Physical Disks", and click on "Rescan physical disks". When the new scan completes the new LUN and its name (leftmost column) will appear in the list of physical disks. Repeat this for the other compute nodes, and then go to the Repositories tab and use the dialogue to create a new repository: select "Physical Disk", choose the new LUN, and in select all the compute nodes in the "Create a Data Repository" panels.

Method 4: Add an external Oracle ZFS Storage appliance

This is the recommended procedure for scale, performance and ease of backup. The process is documented in the white paper "Expanding Oracle’s Private Cloud Appliance Using Oracle ZFS Storage Appliance" at http://www.oracle.com/technetwork/server-storage/private-cloud-appliance/expanding-oracle-private-cloud-4305175.pdf  It describes how to discover a storage server and configure a repository similar to the above method.

Conclusions

 This note describes how to extend the storage capacity on the Oracle Private Cloud Appliance if the default repository allocation is not sufficient. This can be done by enlarging the repository that is provisioned when the Oracle Private Cloud Appliance is installed, by adding an additional repository, or by adding external storage.

This solution has no attachment