Blog Archive

I am an Oracle database Consultant My Areas of Interests being High availabilty,Infrastructure consulting and Performance Tuning. I am posting Topics of My interests Related to these Technology Areas. Do post Your comments as well.

Sunday 11 February 2007

Real Application Clusters - Disk Sub System Overview

Raw Partitions and Cluster File System

Oracle RAC can use either Cluster file system files or raw partitions from a shared disk group. Raw device a disk drive that does not have a file system set up. Raw partition is a portion of the physical disk that is accesses at the lowest possible level. Traditionally they were in use for OPS and provided a high performance with bypassing the file system overhead. RAC supports both the cluster file systems and raw devices to store the shared data. Popular and widely Cluster File System products include: HP Tru64, Veritas CFS, Sistina GFS, Polyserve Matrix CFS and Oracle Cluster File system (for windows).

Cluster File System offers
  • Simple management
  • Use of Oracle managed file with RAC
  • Single Oracle Software Installation
  • Auto-extend Enabled on Oracle Data Files
  • Uniform accessibility of Archive Logs

When to Use Raw over CFS

When CFS is not available , Raw devices offer best performance without any interference layer between Oracle and the Disk.
However, with the use of Logical Volume Manager, creation and administration of raw volumes became relatively simple.
However, using Raw Partition need careful management and planning.


Features of a typical C.F.S like Tru64 CFS

The Cluster File System (CFS) makes all files, including the root (/), /usr, and/var file systems, visible to and accessible by all cluster members. There is single copy or image for all cluster members.A single cluster member serves each file system. Other members access that file system as CFS clients with significant optimizations for shared access.Oracle 9i RAC automatically does direct-I/O on Tru64 UNIX file system storage. This can significantly improve I/O performance for the database since Oracle 9i does its own caching and file region synchronization.

Advantages with C.F.S:

Since CFS provides single namespace, there is a single copy of most configuration files. With few exceptions, the directory structure of a cluster is identical to that of a standalone system. However wherever files are not to be shared (like /etc/passwd, /etcsysconfigtab), there is provision of CDSL (context-dependent symbolic link), which manages the own copies for cluster members. Node specific Oracle files like tnsnames.ora, listener.ora, sqlnet.ora can also be created using CDSL system.

Volume Manager

Database is constructed and maintained on file system files or raw partitions, which are entities at operating system level. They are all based on the storage devices or SAN. There are several different layers of abstraction between the physical disks and the objects that database uses. At the lowest level, there are physical drives, then move up into LUNS (logical unit numbers) of manageable sizes with suitable RAID level. LUNS are presented to host system’s Volume Manager. Within Volume Manager, devices are grouped into Disk Groups (e.g. Veritas Disk group). From within a disk group, create a Logical Volume with optional Raid level. Finally Logical Volume is either presented as a File system after mounting or presented as a raw partition to system use. VM tool provides a very flexible way of managing the volumes and file systems. With out Volume manager LUNS are physical disks with partitions.
Thus Volume Manager hides the details about where data is stored, on which actual hardware and where on that hardware, from the entire system. Volume management lets you edit the storage configuration like setting up of software RAID and extend volume size etc without actually changing anything on the hardware side. Some Volume Manager tools allow taking snapshot copies of volumes to move or backup. Thus, for either creating the raw partitions (or volumes) or for setting up of file systems, V.M. is very handy tool.
Veritas volume manager has been in use on many platforms like Solaris, HP and AIX. Now Veritas has VM to support the Windows systems. On the Linux side, Suse LVM has been in use for a long time.
When possible, advanced file systems such as xfs, jfs, or vxfs should be preferred over ufs file systems. This is to utilize the improved journal and performance features (such as elimination of double buffering) that advanced file systems offer over ufs. It must be noted that if you configure20 your system using ufs file systems the necessary file system level parameters need to be configured and tuned, as the default values will not provide optimal performance. When using Veritas vxfs file systems, Quick I/O can provide raw device comparable performance without losing the benefits of a file system. This is because the Quick I/O driver intercepts all DBWR writes (when enabled) thus bypassing the file system buffer cache. This provides raw device comparable performance, as the classic problems of double buffering and wasted CPU cycles in managing the file system buffer cache are avoided. When using other types of advanced file systems, Direct I/O can be preferred if supported by the OS, as it too provides raw device comparable performance. The use of raw devices does not add any significant value when compared to advanced file systems with Quick/Direct I/O drivers (wherever applicable). Raw devices do add a level of operational complexity without justifiable performance benefits, When compared to advanced file systems configured with Veritas Quick I/O or Direct I/O (supported by the OS). In these cases, the configuration and use of raw devices should be reserved for Oracle Parallel Server implementations. Asynchronous I-O Oracle configured with asynchronous I/O has been found to work effectively only on raw devices across most flavors of UNIX. Asynchronous I/O on regular file systems is supported on some operating systems such as Solaris. Depending on your I/O system configuration, you may observe that Direct I/O or Quick I/O can offer comparable performance when using specialized file systems such as Veritas (vxfs). You may also have the option of configuring the relevant Oracle instance parameters for multiple database writer support. But normally you either enable asynchronous I/O or multiple database writers, not both.

Oracle Disk Manager

Oracle Disk Manager (ODM) is oracle's innovative I/O and file management infrastructure.ODM interface is a set of API calls that Oracle co-developed with veritas for incorporating underlying volume managers and file systems.Benefits include simplified file administration,improved file integrity and reduced system overhead.Veritas and Oracle studies indicate that ODM files perform equal or better than the raw partitions.

No comments: