Storage Management in AIX 5L Version 5.3

Introduction

This article is an update to the "Storage Management in AIX 5L Version 5.3" article published in June 2006. It contains a summary of the scope, functionality, and performance of the Logical Volume Manager (LVM) and some of the features of Enhanced Journal Filesystem (JFS2), available in AIX 5L™ as of the 5300-05 Technology Level (TL), released in August 2006. In the following discussion, the features available only in 5300-05 TL will be identified as such.

---

Back to top

LVM command enhancements

In AIX 5L Version 5.3, changes have been made to the following LVM commands to enhance their performance, such as they require less execution time than their counterparts in prior releases of AIX®:

• extendvg
• importvg
• mkvg
• varyonvg
• chlvcopy
• mklvcopy
• lslv
• lspv

---

Back to top

Concurrent mode (classical and enhanced)

The classical concurrent mode volume groups (VGs) only supported Serial DASD and SSA disks in conjunction with the 32-bit kernel. AIX 5L Version 5.1 overcame the restriction of supported disk types by introducing the so-called enhanced concurrent mode VG, which extended the concurrent mode support to all other disk types. While AIX 5L Version 5.2 did not allow creation of classical concurrent mode VGs, it did support them. The support for classical concurrent mode VGs has been completely removed from AIX 5L Version 5.3. When trying to import a classical concurrent mode VG in AIX 5L Version 5.3, an error message informs the user to convert the VG to enhanced concurrent mode.

---

Back to top

VGs (normal, big, and scalable)

The VG type, commonly known as standard or normal, allows a maximum of 32 physical volumes (PVs). A standard or normal VG is no more than 1016 physical partitions (PPs) per PV and has an upper limit of 256 logical volumes (LVs) per VG. Subsequently, a new VG type was introduced which was referred to as big VG. A big VG allows up to 128 PVs and a maximum of 512 LVs.

AIX 5L Version 5.3 has introduced a new VG type called scalable volume group (scalable VG). A scalable VG allows a maximum of 1024 PVs and 4096 LVs. The maximum number of PPs applies to the entire VG and is no longer defined on a per disk basis. This opens up the prospect of configuring VGs with a relatively small number of disks and fine-grained storage allocation options through a large number of PPs, which are small in size. The scalable VG can hold up to 2,097,152 (2048 K) PPs. As with the older VG types, the size is specified in units of megabytes and the size variable must be equal to a power of 2. The range of PP sizes starts at 1 (1 MB) and goes up to 131,072 (128 GB). This is more than two orders of magnitude above the 1024 (1 GB), which is the maximum for both normal and big VG types in AIX 5L Version 5.2. The new maximum PP size provides an architectural support for 256 petabyte disks.

Table 1 below shows the variation of configuration limits with different VG types. Note that the maximum number of user definable LVs is given by the maximum number of LVs per VG minus 1 because one LV is reserved for system use. Consequently, system administrators can configure 255 LVs in normal VGs, 511 in big VGs, and 4095 in scalable VGs.

Table 1. Configuration limits for VGs

VG type	Maximum PVs	Maximum LVs	Maximum PPs per VG	Maximum PP size
Normal VG	32	256	32,512 (1016 * 32)	1 GB
Big VG	128	512	130,048 (1016 * 128)	1 GB
Scalable VG	1024	4096	2,097,152	128 GB

The scalable VG implementation in AIX 5L Version 5.3 provides configuration flexibility with respect to the number of PVs and LVs that can be accommodated by a given instance of the new VG type. The configuration options allow any scalable VG to contain 32, 64, 128, 256, 512, 768, or 1024 disks and 256, 512, 1024, 2048, or 4096 LVs. You do not need to configure the maximum values of 1024 PVs and 4096 LVs at the time of VG creation to account for potential future growth. You can always increase the initial settings at a later date as required.

The System Management Interface Tool (SMIT) and the Web-based System Manager graphical user interface fully support the scalable VG. Existing SMIT panels, which are related to VG management tasks, have been changed and many new panels added to account for the scalable VG type. For example, you can use the new SMIT fast path _mksvg to directly access the Add a Scalable VG SMIT menu.

The user commands mkvg, chvg, and lsvg have been enhanced in support of the scalable VG type.

---

Back to top

Striped column support for LVs

AIX 5L Version 5.3 provides striped columns support for LVs. This new feature allows extension of a striped LV, even if one of the PVs in the disk array becomes full. In previous AIX releases, you could enlarge the size of a striped LV with the extendlv command, as long as enough PPs were available within the group of disks which defined the RAID (redundant array of independent disks) disk array. Rebuilding the entire LV was the only way to expand a striped LV beyond the hard limits imposed by the disk capacities. You needed to back up and delete the striped LV, and then recreate the LV with a larger stripe width followed by a restore operation of the LV data. To overcome the disadvantages of this time-consuming procedure, AIX 5L Version 5.3 has introduced the concept of striped columns for LVs.

Prior to AIX 5L Version 5.3, the stripe width of a striped LV was determined at the time of LV creation by either of the following two methods:

• Direct specification of all PV names
• Specification of the maximum number of PVs allocated to the striped LV

Prior versions of AIX 5L do not allow you to configure a striped LV with an upper bound larger than the stripe width. In AIX 5L Version 5.3, the upper bound can be a multiple of the stripe width. One set of disks, as determined by the stripe width, is considered as one striped column. Note that the upper bound value is not related to the number of mirror copies in case you are using a RAID 10 configuration.

If you use the extendlv command to extend a striped LV beyond the physical limits of the first striped column, AIX uses an entire new set of disks to fulfill the allocation request for additional logical partitions. If you further expand the LV, more striped columns might get added as required, as long as you stay within the upper bound limit. The chlv -u command allows you to increase the upper bound limit to provide additional headroom for striped LV expansion. You can also use the -u flag of the enhanced extendlv command to raise the upper bound and extend the LV all in one operation.

The user commands mklv, chlv, extendlv, and mklvcopy have been enhanced to support the introduction of the striped column feature in AIX 5L Version 5.3.

---

Back to top

Volume group pbuf pools

The LVM uses a construct named pbuf to control a pending disk I/O. A pbuf is a pinned memory buffer. The LVM always uses one pbuf for each individual I/O request, regardless of the amount of data that is transferred. AIX creates extra pbufs when adding a new PV to a VG. In previous AIX releases, the pbuf pool was a system-wide resource, but the LVM assigns and manages one pbuf pool per VG with AIX 5L Version 5.3. This enhancement supports advanced scalability and performance for systems with a large number of VGs and applies to all VG types. As a consequence of the new pbuf pool implementation, AIX displays and manages additional LVM statistics and tuning parameters.

AIX 5L Version 5.3 now includes the lvmo command. It provides support for new pbuf pool-related administrative tasks. You can use the lvmo command to display pbuf and blocked I/O statistics and settings for pbuf tunables, regardless of whether the scope of the entity is system-wide or VG-specific. However, the lvmo command only allows the settings to change for the LVM pbuf tunables that are dedicated to specific VGs. The ioo command continues to manage the sole pbuf tunable with system-wide scope. Also, the vmstat -v command still displays the system-wide number of I/Os that were blocked due to lack of free pbufs like in prior releases of AIX.

---

Back to top

Variable logical track group

When the LVM receives a request for an I/O, it breaks the I/O down into what is called logical track group (LTG) sizes before it passes the request down to the device driver of the underlying disks. The LTG is the maximum transfer size of an LV and is common to all the LVs in the VG. AIX 5L Version 5.2 accepted LTG values of 128 KB, 256 KB, 512 KB, and 1024 KB. However, many disks now support transfer sizes larger than 1 MB. To take advantage of these larger transfer sizes and get better disk I/O performance, AIX 5L Version 5.3 accepts values of 128 KB, 256 KB, 512 KB, 1 MB, 2 MB, 4 MB, 8 MB, and 16 MB for the LTG size.

In contrast to previous releases, AIX 5L Version 5.3 also allows the stripe size of an LV to be larger than the LTG size in use and expands the range of valid stripe sizes significantly. Version 5.3 adds support for 2 MB, 4 MB, 8 MB, 16 MB, 32 MB, 64 MB, and 128 MB stripe sizes to complement the 4 KB, 8 KB, 16 KB, 32 KB, 64 KB, 128 KB, 256 KB, 512 KB, and 1 MB stripe size options available in prior releases of AIX.

In AIX 5L Version 5.2, the LTG size was set by the -L flag on the chvg or mkvg command. In AIX 5L Version 5.3, it is set by the varyonvg command, using the flag -M. The LTG size thus created is called the variable LTG size.

The following command sets the LTG size of the tmpvg VG at 512 KB:

# varyonvg -M512K tmpvg

The LTG size is specified either in K or M units, implying KB or MB respectively. When the LTG size is set using the -M flag, the varyonvg and extendvg commands might fail if an underlying disk has a maximum transfer size that is smaller than the LTG size. To find out the maximum supported LTG size of your hard disk, you can use the lquerypv command with the -M flag. The output gives the LTG size in KB, as shown in example below.

# /usr/sbin/lquerypv -M hdisk0 256

The lspv command displays the same value as MAX REQUEST, as shown in Listing 1 below.

Listing 1. lspv command

# lspv hdisk0 PHYSICAL VOLUME: hdisk0 VOLUME GROUP: rootvg PV IDENTIFIER: 000bc6fdbff92812 VG IDENTIFIER 000bc6fd00004c00000000fda469279d PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: yes PP SIZE: 16 megabyte(s) LOGICAL VOLUMES: 9 TOTAL PPs: 542 (8672 megabytes) VG DESCRIPTORS: 2 FREE PPs: 431 (6896 megabytes) HOT SPARE: no USED PPs: 111 (1776 megabytes) MAX REQUEST: 256 kilobytes FREE DISTRIBUTION: 108..76..30..108..109 USED DISTRIBUTION: 01..32..78..00..00

You can list the value of the LTG in use with the lsvg command, as shown in Listing 2 below.

Listing 2. lsvg command

# lsvg rootvg VOLUME GROUP: rootvg VG IDENTIFIER: 000bc6fd00004c00000000fda469279d VG STATE: active PP SIZE: 16 megabyte(s) VG PERMISSION: read/write TOTAL PPs: 542 (8672 megabytes) MAX Lvs: 256 FREE PPs: 431 (6896 Megabytes) LVs: 9 USED PPs: 111 (1776 megabytes) OPEN LVs: 8 QUORUM: 2 TOTAL PVs: 1 VG DESCRIPTORS: 2 STALE PVs: 0 STALE PPs: 0 ACTIVE PVs: 1 AUTO ON: yes MAX PPs per VG: 32512 MAX PPs per PV: 1016 MAX PVs: 32 LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no HOT SPARE: no BB POLICY: relocatable

Note that the LTG size for a VG created in AIX 5L Version 5.3 will be displayed as dynamic in the lsvg command output, as shown Listing 2 above.

By default, AIX 5L Version 5.3 creates VGs with a variable LTG size. If you want to import it to a previous release of AIX, you first need to disable the variable LTG by using the -I option for mkvg or chvg and then do a varyoffvg followed by exportvg, otherwise the importvg command on the previous release fails.

Geographic Logical Volume Manager (GLVM)

It extends the LVM mirroring function and supports copy of a logical volume on a remote AIX system connected using TCP/IP network. A complete copy of application data can be quickly and easily brought back online on a remote system.

The mirscan command

This command searches for and corrects physical partitions that are stale or unable to perform I/O operations. This is useful for the following type of situations:

1. A physical partition on the underlying storage is incapable of performing I/O operations but, for a long time, no I/O operations have been attempted for that physical partition. The customer needs a way to detect and correct this condition.
2. A disk is about to be replaced. The customer needs to make sure they are not about to remove the last good copy of their data from the system.

Multiple instances of AIX on a single root volume group (multibos)

This feature allows the user to create a new instance of the AIX Base Operating System (BOS) within the running rootvg. This new instance, based on the running rootvg, contains private and shared data. A similar offering already available is Alternate Disk Installation. While somewhat similar, multibos varies in a few very important aspects:

• The new instance is built from the running root volume group (similar to the alt_disk_install clone operation).
• The new instance is housed within the current root volume group (for example, the same disks).
• Certain data within the rootvg might be shared between the instances.

In the 5300-05 Technology Level, released in August 2006, this feature has been made more robust and user-friendly. Enhancements have been made in the area of setup, customize, and verify operations through script controls. Improvements have been made to secure hard links and filesystem stub permissions.

Rollback function (available for JFS2 file system only)

Restores an entire file system to a valid point-in-time snapshot (target snapshot). Rollback attempts to restore the snapshots present at the time of the target snapshot. Snapshots taken after the target snapshot are lost.

Disk quotas support for JFS2

AIX 5L Version 5.3 extends the JFS2 functionality by implementing disk usage quotas to control usage of persistent storage.

Disk quotas might be set for individual users or groups on a per file system basis.

Version 5.3 also introduces the concept of Limit Classes. It allows the configuration of per file system limits, provides a method to remove old or stale quota records, and offers comprehensive support through dedicated SMIT panels. It also provides a method to define a set of hard and soft disk block and file allocation limits and the grace periods before the soft limit becomes enforced as the hard limit.

The quota support for JFS2 and JFS can be used on the same system.

Shrink a file system

AIX 5L Version 5.3 supports shrinking a JFS2 file system dynamically. When the size of the file system is decreased, the LV on which the file system resides is also decreased.

JFS2 logredo scalability

AIX 5L Version 5.3 provides the following enhancements in the area of logredo to improve performance and to support large numbers of file systems:

• Support for minor numbers greater than 512 in a volume group
• Support for copy-on-write and cached updates to reduce I/O activity
• Support for shrink file system

Performance of JFS2 Logredo has been further enhanced in the 5300-05 Technology Level released in August 2006. The goal was to improve restart time for both quiesced file system and normal restart.

Access time (atime) update

JFS commits atime updates asynchronously. Prior to the 5300-05 Technology Level release in August 2006, JFS2 treated them like all other commits. JFS2 has been modified to work like JFS. This reduces the number of I/O operations to the log.

JFS2 file system check scalability

AIX 5L Version 5.3 enhanced the implementation of the helper, which specifically performs the file system check for JFS2 file systems. The new code makes a better use of system resources and includes algorithms that improve the scalability and performance.

JFS2 ACL support for NFS V4

Starting with AIX 5L Version 5.3, the Enhanced Journaled File System now supports ACLs for NFS version 4. This allows you to establish fine-grained access control for file system objects and support inheritance features.

---

Back to top

Conclusion

AIX 5L Version 5.3 has many more features than have been discussed here. I hope this article has given you a flavor of the type of enhancements you can expect in the latest release of AIX. For a detailed description of these enhancements, visit the Web sites listed under the Resources section.

Resources

Learn

• AIX 5L Differences Guide Version 5.3 Edition: This redbook focuses on the differences introduced in AIX 5L Version 5.3 when compared to AIX 5L Version 5.2.
  
  
• IBM pSeries® and AIX Information Center: Visit this site for more information on LVM in AIX 5L.
  
  
• The AIX and UNIX developerWorks zone provides a wealth of information relating to all aspects of AIX systems administration and expanding your UNIX skills.
  
  
• New to AIX and UNIX: Visit the New to AIX and UNIX page to learn more about AIX and UNIX.
  
  
• developerWorks technical events and webcasts: Stay current with developerWorks technical events and webcasts.
  
  
• AIX 5L Wiki: A collaborative environment for technical information related to AIX.
  
  
• Podcasts: Tune in and catch up with IBM technical experts.
  
  

Get products and technologies

• IBM trial software: Build your next development project with software for download directly from developerWorks.
  
  

Discuss

• Participate in the AIX and UNIX forums:
  • AIX 5L -- technical
  • AIX for Developers Forum
  • Cluster Systems Management
  • IBM Support Assistant
  • Performance Tools -- technical
  • Virtualization -- technical
  • More AIX and UNIX forums
  
  
• Participate in the developerWorks blogs and get involved in the developerWorks community.