An is auditor should use statistical sampling, and not judgment (nonstatistical) sampling, when:

What is Non-Statistical Sampling?

Non-statistical sampling is the selection of a test group that is based on the examiner's judgment, rather than a formal statistical method. For example, an examiner could use his own judgment to determine one or more of the following:

• The sample size

• The items selected for the test group

• How the results are evaluated

To reduce the amount of variability in a non-statistically determined sample size, an examiner usually refers to a table that sets forth the approximate sizes to be used. For example, a low-risk situation may call for the selection of 25 records, while a high-risk situation might mandate the selection of 100 records.

When using a non-statistical approach to select the items for a test group, the examiner should not introduce too much bias into the selections. For example, do not lean too heavily on supplier invoices where the invoice amount exceeds $10,000 and the name of the supplier begins with a "P". Instead, the selection should come as close as possible to representing the entire population of records.

When to Use Non-Statistical Sampling

It can make sense to use non-statistical sampling when the population size is very small. In this case, it is not efficient to spend the extra time to set up a statistical sample. This approach is also useful in areas where specific records contain sensitive information, and so must be examined. For example, an examiner might want to select the invoices of specific law firms, because these firms deal with environmental obligations, which may involve substantial liabilities.

December 1994

To accomplish this, the auditor needs to examine a representative sample or cross-section of the various type of accounts (e.g., legal, good telephone, skip, payment arrangements, settled, closed) as well a review of the remittance history.

How the sample should be selected and how large the sample should be are critical issues for researchers as well as auditors.

According to researchers M. Hanson and P. Hauser, in their article "Principles of Sample Design," "The science of sampling design involves: (1) looking at the resources available, the restrictions under which one must work, the mathematical and statistical tools available, the accumulated knowledge of certain characteristics of the populations to be sampled; and (2) putting these together to arrive at the optimum design for the purpose at hand."

Hanson and Hauser point out that the overall criterion that should be applied in choosing a sampling design is to design the sample so that it will yield the desired information with the reliability required at a minimum cost; or conversely, that "at a fixed cost it will yield estimates of the statistics desired with the maximum reliability possible."

STATISTICAL VS. NONSTATISTICAL SAMPLING

Simply stated, a sampling plan is nonstatistical when it fails to meet at least one of the criteria required of a statistical sampling plan. Auditors should know the requirements of statistical plans, because, by definition, any deviation constitutes a nonstatistical approach.

The difference between the two types of sampling is that the sampling risk of a statistical plan can be measured and controlled, while even a perfectly designed nonstatistical plan cannot provide for the measurement of sampling risk.

The basic similarity between the two types is that both sampling approaches require the exercise of auditor judgment during the planning, implementation and evaluation of the sampling plan. In other words, the use of statistical methods does not eliminate the need to exercise judgment.

In addition, the actual audit procedures performed on the items in the sample will be the same, whether a statistical or nonstatistical approach is used. The employment of a statistical plan does not mean the auditor can alter the procedures designed to collect evidence to draw an audit conclusion.

It is up to the auditor to evaluate the individual and situational costs and benefits associated with each sampling approach before making a determination.

In some circumstances, statistical sampling is more appropriate than judgment sampling. Before deciding whether to use statistical or judgmental sampling, the auditor must determine the audit objectives; identify the population characteristics of interest; and state the degree of risk that is acceptable. After making those determinations, it may be advisable to use statistical sampling if the auditor has a well-defined population and can easily access the necessary documentation.

Obviously, if the audit methodology and parameters limit the on-site portion of an agency audit to one or two days, the sample design and size must be a realistic reflection of this time constraint.

It is a fallacy that the "statistical rule of thumb" is to sample 10% of the accounts. There is no such magic number. If the entire population is 10, a 10% sample equals one account -- not very representative. Therefore, it is the absolute numbers, not the percentage, that is important.

STATISTICAL PROBABILITY SAMPLING

Accounts to be reviewed during an audit are normally selected through one of the probability sampling methods -- random, systematic or stratified. Probability sampling provides an objective method of determining sample size and selecting the items to be examined. Unlike nonstatistical sampling, it also provides a means of quantitatively assessing precision (how closely the sample represents the population) and reliability (confidence level, the percentage of times the sample will reflect the population).

Simple Random Sampling

In auditing, this method uses sampling without replacement; that is, once an item has been selected for testing it is removed from the population and is not subject to re-selection. An auditor can implement simple random sampling in one of two ways: computer programs or random number tables.

Systematic (Interval) Sampling

This method provides for the selection of sample items in such a way that there is a uniform interval between each sample item. Under this method of sampling, every "Nth" item is selected with a random start.

Stratified (Cluster) Sampling

This method provides for the selection of sample items by breaking the population down into stratas, or clusters. Each strata is then treated separately. For this plan to be effective, dispersion within clusters should be greater than dispersion among clusters. An example of cluster sampling is the inclusion in the sample of all remittances or cash disbursements for a particular month. If blocks of homogeneous samples are selected, the sample will be biased.

Remember, an essential feature of probability sampling methods is that each element of the population being sampled has an equal chance of being included in the sample and, moreover, that the chance of probability is known. Only in this way, is a probability sample representative of a population.

NONSTATISTICAL SAMPLING

Some selection methods can be used only with nonstatistical sampling plans.

Haphazard Selection

In this method, the auditor selects the sample items without intentional bias to include or exclude certain items in the population. It represents the auditor's best estimate of a representative sample -- and may, in fact, be representative. Defined probability concepts are not employed. As a result, such a sample may not be used for statistical inferences. Haphazard selection is permitted for nonstatistical samples when the auditor believes it produces a fairly representative sample.

Block Selection

Block selection is performed by applying audit procedures to items, such as accounts, all of which occurred in the same "block" of time or sequence of accounts. For example, all remittances in the month of November. Alternatively, remittances 300-350 may be examined in their entirety. Block selection should be used with caution because valid references cannot be made beyond the period or block examined. If block sampling is used, many blocks should be selected to help minimize sampling risk.

Judgment Selection

Judgment sample selection is based on the auditor's sound and seasoned judgment. Three basic issues determine which items are selected:

1. Value of items. A sufficient number of extensively worked or older accounts should be included to provide adequate audit coverage.

2. Relative risk. Items prone to error due to their nature or age should be given special attention.

3. Representativeness. Besides value and risk considerations, the auditor should be satisfied that the sample provides breadth and coverage over all types of items in the population.

SAMPLING STATISTICS AND THE AGENCY AUDIT

An agency audit need not be based on a statistical sample to be considered valid. In fact, to concentrate on a statistical selection method is to miss the point of the agency audit. It is more important to be able to identify areas in need of improvement than to identify the standard deviation of the population mean. It is more valid to address issues of concern than calculate the confidence level of the sampling statistic.

An experienced auditor with good judgment and a well-defined audit goal need only review a random cross-section to know if the agency is in compliance and what steps must be taken to improve performance and maximize profits.

Remember the goals of an agency audit:

1. To insure compliance with the client's work standards

2. To evaluate current agency performance; and

3. To maximize profits for both client and agency.

With these goals clearly in view, experienced auditors balance the resource available, the restrictions of each audit, the mathematical and statistical tools available, and his or her accumulated knowledge of the characteristics of the population being sampled and arrive at the optimum audit design for the purpose at hand. Just as Hanson and Hauser recommend.

When would the IS auditor use statistical sampling instead of judgment or Nonstatistical sampling?

What is the difference between statistical and Nonstatistical sampling?

Which is better statistical sampling or Judgemental sampling?

Why do auditors use non