To measure performance during off hours, a scheduled data collector set should be used.

Graph types

Performance Monitor has three views of counter data: Line, Histogram, and Report view. To change the graph type, click on the third icon from the left on the top menu. The drop-down list will show Line, Histogram bar, and Report (Figure 2.6).

Earlier in this chapter, I mentioned that when counter data is collected, Perfmon generates several statistics (Minimum, Average, Maximum, and Last) based on the data currently being displayed. A line chart shows the data points across time and shows the statistics of each selected counter in the fields.

The Histogram and Report charts can only display one value of a counter at a time. Perfmon designates which value it will show based on the general properties of Perfmon. To change the statistical value, right-click anywhere on the chart and click Properties. The Performance Monitor Properties dialog box will show. Navigate to the General tab and locate the Report and histogram data section (Figure 2.7).

The default value shown in Report and Histogram charts is Default. While the other values are obvious in what they show, Default is a bit mysterious. In my experience, it is the current value and my guess is that value of Default is chosen by the creator of the counter object. In any case, when I use Histogram or Report charts, I commonly change this setting from Default to the statistical value I'm looking for just to be sure.

Adding Counters to Reliability and Performance Monitor to Monitor Print Servers

By using Performance Monitor, a network or server administrator can identify bottlenecks on a wide variety of server and network components. As an administrator, you can also use Performance Monitor to capture performance counters over time and identify baselines and trends.

In Windows Server 2008, Performance Monitor includes many counters that enable administrators to keep an eye on many network printers at the same time or even capture data for later analysis. You can view these performance counters in a Line Graph, Histogram, or Report view. Launch Performance Monitor by navigating to Server Manager | Diagnostics | Reliability and Performance | Monitoring Tools | Performance Monitor. Figure 7.40shows the Report view in Performance Monitor.

To monitor printing statistics such as Bytes Printed/sec, Jobs Spooling, and Out of Paper Errors in real time, add counters by clicking the plus button and then expand the Print Queue counters, as shown in Figure 7.41.

Monitoring Performance

☑

The Performance Monitor in earlier versions of Windows has been redesigned and renamed the Reliability and Performance Monitor. The Reliability Monitor keeps track of software installs/uninstalls and all system, application, and hardware failures over the lifetime of the system.

The Performance Monitor can be used to display real-time performance information as well as collect performance data using Data Collector Sets and by saving the information in log files. You can also generate performance alerts based on specific thresholds for performance objects such as the processor, the hard disk, memory, networking interfaces and protocols, and so on.

The Performance Monitor can be used to compare the performance information stored in two or more log files. You must use the Performance Monitor as a standalone utility to use this feature.

Monitoring system committed memory with performance monitor

If you are analyzing a performance counter log, monitoring a remote Windows or Windows Server computer, or collecting system committed data throughout an enterprise environment, then using performance counters is probably the most convenient way to check the system committed memory.

Performance Monitor has three counters related to system committed memory, \Memory\Commit Limit, \Memory\Committed Bytes, and \Memory\% Committed Bytes In Use. Figure 6.9 shows these counters in report view.

Tip

Remember, even though some performance counters such as commit limit are typically very large numbers, they are still double values—meaning they have a decimal point and numbers after the decimal point.

The initial indicator to warrant more investigation into system committed memory is to look for a value of greater than 75 for the counter \Memory\% Committed Bytes In Use counter (Figure 6.10). When it breaks this threshold, go to “Where did all of the system committed memory go?” section later in this chapter.

Some tools for measuring process private committed memory

•

Microsoft Performance Monitor using the counters \Process(*)\Private Bytes or \Process(*)\Page File Bytes.

Microsoft Task Manager using the Commit Size column on the Details tab. On older versions of Windows or Windows Server, it will be either the Memory—Commit Size (Commit Size) column or the Virtual Memory Size (VM Size) column under the Processes tab. These columns might need to be added.

Microsoft Resource Monitor using the Commit (KB) column on the Memory tab or grouping.

Windows Sysinternals Process Explorer using the Private Bytes column under Process Memory.

Microsoft Windows Management Instrumentation (WMI) using the Win32_Process.PageFileUsage property.

Tip

Most tools refer to process, private, committed memory as Private Bytes.

Monitoring the Hyper-V Switch

You can use Performance Monitor counters to better understand the traffic flow on the Hyper-V Switch. Just like any other physical switch, many questions will come up once the environment is in production, such as:

What are the characteristics of the traffic on this port?

Do we have any problems on this port?

Is any packet getting denied in the switch port level?

In some cases, you will deal with some of those questions built into a broader scenario, such as the example below:

Scenario

Chris is a Private Cloud administrator that needs to understand the amount of traffic generated by one particular VM from one of his tenant. Chris also needs to understand if there are incoming or outgoing packets that are getting dropped.

Before understanding how to resolve this scenario, it is important to understand the basic components of a Hyper-V Switch. The Hyper-V Switch (vmSwitch) supports creation of multiple virtual switches (vSwitch). Each vSwitch can have any number of virtual ports (vPort). These ports can be dynamically added or removed from the switch. Each vSwitch keeps its own FT (Forwarding Table) and performs traffic forwarding based on a MAC address and VLAN tag that are presented in the packet. There are three types of vSwitches which are based on the type of the NIC that is connected to the switch. The available types are:

External Switch: is connected (think of it as “bound”) to a physical NIC and allows communication with the external networks outside the physical machine.

Internal Switch: is not connected to (or “bound to”) a physical NIC, but it is connected to at least one virtual NIC in the root partition and zero or more virtual NIC in the child partitions. The configuration enables the host operating system to communicate with the virtual machines connected to the virtual switch and the virtual machines to communicate with other virtual machines connected to the same virtual switch. However, the virtual machines do not have a direct connection to a physical network. It is possible to configure the host operating system to enable access to physical networks so that it can provide a bridge from the Internal virtual switch to external networks.

Private Switch: is only connected to virtual NIC in the child partitions and it only provides communication between child partitions. What this means is that all virtual machines connected to the Internal switch are able to communicate with one another but are not able to communicate with hosts not connected to the virtual switch unless there is another virtual device that bridges/routes connections between the private virtual switch and a destination that is not connected to the virtual switch. Note the with the Private virtual switch you do not have direct connective between the host operating system and the guests; any such communications would have to take place over the aforementioned bridging gateway.

It is important to keep in mind that when you create a virtual machine and connect it to a Hyper-V virtual switch, that virtual machine is connected using a default virtual NIC that connects to a port on the virtual switch. It is possible to add more virtual NICs to a virtual machine using the user interface in the Hyper-V console. You can also use PowerShell to add virtual NICs. This is important in the event that you want to connect the host operating system to the virtual switch. In this case, you can use PowerShell to create virtual NICs that attach the host operating system to the switch. When you do this, the host operating system can take advantage of the security, performance, and availability features you obtain when using the Hyper-V switches capabilities in these areas.

When designing your virtualization security, it is very important to use the correct switch type so as to prevent VMs from sending traffic to locations that they are not supposed to. You can obtain more information about the Hyper-V Switch you can use the Get-VMSwitch PowerShell cmdlet as shown in the example below:

PS C:\Users\Administrator > Get-VMSwitch "Private VSwitch" | Get-VMSwitchExtension

Id: EA24CD6C-D17A-4348-9190-09F0D5BE83DD

Name: Microsoft NDIS Capture

Vendor: Microsoft

Version: 6.2.8250.0

ExtensionType: Capture

ParentExtensionId:

ParentExtensionName:

SwitchId: 73679314-AC82-4BB8-9321-7F74C7CE1E91

SwitchName: Private VSwitch

Enabled: False

Running: False

ComputerName: W8HV2012

IsDeleted: False

Id: E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A

Name: Microsoft Windows Filtering Platform

Vendor: Microsoft

Version: 1.0

ExtensionType: Filter

ParentExtensionId:

ParentExtensionName:

SwitchId: 73679314-AC82-4BB8-9321-7F74C7CE1E91

SwitchName: Private VSwitch

Enabled: True

Running: True

ComputerName: W8HV2012

IsDeleted: False

Important

Among other options, the SwitchID might be very important when performing queries against the switch itself.

In order to monitor the Hyper-V Switch and its ports, you can use the following Performance Monitor Counters:

Hyper-V Virtual Switch

Hyper-V Virtual Switch Port

For the scenario that was presented in the beginning of this section, the counters that should be used are:

Hyper-V Virtual Switch Port

Bytes Sent/sec

Dropped Packets Incoming/sec

Dropped Packets Outgoing/sec

Note

During the perfmon configuration, you will need the Switch ID to identify which switch you want to monitor; that is why it is important to use the Get-VMSwitch cmdlet.

17.2.1.2 Perfmon

Perfmon, or Performance Monitor, is a more advanced performance measurement tool that comes loaded as a part of Windows operating systems. This section provides just a brief introduction of this tool. For more complete information on it, you can either perform an Internet search or click menu item Help | Help Topics. Some of the significant features of Perfmon that I’ve found useful are:

You can select the statistics you want displayed from a wide range of activity categories.

Statistics data can be exported to either a log file or a database. You can open Perfmon at a later date or time and view the statistics from those sources.

You can control how frequently measurements are taken. Be aware that if measurements are captured too frequently, Perfmon can impact the server’s performance.

Perfmon can be set up to run continuously or it can periodically write data to a new file.

A set of measurements being captured are contained within an object called a “Counter Log.” You can have more than one Counter Log active on a server at the same time. One reason for doing this would be if each group captured different performance statistics at different frequencies.

Two ways to launch Perfmon are described here:

In a command window, enter the command “perfmon” and press the Enter key

Select Start | Administrative Tools | Performance

The initial screen you’ll see when Perfmon is opened is shown in Figure 17.4. Some of the major points on this screen are:

The default displays shows real-time statistics, as opposed to stats from a log file. The activities shown in the lower right panel are what is being monitored.

The three default activities that are monitored are

Pages/sec which is a measurement of how much memory is being used

Average Disk Queue Length which measures the amount of disk activity

% Processor Time which measures how busy the CPU is

Expand “Performance Logs and Alerts” in the left panel to list Counter Logs or create a new one.

The solid vertical red bar moves from left to right. When it reaches the right hand edge of the screen, it reappears at the left hand margin. The most recent statistics are written to the left of the vertical bar.

Each statistic is assigned a unique color. Colors for statistics can be reassigned by right-clicking on the chart panel and selecting Properties. Then click on a statistic and change the color by selecting a new color in the drop-down list box.

Clicking on the light bulb menu icon will highlight in white the chart for the statistic that is currently selected. In the figure below, the Page/sec status would be displayed in white instead of blue. Use the up and down arrow keys to change which statistic currently has focus.

To change which statistics are being monitored right-click on the window and select “Properties.” Figure 17.5 is a screenshot of the Properties screen. To delete an existing statistic, select it and click the Remove button.

Some of the more useful counters in Perfmon are listed in the following table.

Adding a new statistic to the list of what is being monitored is relatively easy to do. Perfmon provides so many choices that it can be difficult to identify which one it is that you want. Figure 17.6 is a screenshot of the screen that the following steps refer to. The steps to add a statistic are:

Click the Add button

Select a value from the Performance object drop-down list box

Select one or more counters in the list box labeled “Select counters from list”

Select one or more instances from the list box labeled “Select instances from list”

Click the Add button

Repeat steps 1-5 for the next statistic that you want to be captured

Click the Close button

Click the OK button

It’s extremely useful to set up Perfmon to capture statistics and write them to either a log file or a database. For example, you can have statistics captured on a server constantly and have a new log file created every night at midnight. Then on the following day, you can review the statistics captured to see if anything unusual happened. To display statistics that were captured in a log file or a database, right-click on the main Perfmon screen area, select Properties, and click the Source tab. Figure 17.7 shows the screen from which you select the source of statistics.

To display statistics from a log file, follow these steps:

Click on the “Log Files” radio button option

Click the Add button

Select the log file that contains the desired stats

Click the Open button

Click the Time Range button

Click the Apply button

Figure 17.8 shows the statistics that were captured the previous day and written to a log file. The collection of statistics being displayed was defined to troubleshoot performance problems with a specific application. The statistics that are being collected fall into three categories:

The first measurement is the total CPU processing time expressed as a percentage

The next 3 measurements are the percentage of CPU time three specific processes consume

The final two measurements show busy the disk is

As this screenshot demonstrates, Perfmon allows you to capture statistics that are either generic to the server or specific to your environment.

4.1 Path Profiling and Trace Selection

One area in which hardware performance monitors and multicore processors have been leveraged is in path profiling. Some of the first research in this area was performed by Conte et al. [45]. In this work, traditional branch handling hardware was used to generate profile information in real time with an execution slowdown of only 0.4–4.6%. Ammons et al. [46] later analyzed how to improve the precision of reporting hardware performance metrics such as instructions executed, cycles executed, instruction stalls, and cache misses to more effectively identify paths through a program’s call graph. Other techniques have been designed to determine, through profiling, where program execution time is being spent [47]. Merten et al. [48] additionally explored using a branch behavior buffer to collect branch profile data for edge execution. Other research has used performance counters and monitors to predict phase and program path behavior based on observed events [49–51]. Profiling has also been performed for multithreaded programs using multicore systems [52].

Hardware mechanisms and multicore technology have also been used to form dynamic hot traces with low program overhead. Chen et al. [53] developed a technique to take advantage of the branch trace buffer of the Itanium to assist in trace selection. The Adore system [54] also proposes using the branch trace buffer to identify hot traces. Mars and Soffa [55] extended these techniques to exploit the multicore architecture to form higher quality traces without perturbing program execution.

Performance monitoring

As a Windows administrator, it is important that you monitor the performance of your servers. Performance monitoring should be done proactively and used to create baseline performance statistics for your servers. By establishing baselines for “normal” performance, you can locate performance issues more quickly by looking for deviations from the baselines established over time. Some performance statistics do have best practice results that have been established by Microsoft product groups; however, this does not negate the need for you to establish your own baselines.

Understanding Performance Monitor

Windows Server 2008 R2 includes the Performance Monitor utility to help administrators easily gather and analyze performance data. Using Performance Monitor, you can monitor and capture data from various counters provided by the operating system. Before using Performance Monitor, you should understand the following terms:

▪

Performance counters—Counters are the various components and objects that can be monitored using Performance Monitor. These are installed either as part of the operating system or by an application running on the server. Counters are also added when new roles are added. Examples of counters include % of Processor Time, Memory—Available Bytes, Logical Disk—% Free Disk Space.

▪

Instances—Instances allow you to view data more granularly from a specific counter. For example, you may want to use the Processor “% of Processor Time” counter to view processor utilization. You can use the instances option to limit viewing the utilization of processors 1 and 3 only or of all the processors in the server.

In the following exercise, we will go through the process of using Performance Monitor to view performance data in real time and start collecting data to establish a baseline for some key performance indicators.

Performance Monitor is located under the Diagnostics node in Server Manager. To open Performance Monitor, perform the following:

1.

Open Server Manager.

2.

Expand the node Diagnostics | Performance | Monitoring Tools.

3.

Select the node Performance Monitor (see Figure 12.1).

Figure 12.1. Windows Performance Monitor.

To monitor performance of a specific Windows component, you simply need to add it to the Performance Monitor main window. This is done by clicking the Add button at the top of the Performance Monitor window. This button is represented by a green plus sign as seen in Figure 12.2.

Figure 12.2. Add Performance counters.

After clicking the Add button, the Add Counters window will appear (see Figure 12.3).

Figure 12.3. Add Performance counters.

In this window, you can select the counters and instances you want to monitor. If you need a brief description of a counter, you can select the Show Description option. Table 12.1 lists some of the common objects and a brief description of what each object's counters captures:

Table 12.1. Common Performance Objects

Object	Description
Logical disk	Logical disk counters gather performance data related to logical disk drives (C:, D:, etc). These counters include % of Free Space, % Disk Read Time, % Disk Write Time, Average Disk Queue Length, and Free Megabytes. You will especially want to establish thresholds for free disk space counters to ensure you don't run out of free space on your server disk drives.
Memory	Memory counters allow you to monitor everything from available free megabytes to number or pages per second. Memory is often the number one bottleneck on servers so you should pay special attention to these counters. Since Windows servers page certain memory information to disk, you will want to ensure that excessive paging does not occur by monitoring Pages Per Second. You additionally will want to monitor available megabytes and committed bytes.
Processor	The processor performance counters monitor various aspects of processor performance. Though modern day processors are very fast, some processor intense applications such as SQL Server, can still see bottlenecks caused by poor processor performance. You will specifically want to monitor % of Processor Time. If you see a processor flat lined at 100% utilization, you know your server is very unresponsive and need to action immediately.
Physical Disk	Physical disk counters are used to measure the performance of the physical disk drive. Common counters include % Disk Read Time, % Disk Write Time, and Average Disk Queue Length.
Network Interface	Network Interface counters allow you to measure the performance of server network adapters including bytes sent or received per second, Current bandwidth, Packet Outbound Errors, and Packet Receive Errors. These counters can be helpful in monitoring utilization of the network adapter and troubleshooting adapter connectivity issues.

Notes from the field

Performance Monitoring management server

Performance Monitor can not only monitor the local server, but also has the ability to connect to remote servers and workstations to collect performance information from them as well. You may find it beneficial to set up a management server to centrally collect performance data from a set of servers opposed to running Performance Monitor on each system individually.

Now that you have an understanding of Performance Monitor, let us take a look at adding counters to be display performance information:

1.

Within the Performance Monitor window, click the Add button to open the Add Counters window.

2.

Select a counter you wish to monitor, such as the % Processor Time counter. Select the instances; in this case, we wish to monitor all the processors in the system so select All Instances. Then click Add. This will move the counter to the Added Counters pane as seen in Figure 12.4.

Figure 12.4. Select counter to view.

3.

Click OK. You will see a line graph with lines representing the percentage of utilization for each processor in the system (see Figure 12.5).

Figure 12.5. Processor utilization performance graph.

You can optionally change how you view the graph, using the change graph button (see Figure 12.6). You can choose between a line, histogram bar, or a table displaying values.

Figure 12.6. Change performances graph type.

Later in this chapter, we will explore using Data Collector Sets to capture performance data over a period of time.

Notes from the field

Where is the reliability monitor?

You may remember that Windows Server 2008 R1 included the Reliability Monitor that continuously monitored your system for reliability. This included critical system errors and configuration changes such as installing new applications. You will notice that the Reliability Monitor is no longer available in Server Manager in Windows Server 2008 R2. The monitor still exists and is disabled by default. To enable the reliability monitor, you will need to change the registry key HKEY_LOCAL_MACHINE\SOFTWRE\Microsoft\Reliability Analysis\WMI. Set WMIEnable equal to “1.” You will then need to ensure that the RacTask scheduled task is set to run. You can then access the reliability monitor from the Windows Action Center.

Using ISA Server 2004's Performance Monitor

ISA Server 2004 installs the ISA Server Performance Monitor (a customized view of the Windows System Monitor that includes only ISA Server-related counters) when you install the ISA Server 2004 software.

A number of counters are added to the Performance Monitor for the ISA Server Firewall Packet Engine object, the ISA Server Firewall Service object and the ISA Server Web Proxy object.

You can add or remove counters by right-clicking any column header in the bottom pane of the System Monitor view and selecting Properties.

The ISA Server Performance Monitor is configured in the same way as the Windows Performance Monitor, and you can create counter logs, trace logs, and alerts just as you do when monitoring other aspects of Windows computers.