Performance Testing, LoadRunner Tips and TricksPerformance Testing, LoadRunner Tips and TricksThis page is dedicated to performance testing using LoadRunner. Current development also includes tuning on various technologies and business perspective in load testing and usage of other load testing tools. Articles
LoadRunner TnT
2007-10-29 14:57:00 Just to bring to your attention that if you had not subscribed to the contents of my new site, now is the time to do so. I'm still using FeedBurner to deliver quality LoadRunner and Performance Testing content to you. I'm also including information to analyze, troubleshoot and tune whenever I have anything to share with you guys out there. So why wait any further if you are new to this field, take the opportunity to subscribe to the new site now! :) It's free anyway. :)Enter your email address:Delivered by FeedBurner
It's a brand new start for LoadRunner TnT
2007-09-06 13:58:00 Many thanks to all who had been following and supporting this website. The time to move to a bigger space has come and we are excited to announce the new website LoadRunner TnT is up and running.Log on to LoadRunner TnT now!The new site will continue to provide quality information on LoadRunner and Performance Testing to help newbies or mid-career performance tester in their work. Posting on this site will be discontinued but for those existing articles, they will be moved to the new home not shortly but surely. More About: Start , Brand
General: Load Testing Objectives
2007-08-27 03:40:00 There are two approaches from the user perspective on defining load testing objectives, either top-down (business perspective) or bottom-up (techincal perspective). The top-down, meaning the transaction time take to submit a request and having it responded on the user's machine. This is a business perspective where they just want to know how the application is responding. The user do not want to know how many Hits/sec or Throughput but just want to know how much time was taken for a transaction (therefore the terminology of "transcation" must be aligned with you and the user). The bottom-up, is a techincal perspective as you had pointed out such as the ASP.NET transactions or the database transactions (or commits). This is usually a persepctive from a technical manager so they can configure the system accordingly (or whatever they are going to do their system).Hopefully the above is helpful in understandin your users/clients a little bit more before giving them a satisfying an... More About: General , Testing , Load
Scripts: Parameterization
2007-08-21 14:14:00 Point to note that parameterization may result in data dependency which the tester should be aware of and handle it appropriately. Every load test should at least require parameterization. Parameterization includes passing different types of data into the application to emulate the real world users performing/entering different values. What do clients want to achieve from parameterization? To test different data Just to load/emulate the data Parameterization requires identifying with the clients which areas required data input. This will be useful when discussing in the Application Design. From there, they can prepare the data from the database which is a more efficient method. Preparing the data Usually after walking through the application, I will request the client to prepare the date via the database and export it out as an excel file. This will reduce the time in preparing lengthy excel or notepad files. From there, you can manipulate the data ea... More About: Scripts , Meter
Scripts: Any compatibility issues after upgrading LoadRunner versions?
2007-08-19 07:20:00 Personally, I encountered problem while porting the scripts recorded in 8.0 to 8.1 . This was actually pertaining to Citrix protocol where there is an additional parameter needed for the APIs. This will be particular to users that are using scripts in BPM (Business Process Management) of BAC (Business Availability Center). Take note that BPM uses the same recording tool, VuGen in LoadRunner, however at a version lower than the currently released LoadRunner Vugen).If I didn't recall wrongly, the Citrix API ctrx_sync_on_bitmap, had an additional parameter, "CTRX_LAST" at the end of its parameter list in 8.1 as compared to 8.0. Since then, I haven't had any problem with other protocol scripts so far.Anyway, my advice are as the followed:Read the upgrade readme file.Clone an image prior the upgrade. In this way, you can revert back to the older version.Test the previously recorded scripts from the older version.Do you have any incompatibility issues of scripts before? Feel free... More About: Issues , Scripts , Compatibility , Compa , Version
Understanding Network: Performance Measurements
2007-08-15 07:05:00 Evaluating path performance means doing three types of measurements. measurements will give you an idea of the hardware capabilities of your network, such as the maximum capacity of your network. BandwidthThroughput measurements will help you discover what capacity your network provides in practice, i.e., how much of the maximum is actually available. Traffic measurements will give you an idea of how the capacity is being used. Performance Measurements Two factors determine how long it takes to send a packet or frame across a single link. The amount of time it takes to put the signal onto the cable is known as the transmission time or transmission delay. This will depend on the transmission rate (or interface speed) and the size of the frame. The amount of time it takes for the signal to travel across the cable is known as the propagations time or propagations delay. Propagation time is determined by the type of media used and the distance involved. Once... More About: Network , Standing , Under
Understanding Network: How traceroute works?
2007-08-12 15:51:00 The program was written by Van Jacobson and others. It is based on a clever use of the Time-To-Live (TTL) field in the IP packet’s header. The TTL field is used to limit the life of a packet. When a router fails or is mis-configured, a routing loop or circular path may result. The TTL field prevents packets from remaining on a network indefinitely should such a routing loop occurs. A packet’s TTL field is decremented each time the packet crosses a router on its way through a network. When its value reaches 0, the packet is discarded rather forwarded. When discarded, the ICMP TIME_EXCEEDED message is sent back to the packet’s source to inform the source that the packet was discarded. By manipulating the TTL field original packet, the program traceroute uses information from these ICMP messages to discover paths through a network. Trac eroute sends a series of UDP packets with the destination address of the device you want a path to. * By default, traceroute sends sets... More About: Network , Works , Standing , Under
Understanding Network: How Ping Works?
2007-08-09 05:28:00 One network device sends a request for a reply to another device and records the time the request was sent. The device receiving the request sends a packet back. When the reply is received, the round-trip time for packet propagation can be calculated. The receipt of a reply indicates a working connection. This elapsed time provides an indication of the length of the path. Consistency among repeated queries gives an indication of the quality of the connection. Thus, ping answers the two basic questions. Do I have a connection? How good is that connection? Clearly, for the program to work, the networking protocol must support this query/response mechanism. The ping program is based on Internet Control Message Protocol (ICMP), part of the TCP/IP protocol. ICMP was designed to pass information about network performance between network devices and exchange error messages. It supports a wide variety of message types, including query/response mechanism. The normal operatio... More About: Network , Works , Ping , Understand , Standing
General: Planning for Load Testing - Recommendations
2007-08-07 03:58:00 After identifying trends, it’s time to propose an action plan, proposal or recommendation to your client. This maybe just telling the client that “Dear client, your application is running at this rate, I would recommend to re-look the SLAs with your users.” or “Dear client, the server housing the application is encountering processor contention. I would like to recommend you to distribute the load across with a new server (or processor).” This is of course tied to the initial requirements and the limitations the clients are bounded in. At this point if they have follow up actions to improve the system, do keep a record of the last changed parameter on the server. This will be useful for you to track the changes over time on their application and determined if the changed had enhanced the application. Unless you are part of the server team, you might not be involved in the tuning process. Once the changes have been made, conduct a load test to verify the chan... More About: General , Planning , Testing , Load , Recommendation
General: Planning for Load Testing - Analyzing
2007-08-05 06:37:00 This is the stage where it intertwined with the monitoring. At this stage, similarly to Monitoring, it is important to have a sound knowledge of the server or database internal workings. With that you will know what to further investigate and what the graph represents. A graph looks great with the lines flying up and down but that is information. You will need to have a lot of experience to translate that information to useful knowledge. This will be done with merging and correlating of the load generated against the utilization of resources. You must be able to “see” the trend and identify possible bottlenecks.I would recommend acquiring lots of information of the server (either the software or hardware perspective) you are monitoring as this is required for your analyzing. You can refer a previous consolidated article, "Content Page - General " which provides a proportion of basic information that will be useful to you in facilitating the analyzing.Other than the knowl... More About: Planning , Testing , Load , Anal
General: Planning for Load Testing - Monitoring
2007-08-03 04:01:00 I separated the Monitoring from the Monitor Setup because I feel that the setup is prior the load test runs. In this stage, what we are concerned are the counters and metrics to monitor. If you are starting off without any knowledge or history of the system such as performance issues, it’s advisable to monitor the 4 main categories, namely Processor, Memory, Disk and Network. Once the load test completes, you should have an overview of the application performance. From there, go to a 2nd level of monitoring till the nth level until you are satisfied with the investigation.For example, on the 1st load test, you observed that the page faults are occurring at a level higher than the accepted threshold indicating memory problems. From there, you may want to investigate further on the cause of the page fault such as, if it is a hard page fault or soft page fault. Therefore, you may like to monitor the Transition Faults, Pages Input and Pages Output to answer your doubts. Of cour... More About: General , Planning , Testing , Load
Monitors: Memory Glossary
2007-08-01 07:33:00 This is a compilation of memory-related counters. Do feel free to revert or comment should you have additional information on the counter.Available BytesProcess Working Set growth becomes constrained when Available Bytes Pool Nonpaged BytesThe system’s non-pageable (fixed) memory.Pool Paged Resident BytesThe OS‘s pageable memory that is currently resident in RAM.System Code Resident Bytes[Coming soon]System Driver Resident BytesTotal System Resident Bytes = Pool Nonpaged Bytes + Pool Paged Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes + System Cache Resident Bytes.System Cache Resident BytesThe current amount of RAM used for the file cache.Page Faults/secCan be a grossly misleading number. Page Faults/sec = “soft” Transition Faults + application file Cache Faults + demand zero faults + hard page faults.Page Reads/secThis counter measures the number of requests to the I/O manager to retrieve pages of memory from the disk. Despite the name of thi... More About: Memory , Glossary , Monitors , Gloss
General: Planning for Load Testing - Monitor Setup
2007-07-30 16:25:00 The setting up of monitors poses some challenge and I advised you to cater one or two days to implement the monitors. LoadRunner features agentless monitoring (which other products do too). However, in order to achieve such feature it needs the SUT (System Under Test) to be configured properly. This “configure properly” becomes sort of “out-sourced” to the system admin team and if problem arises, the blame of product issue will be reduced. Depending on the type of monitors you are implementing, you will require time and effort to clear ports and justify your case to the security or the network team before the actual implementation is possible. This is where the initial gathering of information is important in Soliciting Requirements. With the architecture diagram, you can relate to the security or network team on the team of installation of agents or opening of ports. But of course this will come under scrutiny if you do not provide sufficient convincing info... More About: General , Planning , Testing , Setup , Monitor
General: Planning for Load Testing - Protocols
2007-07-25 04:04:00 Load testing is about capturing the traffic sent by the application to the server. And using the captured traffic, the tool scripts out APIs to generate the traffic, which is been emulated during replay and sent back to the server. Through understanding the Application Design, you should be able to determine the type of protocol that is been communicated. As I pointed in previous articles , LoadRunner focuses a lot on protocol and therefore the need to know this information is critical. To be able to read the logs and find meaningful information out of it, I would recommend you to acquire as much knowledge as possible. For example, HTTP, the replay log merely displays the transmission of the HTTP requests in text. From there, you will know information such as authentication headers or the body that was sent. I cannot emphasize again that do not associate functional testing as load testing. Therefore, if you are from the functional testing background, please be sure to acquire more k... More About: General , Planning , Testing , Load , Protocol
Understanding Memory: Virtual Addressing (Illustration)
2007-07-23 03:26:00 The following illustration explains the process of virtual addressing when a program is loaded into memory. For detailed explanation, refer to "Understanding Memory : Virt ual Address ing".Executable program image file loads into memory.Logical memory address range of the application is divided into fixed-size chunks caled pages.Page mapping in Virtual Memory is dynamic.Frequently referenced logical addresses tend to reside in physical memory.Infrequently referenced pages are relegated to paging files on secondary disk storage.Related TopicsContent Page - General Virtual AddressingPage Fault Resolution Page Fault Resolution (Illustration ) Performance Concerns Virtual Memory Shortage Alerts Available Bytes LRU System Working Set Detecting Memory Leaks Measuring Memory Utilization
General: Planning for Load Testing - Application Design
2007-07-22 11:37:00 This is the point that the load tester must understand how the application works. When we refer to how it works, it’s not about just clicks on links and buttons, or the calendar will pop up. It goes deeper such as the following but not limited to the list: What is the type of communication between the client and server?Is the application client-side?Is there client-side activities involved in the entire application?What is the authentication mechanism?Does the application allow multiple login of a single user?How does the application maintain a session?How does the load balancer distribution the load?What are the parameters been sent to the server? Basically, this is actually gathering the technical details of the application unlike the initial Soliciting Requirements which comprises of the “big picture” information. Without knowing how it works and you start clicking the links and buttons, you are bound for a failing load t... More About: Design , General , Planning , Testing , Load
General: Planning for Load Testing - Soliciting Requirements
2007-07-20 18:53:00 Prior to this stage, I always give a “sales talk” of what we do for a living and also to get them excited (if they are not). Gathering and soliciting accurate requirements will save you lots of unnecessary work. Be sure to get hold of the requirements to monitor such as acceptable Transaction Response Time or server utilization. Clients need to be clear of what they want before you can proceed to achieve (unless is you who proposes the benchmark). For example, if CIO wants the response time for a search to be fewer 10 seconds will be a good metric to achieve. This is also the exit criteria as per se for the load test project. They may also like to know the utilization of the servers under load and not really concerned with the response time. Usually, these are applications that are already problematic and it’s what they are hiring you to dig the cause of the bottleneck. Always get an overview of the architecture which includes all components that inhabitant ... More About: General , Planning , Testing , Load , Soli
General: Planning for Load Testing
2007-07-19 05:01:00 After writing a couple of posts for the past few months, I would like to touch on the basic understanding of using Load Runner in Performance Testing . This will provide an overview how you should facilitate the entire load testing process and be aware of the requirements that surfaces. If course, this is not limited to other load testing tools because the concepts are the same.There are few areas to be aware or trained in order to have the load test proceed with minimal difficulties. I’ve broken them down into the following: namely, soliciting requirements, application design, protocols, monitor setup, monitoring, analyzing and recommendations. Soliciting RequirementsApplication Design [Coming soon]Protocols [Coming soon]Monitor Setup [Coming soon]Monitoring [Coming soon]Analyzing [Coming soon]Recommendations [Coming soon] Related TopicsGeneral - Content Page More About: Planning
Understanding Memory: Virtual Addressing
2007-07-15 05:55:00 Virtual memory is a feature supported by most advanced processors. Hardware support for virtual memory includes hardware mechanism to map from logical (virtual) memory address that application programs reference to physical (real) memory hardware addresses. When an executable program’s image file is first loaded into memory, the logical memory address range of the application is divided into fixed-size chunks called pages. These logical pages are them mapped to similar-sized physical pages that are resident in real memory. This mapping is dynamic in that frequently referenced logical addresses tend to reside in physical memory (also know s RAM, real memory, or main memory), while infrequently referenced pages are relegated to paging files on secondary disk storage. The active subset of virtual memory pages associated with a single process’s address space currently resident in RAM is known as the process’s working set, because they are the active pages referenced by the pr... More About: Memory , Virtual , Virt , Address , Understand
General: Understanding Memory - Page Fault Resolution (Illustration)
2007-07-15 05:15:00 Illustration of Page Fault Resolution. To better understand how Page Fault occurs with respect to the illustration, refer to article, "General : Understanding Memory - Page Fault Resolution". Fig 1: Page Fault ResolutionThe sequence of Page Fault resolution as follows.Thread attempts to reference page in memory.Page not resident in real memory thus not found.Hardware interrupt occur to resolve page fault.ISR gains control to validate the referenced address.Locate page on secondary storage.Copy page into available free page in real memory.Resume thread execution cycle.The time taken for Page Fault Resolution thus equivalent from Step [3] to [6]. Having mentioned that, if memory is the performance bottleneck with the number of Page Faults increases over time, the application/system may experience delay due to the time needed to perform Page Fault Resolution pointed in Step [3] to [6].Related TopicsPage Fault Resolution Performance Concerns Virtual Memory Shortage AlertsAvailable B... More About: Illustration
Basics: Network Bottlenecks
2007-07-14 17:18:00 At the network level, many things can affect performance. The bandwidth (the amount of data that can be carried by the network) tends to be the first culprit checked. Assuming you have determined that bad performance is attributable to the network component of an application, there is more likely cause of bad network performance than network bandwidth. The most likely cause of bad network performance is the application itself and how it is handling distributed data and functionality.The overall speed of a particular network connection is limited by the slowest link in the connection chain and the length of the chain. Identifying the slowest link is difficult and may not even be consistent: it can vary at different times of the day or for different communication paths. A network communication path lead from an application through a TCP/IP stack (which adds various layers of headers, possibly encrypting and compressing data as well), then through the hardware interface, through a... More About: Network , Bottle , Basics , Lene , Neck
Tools: Monitoring Unix Resource Script ver 1.0
2007-07-11 03:23:00 Due to the past experiences working in my organization, having the problem of not able to monitor the Unix system resources due to whatever reasons (rstatd configuration, dynamic ports, security reasons or lazy administrators) had always been bothering us.Our team came up with the simple initiative to monitor using the same concept as perfmon. We requested (outsourced) one of our interims to search for unix monitoring scripts and modify it to collect data periodically. And here we are, special thanks to the interim, we finally got the first-cut of the scripts.Copy and paste the below script starting from here ========================================= =========================#!/bin/bash# (C) 2006 Mark Boddington, http://www.badpenguin.co.uk/# (C) 2007 Modified by Hwee Seong, http://loadrunnertnt.blogspot.com# Licensed under the GNU GPL Version 2.# ***** Configuration *****# set LOG to the directory you want to write the performance data to.# set HDD to the number of had disks in your... More About: Tools , Script , Resource , Monitoring
Scripts: Starting a new transaction during iterations
2007-07-09 07:45:00 Recently one of my load testing projects required us to monitor the time taken to upload a file into the application. This sounds simple and straight forward with the implementation of the lr_start_transaction and the lr_end_transaction. However, the catch is that the action block be iterated 10 times and within the 10 iterations, one of the iteration is to upload a different file of different size (thus causing different response time), which they want to measure. What we did was implement a if-then-else to check the file and perform a lr_start_transaction when the script is uploading the different file and end the transaction with the lr_end_transaction when the submission completes. The snippet of the code is as followed:if (!strcmp(lr_eval_string("{file}"), lr_eval_string("{set_file}"))){ lr_start_transaction("tx_check_upload_dif ferent_file");}else{ lr_start_trasaction("tx_tx_check_same_upl oad_file");}...... other codes......if (!strcmp(lr_eval_string("{file}"),lr_eval _... More About: Scripts , Tera , Tran , Ansa
Basics: Memory Bottlenecks
2007-07-08 10:02:00 Maintaining watch directly on the system memory (RAM) is not usually that helpful in identifying performance problems. A better indication that memory might be affecting performance can be gained by watching for paging of data from memory to the swap files. Most current OS have a virtual memory that is made up of the actual (real) system memory using RAM chips, and one or more swap files on the system disks. Processes that are currently running are operating in real memory. The OS can take pages from any of the processes currently in real memory and swap them out to disk. This is known as paging. Paging leaves free space in real memory to allocate to other processes that need to bring in a page from disk. Obviously, if all the processes currently running can fit into real memory, there is no need for the system to swap out any pages. However, if there are too many processes to fit into real memory, paging allows the system to free up system memory to run more processes. Pa... More About: Memory , Memo , Bottle , Basics , Lene
Products: SilkPerformer (a comparison with LoadRunner)
2007-07-06 10:21:00 SilkPerformer is a load testing tool from Borland, originally Segue. SilkPerformer have almost all features that LoadRunner have. Therefore, you may like to read about the product LoadRunner here before proceeding further on this article. The SilkPerformer suite consists of the Workbench, Silk TrueLog and Silk Performance Explorer. The entire load test implementation is the same where the load generators have to be installed at the machines. For the sales talk by the vendor, click here.What SilkPerformer offer is it handles load test in project approach. Using this approach, each load test is stored as a project with various settings of profiles and workloads. This is advantageous when managing the load test in a customer or project perspective where you can track the projects accordingly. Furthermore, within a project, for every load test run, a result directory for each run is created. This is useful in tracking the number of runs that were conducted. In this way, there is a struc... More About: Products , Comparison , Riso , Prod , Silk
Basics: Disks Bottlenecks
2007-07-04 14:46:00 In most cases, applications can be tuned so that disk I/O does not cause any serous performance problems. But if, application tuning, you find that disk I/O s still causing a performance problem; your best bet may be to upgrade the system disks. Identifying whether the system has a problem with disk utilization is the first step. Each system provides its own tools to identify disk usage (Windows has a performance monitor, and UNIX has the sar, vmstat, iostat utilities.) At minimum, you need to identify whether the paging is an issue (look at disk-scan rates) and assess the overall utilization of your disks (e.g. performance monitor on Windows, output from iostat –D on UNIX). It may be that the system has a problem independent of your application (e.g. unbalanced disks), and correcting this problem may resolve the problem issue.If the disk analysis does not identify an obvious system problem that is causing the I/O overhead, you could try making a disk upgrade or a reconfiguration.... More About: Bottle , Basics , Lene , Neck
Basics: CPU Bottlenecks
2007-07-01 05:32:00 Java provides a virtual machine runtime system that is just that: an abstraction of a CPU that runs in software. (Note that this chapter is taken from the "Java Performance Tuning" written by Jack Shirazi and therefore alot of discussions circled around Java technologies.) These virtual machines run on a real CPU, and in this section the book discuss the performance characteristics of those real CPUs.CPU LoadThe CPU and many other parts of the system can be monitored using system-level utilities. On Windows, the task manager and performance monitor can be used for monitoring. On UNIX, a performance monitor (such as perfmeter) is usually available, as well as utilities such as vmstat. Two aspects of the CPU are worth watching as primary performance points. These are the CPU utilization (usually expressed in percentage terms) and the run-able queue of processes and threads (often called the load or the task queue). The first indictor is simply the percentage of the CPU (Or CPUs) being... More About: Bottle , Basics , Lene , Neck
Basics: Underlying Operating System and Network Improvements
2007-07-01 05:11:00 If you control the OS and hardware where the application will be deployed, there are a number of changes you can make to improve performance. Some changes are generic and affect most applications, while some are application-specific. This article applies to most server systems running Java application, including servlets, where you usually specify (or have specified to you) the underlying system, and where have some control over tuning the system. Client and standalone Java programs are likely to benefit from this chapter only if you have some degree of control over the target system, but some tips in the chapter apply to all Java programs.It is usually best to target the OS and hardware as a last tuning choice. Tuning the application itself generally provides far more significant speedups than tuning the systems on which the application is running. Application tuning also tends to be easier (though buying more powerful hardware components is easier still and a valid choice for tuni... More About: System , Operating System , Network , Basics , Rove
Basics: Max. Running Vusers in Analysis is different from Vuser Quantity in
2007-06-29 03:09:00 There are couple of times when an individual starts to pick up LoadRunner ask this question. I've written the explanation here for the convenience of the newbies which I had previously responded in the Yahoo! group discussion. Max. Running VuserIt defines at any point of point of time, the maximum number of vusers running together concurrently (in Run state). This is the "state" or usually the requirement of a load test to reach "X" number of concurrent users. If a load test were to required to run 100 concurrent users, then the Max. Running Vuser must be 100. This is different from Vuser Quantity explained in the following.Vuser Quantity In the Controller, the Vuser Quantity is the total number of participating in the load test but it is different from Max. Running Vuser which is explained above. To view the total number of Vuser participating the load test (Vuser Quantity), open the Vuser Summary graph. Please take note that the Summary Report in the Analysis session displays t... More About: Users , Anti , Rent
Protocols: Java Error - java.lang.NoClassDefFoundError
More articles from this author:2007-06-28 11:42:00 I had a question from a visitor with regards to a Java error. The error message is as followed.java.lang.NoClassDefFoundError : jhook/HookMetaData at java.rmi.Naming.lookup(Naming.java) at pie.engine.atp.ClientRMI.getRemoteModuleI mpl(ClientRMI.java:256) at pie.engine.atp.ClientATPRemoteEx.getRemot eModule(ClientATPRemoteEx.java:89) at pie.engine.atp.ClientATPRemoteEx.getRemot eModule(ClientATPRemoteEx.java:64) at pie.engine.atp.VirtualRemoteServer.setupP ingRemote(VirtualRemoteServer.java:102) at pie.engine.atp.VirtualRemoteServer.pingSe rver(VirtualRemoteServer.java:166) at pie.engine.atp.CacheTimer$1.run(CacheTime r.java:47) at java.util.TimerThread.mainLoop(Timer.java :432) at java.util.TimerThread.run(Timer.java:382) I did not have straight answer for him initially but, he was able to resolve the problem by himself with the following command script (inclusion of the Xbootclasspath).-Xboot... More About: Under , Protocol , Toco 1, 2, 3 |
|
 |
|
 |
