Common Performance Testing Mistakes

Performance engineering can be a challenging field to get into. But it’s a very lucrative career, and people shouldn’t be too scared to get involved. If you’re a test engineer and are looking to learn performance testing, be assured that it’s a great career that’s going to open a lot of doors for you.

However, if you’re just getting started there are a host of common issues that most newbies (and some more seasoned performance testing engineers) encounter.

Check More than the Status Code

When you’re performance testing and automating a transaction, one of those common mistakes occurs when an http status code of 200 comes back. You might think everything is ok at this point but what happens if it was a login test and your login script enters a bad password—basically you would be it redirected you back to the login page.

That login page was delivered. It was http status code of 200. However, that transaction was not successful.

To avoid this mistake, you should definitely place validations or assertions — which are basically string verifies and responds — to ensure those transactions are indeed successful.

Believe it or not, it happens more often than you would think. For example, i heard about a  performance test was against a huge e-commerce company.

Everybody was high-fiving when the performance test reached 40,000 current users without any issues. As it turned out, however, none of the user were actually logged in! They had forgot to make sure the users were successfully logged in before they proceeded with the rest of the scripts and transactions.

Remember to always have your validations and assertions in the right place to avoid this rookie mistake.

Think Time Matters

Another common performance testing mistake has to do with using improper pause and think times. People forget either forget to add them, or else they use two milliseconds. You really need to create a realistic performance test scenario that emulates how a real user is going to interact with your application.

You should look at the tool and say, “Okay — is it going to take five seconds to make a decision on this page? Is it going to take a minute to read this article?”

This can also cause people to create unrealistic results that will in turn cause the team to panic for no reason, so avoid this mistake at all costs.

Don’t Cry Wolf

Crying wolf when you spot a bottleneck will get you into trouble fast, and often happens when you isolate a symptom instead of the root cause. This is one of the performance testing mistakes that drive me crazy.

This goes back to more methodical performance testing, where you create a test that ramps up more slowly so you can see the trends and throughputs, and can see the servers becoming busier and busier. You can tell what server becomes saturated first, and rather than saying, “This is now at 100% use,” or, “This is out of memory,” you should be aware that these could just be symptoms.

False Assumptions

During the running of a test, the tool is doing its job. It’s collecting the KPIs and the data. When it’s presenting that data to you, however, it can look a little skewed.

In order to identify trends and really understand what’s going on, you should let the test run to completion. If you get a very high error rate after completion, that’s the time when you should stop it, then go back and do your analysis.

Analysis is actually one of the most important aspects of a performance test — but it’s also one of the most difficult tasks.

Analysis is Important – Wait Until the Test is Done

If you attempt to analyze a test and arrive at conclusions while it’s still running, more often than not you’ll only be wasting your valuable time

Beware of Sinkholes

Back when I was doing mainly performance testing, I would sometimes go to the extreme; I would say to the team, “I’m not sharing any results until I get three runs and I’m able to analyze them.”

Most people don’t understand the performance engineering process; they want results immediately.

Run a test at least three times before reporting the results, because don't waste folks time on anomalies that aren’t reproducible. More often than not, that’s where the sinkholes are.

Newbie performance engineers may find themselves spending hours and hours researching an issue when it was simply a fluke that happened once and wasn’t reproduced.

More PerformanceTesting Awesomeness

These are just a few of the most common performance testing mistakes most newbie testers make.