Best Performance testing tool I have ever used
What do you like best about the product?
GUI and features, users interfaces, load test and performance test
What do you dislike about the product?
Nothing to dislike I liked all features, especially increasing and decreasing the loads
What problems is the product solving and how is that benefiting you?
To test the performance of cloud applications
LoadRunner Experience Insight
What do you like best about the product?
The UX is exceptional and it is beginner-friendly
What do you dislike about the product?
Its integration to the cloud is kind of technical
What problems is the product solving and how is that benefiting you?
I was able to experience and utilize a new tool for load testing and it has helped in my work. It has given me the result that I needed
Load runner cloud
What do you like best about the product?
Navigation across different modules and connectivity
What do you dislike about the product?
Data parsing and some issues with flat files
What problems is the product solving and how is that benefiting you?
Enterprise Applications performance
Excellent tools for the Devops pipeline - Testing
What do you like best about the product?
Ease of use, integration, scalability, deployment, delivery
What do you dislike about the product?
there is nothing I dislike in this product
What problems is the product solving and how is that benefiting you?
We have integrated for testing in our DevOps pipeline using Jenkins. We have realized 30% TCO due to the automation
Recommendations to others considering the product:
Definitely, I will recommend this product for Integration in DevOps pipeline
Best for performance & load testing
What do you like best about the product?
Easy to use. Easy interface. Large community support. Scope & opportunity is also huge. Market leader.
What do you dislike about the product?
It's not not open source. You need to pay license fee. There is trial with limited features. Only for non functional support.
What problems is the product solving and how is that benefiting you?
Helps to test load & performance of application. Which finally help to identify & mange end application.
It helps much. It creates virtual user which s similar to real user.
Automatic trigger
What do you like best about the product?
Produces reports and automatically triggers.
What do you dislike about the product?
Few limitations to desktop applications.
What problems is the product solving and how is that benefiting you?
No need to worry about load generators.
Short and sleek tool for load testing purposes
What do you like best about the product?
This software helps our engineering team to test the application at times of heavy load. Not only that, its integration with other software is really effortless. Once the application is up and running and the developers are well versed with how to use the application, this application is very useful for testing the application with a heavy load. We load tested the applications manually before using this software but it was time-consuming and ineffective at one and the same time. Moreover, manual testing could not replicate the almost real traffic that the application is going to handle in the recent future. This software exposes the vulnerabilities at quite an early stage of development.
What do you dislike about the product?
The learning curve is very steep and there is a dearth of online resources to take guidance.
What problems is the product solving and how is that benefiting you?
It saves a huge amount of time for the developers and testers as they don't have to generate a large amount of load to test the application under pressure. This application does that efficiently without any hiccups. The time saved can be utilised by the developers to improve the application and develop other complex meaningful business logic.
Recommendations to others considering the product:
It helps to test the application in troubled waters. This application does not seem useful at good times but when the traffic increases beyond a limit, then the testing capability of the software is recognised and rewarded. The strength of any application is tested when it is able to handle heavy traffic.
Load testing made easy
What do you like best about the product?
all the scripts and test results are maintained on cloud. it saves the resources.
What do you dislike about the product?
technical challenges for loadrunner on cloud.
What problems is the product solving and how is that benefiting you?
easy access across geographic locations. saves the reource
Recommendations to others considering the product:
try this and you will benefit from this.
Offers good record and playback and endurance features
What is our primary use case?
It is all about load testing. We have to do a lot of load testing in our IT environment. In the investment banking space, you do a lot of end-to-end endurance testing, to see how to benchmark performance based on resources and the CPU memory. So, we will have to benchmark the performance based on trade volumes. Trade volumes are the basic KPI.
Endurance testing needs to be done to see how many trades can be pumped in my environment. I do endurance testing.
How has it helped my organization?
better decision making was done Business KPI & technical KPI can be set by automating loy of endurance testing where given resources thresholds capacity can be measured against business kpis
What is most valuable?
The record and playback feature is the most valuable feature. It's all driven by the script, so it's a script-based tool where the background tracing starts. Java's background process does a lot of tracing. The process starts in the background. It sees what peaks of volumes the process can handle. It's easy to use because it's script based, record and playback. I play certain tests. I record those tests, see how they pass through the entire environment, and see how many breaks I get.
Then we repeat that for multiple numbers. If the first time, I pumped in 1 million trades, and get 5% of the failures, then I try with 2 million trades and I see that there is a 7% failure. These scenarios are easy to do in the LoadRunner. LoadRunner has been the best tool for more than 20 years in the market.
You can do endurance testing, you can do load testing. These are the different types of load testing. It lets you see how much volume you can benchmark in your entire environment, given the resources like CPU and RAM.
The record and playback is the best feature. You need to use a console. You log in to your console and see the dashboard with how many trades failed and how many trades passed, what the peak loads endurance load were.
What needs improvement?
The performance has really improved in terms of running test cycles. The product used to crash on-premises and when it had a lot of trades being pumped in. Because it is memory intensive, it used to crash if it was running out of memory. That was the limitation of the on-premises thing. Running those states and cycles in the cloud is much faster.
Everything is frozen in the cloud. The RAM, CPU, compute, and storage are provisioned in the cloud, which is becoming easier for running these test cycles. Test cycles are highly effective. Of course, you need to have a test strategy, like volume-based load testing. Configure some test cases and run those test cases in cycles. Cloud performance is much faster. Volume-based endurance testing is easier in the cloud.
For how long have I used the solution?
It is an old solution, around 20 years old. It was called once Mercury LoadRunner and then it was acquired by HP, it became HP QTP. That was around 2007, it was known as Hewlett-Packard HP QTP. Since then it has been acquired by OpenText.
What do I think about the stability of the solution?
What do I think about the scalability of the solution?
We have around 100 users, they are functional experts who use this for load testing.
The people who have been responsible for setting the KPIs will decide if we will expand the solution. They are the end-users. It will depend on the dashboard and the results of the load testing.
How are customer service and support?
LoadRunner is a very old product in this industry. It was a little difficult to get to the right support team. Now there's a professional service group that is better than they were before.
If scaling becomes a problem, then we have to involve the support, the right kind of support to actually fine-tune and troubleshoot the entire configuration. But with the cloud version, that risk is minimized. It has adequate storage available.
What's my experience with pricing, setup cost, and licensing?
It's a subscription-based, pay-per-user license. I wouldn't say it's expensive.
Because it was an on-premise license, it was a costly affair. Only certain stakeholders could use it. Now, it is enterprise-wide, we can have any number of cloud users performing the load testing because it is pay-per-user.
What other advice do I have?
I would recommend LoadRunner. It is an industry-leading tool. You can benchmark the volumes on the dashboard and it is easy to use.
I would definitely recommend it for investment banks and certain businesses like OTC derivatives.
I would rate LoadRunner an eight out of ten. It has good documentation. The recording and playback scripts can be easily done. I would like for there to be better integration with other tools so that when you do load testing you can also do a security check.
Which deployment model are you using for this solution?
Public Cloud
The best feature is that we don't have to build and maintain infrastructure anymore
The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.
In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.
What is most valuable?
StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.
What needs improvement?
More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.
Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.
There is a steep learning curve for the product, too.
What do I think about the stability of the solution?
Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.
What do I think about the scalability of the solution?
We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.
At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.
How is customer service and technical support?
I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.
Which solutions did we use previously?
I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.
How was the initial setup?
I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.
Which other solutions did I evaluate?
The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.
What other advice do I have?
I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.
For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:
* Is the application under test? Is it customer facing?
* Is how often do you test it?
