Core Performance Engineering (LoadRunner Cloud)

In my daily use case for OpenText LoadRunner Cloud , I am using the LRE version R2 2023, utilizing four load generators, and while it is okay, sometimes I face issues with the script where the support team takes more time to resolve those issues.

The most useful features of OpenText LoadRunner Cloud  include getting load through the Performance Center , which is part of the cloud version, where we can monitor CPU and memory utilization, and response times, but we do face configuration issues related to network discrepancies.

I am satisfied with OpenText LoadRunner Cloud as a product, but the ticket resolution time is concerning. The technical personnel are not able to fix issues quickly, which becomes problematic during critical situations.

Compared to previous support, I notice that while experts previously resolved issues immediately, current experts take more time to resolve issues, which is the main challenge we are facing.

They are now lacking regional support, which takes more time than it used to.

My suggestions for improvements to OpenText LoadRunner Cloud would be to have specific experts available who can resolve issues more quickly, as delays can impact project timelines significantly.

I have been working with OpenText LoadRunner Cloud for around four years.

OpenText LoadRunner Cloud can handle a decent amount of tests simultaneously, but as a small company, we primarily test 1,500 concurrent users and sometimes face issues such as load balancer problems.

Support-wise, I would rate OpenText LoadRunner Cloud a six out of ten.

I faced issues with OpenText LoadRunner Cloud support when a problem took three to four months to resolve, which negatively impacted our project, especially when key team members were unavailable during leave periods.

Neutral

Regarding my experiences with the initial setup and deployment of OpenText LoadRunner Cloud, everything is working as expected, but upgrades can take time, and I often encounter correlation issues that require debugging.

Currently, I am exploring other tools besides OpenText LoadRunner Cloud, such as NeoLoad , but my company is hesitant to invest due to the need to evaluate the advantages of LoadRunner  and the associated license costs.

LoadRunner  was chosen over tools such as NeoLoad  because, despite NeoLoad being a good option for maintenance, we face script maintenance issues in OpenText LoadRunner Cloud, which often leads to delays.

I am describing the support of OpenText LoadRunner Cloud regarding LoadRunner. In LoadRunner, I actually do not use real-time analytics. There are more monitoring tools available, but until now, we are just using the Performance Center  for monitoring and integrating with SolarWinds, while also considering integrating with Prometheus for real-time log checks.

I have not used OpenText LoadRunner Cloud with a CI/CD pipeline yet, but I think it would be beneficial to implement.

As for the integration needs of OpenText LoadRunner Cloud, I am considering Prometheus because I am not doing real-time monitoring at present. The DBA team handles the SolarWinds part, and we require reports based on load testing duration.

I rate OpenText LoadRunner Cloud a seven out of ten.

Hybrid Cloud

Other

I use OpenText LoadRunner Cloud  primarily to run scripts and tests within our performance testing workflows.

OpenText LoadRunner Cloud ’s features are technically similar to LoadRunner  Enterprise or the standalone controller. However, a significant difference is in its depth of analysis. In the standalone LoadRunner  analysis, I can merge graphs or set granular filters to obtain custom reports. Currently, LoadRunner Cloud provides direct custom reports, a feature that, if improved, might enhance our analysis. Another valuable aspect is the option to use JMX files, similar to JMeter  scripts, with LoadRunner.

The analysis feature in OpenText LoadRunner Cloud requires improvement. In-depth analysis tools found in the standalone LoadRunner analysis, such as graph merging and setting granularity, would be beneficial. Additionally, handling JMX scripts in LoadRunner Cloud could be more comprehensive. It would also be convenient if there were options to convert scripts from competitor tools like NeoLoad  to LoadRunner.

I have been using OpenText LoadRunner Cloud for about one and a half years. Initially, I used it a year ago, had a gap, and then started using it again in the last six months.

There is no difficulty in deploying OpenText LoadRunner Cloud from my perspective because a specific team in our organization handles the deployment of LoadRunner to our systems. We only use it once deployed.

OpenText LoadRunner Cloud is extremely stable for our use case, and I would rate it a nine out of ten.

OpenText LoadRunner Cloud is absolutely scalable. With load generators available, it is easily scalable to meet our needs.

I have not interacted directly with OpenText  support. However, based on feedback from colleagues, the support could be faster.

Neutral

I have used NeoLoad  in past projects, but currently, I use only OpenText LoadRunner Cloud.

Our own organization has a team responsible for deploying all software, including OpenText LoadRunner Cloud. We did not rely on third-party services for deployment.

I have evaluated NeoLoad in my previous work assignments.

I would appreciate documentation or some links to videos on AI-driven testing strategies with OpenText LoadRunner Cloud. I rate OpenText LoadRunner Cloud ten out of ten.

Private Cloud

Microsoft Azure 

Our clients use OpenText LoadRunner Cloud  to ensure performance by running load tests. OpenText LoadRunner Cloud  is one of the most accepted solutions available, and running it from the cloud can be a very effective solution depending on the customer.

The functionality of OpenText LoadRunner Cloud is highly valuable. Its LoadRunner  functionality allows us to record a solution's networking protocol and replay them. We currently have a customer where we need to run 350,000 web services calls per hour. The ability of OpenText LoadRunner Cloud to handle such extensive functionality is impressive. It's the functionality of the LoadRunner  tool rather than the cloud solution itself that is crucial.

I have no real input on areas for improvement. It is fine as it is.

We have three to four years of experience with OpenText LoadRunner Cloud.

We rate the stability a seven out of ten. We haven't explored the tool deeply enough to find its limitations, so the score reflects our limited experience.

I would rate OpenText LoadRunner Cloud a ten out of ten for scalability. It is very scalable, and on the cloud, it's even more scalable, potentially unlimited.

The initial setup is complex, but that is the nature of the technology. It's not more complex than other tools we use, like QTest or JMeter . Given the complexity of emulating an application and ensuring security, it is expected to be complex.

The ROI is not necessarily cost savings. Sometimes a customer wants to use OpenText LoadRunner Cloud, or it's the only tool that will solve the problem depending on the application.

I would prefer it to be cheaper. On a scale of one to ten, the price is a five. It's delivering functionality, but we also use JMeter , which is free.

While using OpenText LoadRunner Cloud, we also use other solutions and are not tool-dependent. I rate the overall solution a six out of ten.

Public Cloud

Other

I use OpenText LoadRunner Cloud for projects with cloud-based applications, particularly for public websites and intensive user scenarios.

OpenText LoadRunner Cloud can scale in a cloud-based environment to support up to ten thousand concurrent users without capacity loss, which is not possible with on-premise solutions on personal machines. This scalability and network bandwidth capability are essential for high-scale load testing. Additionally, it allows me to upload common LoadRunner scripts to the cloud for testing without requiring a license for scripting tools, offering a flexible and comfortable high-scale load testing solution.

I expect an improvement in the cloud location offering to better serve local applications, particularly to enhance testing accuracy for users in regions like Thailand. AI-driven testing strategies also need more enhancement.

I have been working with OpenText LoadRunner Cloud for around three years.

OpenText LoadRunner Cloud is web-based, so there is no need for installation.

I have faced several problems using the cloud in different locations, so I rate the stability a six out of ten.

The solution is highly scalable, which is its main feature. I would rate scalability a nine out of ten.

The technical support is rated as six out of ten. They have been slow in response, but it's important to note that OpenText has recently taken over Micro Focus. I believe this requires improved support.

Neutral

LoadRunner Cloud helps with risk elimination by reducing performance degradation in production, ensuring a better end-user experience.

OpenText LoadRunner Cloud pricing is flexible, offering a more affordable solution compared to the more expensive on-premise LoadRunner. The hourly usage model allows cost-saving when used rightly.

There is no need for maintenance with OpenText LoadRunner Cloud since it is a subscription-based platform handled by its provider. Overall, I rate OpenText LoadRunner Cloud a seven out of ten. I recommend it based on its capability and reasonable pricing, especially for customers already using LoadRunner with limited user licenses.

Public Cloud

Other

It supports business applications for performance testing across various domains, such as retail, financial services, and banking. They had already procured a tool, LoadRunner, for use in their environment, whether on-prem or in the cloud. I've worked on numerous use cases that vary depending on the sector, like banking or insurance, as each sector has its requirements.

This tool has specific components, like LoadRunner Virtual User Generator for scripting, and execution windows, such as LoadRunner Cloud or Performance Center. It’s divided into three parts: scripting and code development are handled by one component, execution by another, and analysis by a separate file. This makes it user-friendly and easy to understand. In comparison, JMeter, which is open-source, can feel a bit clumsy. NeoLoad, on the other hand, is a good alternative and performs well, as I’ve worked with both. However, due to customer requirements, I procured LoadRunner, and we continue to use it. So, there was no option but to go with LoadRunner. In terms of setup and usability, it’s straightforward. If I were to rate the leading tools in the market, LoadRunner comes first, followed by NeoLoad. Based on my experience and colleague feedback, these are the most commonly used project tools.

The main difference is the interface; the look and feel have changed, but the background setup and configuration remain the same. The project admin team had already set up LoadRunner Cloud in our environment.

Since AI plays a major role in today's world, many tools are expected to integrate with it. If LoadRunner has AI integration, that would be a great feature.

In past projects, including those with LoadRunner and NeoLoad, clients often asked about integrating CI/CD pipelines, such as using Jenkins to automate the triggering process. I’ve done POCs on this, and it’s possible. Once set up, the pipeline can automatically execute tests without manual intervention.

I have been using OpenText LoadRunner Cloud for ten years.

From your experience, there are typically two options when purchasing a license: with or without support. Opting for support has been beneficial, especially when issues arise.

For instance, you mentioned a colleague facing a problem where they couldn't capture an application's network due to some background package being installed. After raising a request, the vendor support team from OpenText connected within 24 hours, set up a meeting, and provided helpful suggestions. Although it took a few days to understand and resolve the issue fully, the support was proactive and responsive.

Both LoadRunner and NeoLoad seem to offer efficient and timely support, ensuring users can get assistance when needed, which adds value to the tools themselves.

Positive

It's quite simple and user-friendly. Anyone with some LoadRunner experience can easily adapt.

The cloud configuration setup and server access can be tricky, as we don't usually get direct access. Sometimes, even after the configuration is supposedly done, problems arise when we start testing. For example, after developing a script locally, I occasionally face issues executing it on the Cloud LoadRunner machine. The test doesn't start, and I can't review the results. This usually happens if the configuration isn't done properly. I have to check the cloud-based machines and verify the IP range of the virtual systems because Cloud LoadRunner doesn't use static IPs—it has a dynamic range. The IP range can vary, and it picks one by one where the problem arises.

When that happens, I need to check whether my application can access the cloud machines. We need to request firewall access, specifying the source and destination, to enable communication between the cloud machine and our application. On-prem setups are different; there’s a physical machine, and I have direct access. I can check the hostname of my application and verify access to the load generator. I can ping the machine, use Telnet, and quickly resolve the issue. The process feels more cumbersome in the cloud, especially when facing execution challenges.

I transitioned to the cloud just six months ago. I've worked on cloud-based environments for only six months after migrating from on-prem applications. This involved migrating existing scripts to the cloud and re-executing previous tests to compare performance between on-prem and cloud. The goal was to assess the differences between the two environments, as the client expects improved response times and overall performance due to the cloud's higher configurations. I’ve shared the results of these tests, which reflect those improvements

The pricing is high compared to other licensing tools like NeoLoad. It's not excessively expensive but higher than NeoLoad. However, in my experience, clients often weigh NeoLoad and LoadRunner equally. LoadRunner has been around for a long time, and while some companies have moved away from it, the higher pricing can be attributed to its support for multiple protocols. For instance, it supports over 40 protocols, including SAP, Citrix, Oracle databases, and batch testing. This wide range of protocol support is one reason for the higher cost. Depending on the applications being tested, whether desktop, Citrix or something else, you may need to purchase specific protocol-based licenses.

LoadRunner offers a free license for up to fifty users, which may not be adequate for most projects. Projects require more load to simulate real business scenarios, so a more comprehensive pricing model may be necessary.

Two main licensing budgets are to consider: one for LoadRunner and another for NeoLoad. If a project has no budget for purchasing a product and is looking for open-source tools, I highly recommend starting with Apache JMeter. However, it's important to note that JMeter may not be flexible enough for all requirements, especially for desktop applications, as it has limitations.

JMeter operates in a single interface and lacks commercial tools' structured reporting and usability features, making it less user-friendly. While it’s a good starting point due to being free, users may need extra effort to organize and interpret results.

We recently evaluated OpenText's LoadRunner and NeoLoad for a project. We engaged with both vendors to take demos and understand their licensing models. Generally, NeoLoad's pricing was comparable to LoadRunne

If it's in the cloud, there's a separate version called LoadRunner Cloud. We have both options on-prem and cloud. Some applications only support on-prem, while others are cloud-based, so we must test them in the cloud environment. In this account, we work with both environments. It's worth noting that around 90% of applications are moving to the cloud these days. In the cloud version of LoadRunner, the concept of manually adding load generators doesn't exist. You don't need to worry about adding load generators to the tool, a common challenge with on-prem setups. In on-prem projects, if there are ten load generators and others are using them, you must wait for availability before running your tests. LoadRunner Cloud automatically assigns the load generators.

If someone in my network is looking for a performance testing tool, I recommend LoadRunner, especially if the client has a sufficient budget. For clients using cloud-based applications, it's essential to consider LoadRunner's cloud environment, which requires specific infrastructure from OpenText. They can opt for the Performance Center without worrying about cloud infrastructure if their applications are on-premises.

The recommendation ultimately depends on whether their applications are cloud-based or on-premises. Additionally, pricing and supporting the required protocol play significant roles in decision-making. It’s important to assess which types of applications they are working with, such as web, Citrix, or Windows applications, to determine the necessary protocol licenses. Buying unnecessary protocols can lead to wasted expenses.

Overall, I rate the solution a nine-point five out of ten.