I'm currently evaluating Test Studio - to use it as performance test tool in our company.
The performance test captured time for opening a page as 9.564sec but when I tried the same step manually, the page opened up in less than a 2seconds.
It is not the case with only this page but I see the tool adding some delay to all steps.
This highly impacts the performance report that is seen as a tool to improve the applications performance.
Also, I saw the CPU usage to spike up to 89% when using Test Studio for executing a test step.
But the same step when executing manually doesn't have much impact on CPU usage.
Kindly advice.
Thanks,
Kruba
Note: Attached config details of the machine on which Test Studio is run.
4 Answers, 1 is accepted
I believe this is a misconception on how Test Studio reports step times in Performance runs. Say you have the following test:
- Navigate to page
- Click button
The Navigate step occurs almost immediately, however this does not necessarily mean that the page is fully loaded. For that information, Test Studio relies on the Find Element time for the next step. That time is included in step one's time. So, in our example above, if the Navigation takes 2 seconds and the button takes 10 seconds to be found, then step 1 will report a time of 12 seconds.
We'll need more information to troubleshoot your second point:
- What Test Studio process spikes?
- Is it standard test execution or a Performance run?
- What type of application is under test (HTML, Silverlight, etc.)?
- Is it a specific step or the entire test?
Ideally we'd like to reproduce the issue locally. Please provide reliable reproduction steps against a publicly accessible site so we can see the behavior first-hand.
Greetings,
Anthony
the Telerik team
Rank 1
Thanks for your reply!
Please find below, my answers.
I've attched the screenshot of the results and would require few inputs from you.
- How to calculate the Time Taken?
If you look at Step 4: Time for Login shows 26.638s and 1.298s for Step 5 where it waits for the page to load and thus find an element.
But, according to your reply the time for step should also be calculated as part of Step 4 - please correct me if I misunderstood.
2. If possible, can you please do a manual run of the steps and compare the time taken - time delay will be very much evident.
3. The size column displays the total data transfer during the step execution - is there anyway we can get a break down wrt Server and client?
4. If I'm running a Performance Test List, is there any way I can have the test to continue from one test to another without closing the browser?
For example: If I break the attached step into 2 parts namely Login and page verification. Is there a way where in I add these two as part of a list and run it continuously? The first test(Login) used for login and second test(page verification) continues from there in the same browser.
Thanks for your time!
Do let me know if you would require more information.
Regards,
Kruba
Note: I've attahced all the required information.
Thank you for the files.
1. My previous explanation was in regards to an action step, where a built-in "wait for exists" always occurs first. In this case, step 5 is a Wait step itself and not an action step. The time taken to find the LogOutGrid element is added to step 4, and the 1.298 seconds reported in step 5 is the time between when it was found and when it became visible.
Keep in mind that Waits and Verifications don't really have a place in Performance testing, as explained here, and it's hard to interpret those types of steps in this context.
2. I did not see a dramatic difference in times between running your test with and without Test Studio.
I first ran through the steps in IE, outside of Test Studio, and saw the CPU usage spike as high as 80% at certain times. When run through Test Studio the average CPU usage was slightly higher, which I expected due to the additional automation overhead. The maximum usage never exceeded what was seen when run manually, however.
3. The Size column represents the incoming data size received from the server. No client information is included.
4. Set RecycleBrowser to True on the Web tab of Test List Settings.
Anthony
the Telerik team
Rank 1
Happy New Year!
And thanks for your reply!
Things look more clear to me now, than it was before.
Just a few test runs for a week or two and I should be in a state to complete my POC.
Thanks for your continuous support!
Regards,
Kruba