Hi,
When starting a remote run of a test list, it always takes 4-5 minutes, from list initialisation to execution. Since my test lists seems to produce more false Fails the more items they contain, I tend to run higher frequency shorter lists, but then this delay becomes a bigger issue.
Is this normal behaviour? Does a list need that much time to transfer/load?
I'm running from a Windows 10 desktop to a Windows 7 virtual machine.
Hope you can help with this.
Thank you,
Jakob
6 Answers, 1 is accepted
This is expected behavior since the project is transferred to the remote execution machine in order to run the test lists. If your project is larger it will take longer time to be sent to the execution.
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
Hi Elena
Thank you for you reply. Incidentally we just realised yesterday that what you described took place, and as we have quite a lot of test lists this is causing the delay.
So the only workaround would be to split the tests over more, smaller projects?
Thanks,
Jakob
Please let me know how large is your current project - how many tests, how many test lists there are, are the test lists quite large?
If the delay is bothering your work that much you might think of how to split your project into few smaller. Additionally please let me know which your Test Studio version is? There are quite a lot improvements implemented with each next build so it is recommended to upgrade to the latest available.
Thank you for the cooperation!
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
Hi Elena
Ok, so I have about 60 tests in this project. I have a lot of issues with instability with larger test lists, especially when running the tests remotely on the run-time licence, so I almost have a test list for each test case and need to run them individually. When loading elements on TS startup I think I have about 345. If I could add all the test cases to a handful of lists and reliably run them, the delay would not be much of an issue, but seeing as how I get a lot of false positives that way, it's not really worthwhile. Locally the tests run much more consistently.
After seeing your post I realised that I was indeed running a rather old version, in fact the last 2015 release. So I've tried updating both Test Studio (Web and Desktop) and the run-time to current versions, 2016.2.630.1.
However, the update took med from bad to worse, as a whole new problem was introduced. Now the application that I'm testing closes mysteriously after the first step is run. I don't have enough technical expertise to evaluate why this current problem appeared, so I will ask a developer to look at it tomorrow and update the ticket.
Regards,
Jakob
Rank 1
I’ll be out of office for a few weeks, so a quick update on this matter before I go. The new problem introduced by updating to the 2016.2.630.1. version has been solved by one of our developers who found a workaround.
The update might also have solved the “first test always fails” issue, but I’ll need to do a few more runs to know if it’s consistent.
And with regards to the long delay initialising tests, it looks to be the size of my project as you suggested. I tried opening a new project and recording a basic test, and could notice and obvious difference when running that remotely.
Regards,
Jakob
I hope the upgrade has solved this issue as well because it sounds quite weird to me. However I would like to mention it is important for test execution to ensure an active session on the execution machine.
Though if you still face the issue please generate both application and execution logs during such an execution which first test fails. Please clear both logs prior the faulty run to ensure only relative records would be captured within.
Thank you once again for your continuous cooperation! Please let me know of your observations once you gather some.
Regards,
Elena Tsvetkova
Telerik by Progress