I have setup a test where page elements are data bound and the test is to navigate to each of the 134 pages accessible from the menu bar in our system and verify the page successfully loads by interrogating the page title. I regularly find that 1 iteration fails for transient reasons and it is subsequently successful on a second test.
I have enabled the option to "Rerun Failed Tests" and while this works, it would be ideal if it only reruns the failed iteration rather than the entire list of 134 iterations. Is there a mechanism available to rerun only the failed iteration?
If retesting just the failed iteration is not possible at the test list level, is there a way to setup my test in a for loop with two iterations, only performing the second iteration on failure of the first, and failing the test only if both iterations fail?
Thank you.
1 Answer, 1 is accepted
Hello Todd,
Thank you for sharing details about your setup and test scenario.
The Rerun Failed Tests option on a test list, when tests are data driven, works as you have already discovered. Our test runner does not consider individual iterations as separate test runs and will rerun the whole test from the start. I understand that in your case this takes a lot of time, but iterations can be used in different ways. So, someone might have their tests setup in a way that they need to start from the first iteration, when there is a failure. This makes it hard for us to make changes in this behavior, so I have to admit that we do not plan to make any changes here.
I am sorry that there is no direct way to configure the test list for your scenario. That said, in my mind there are two things that you can do to optimize the test list results and overall behavior.
- Analyze the test failure and identify why the test might be failing the first time, but passing on the rerun. Is this a timing issue? If so, you can add a Wait for Exists step and increase its timeout. It really depends on the type of step that is failing. If you attach the exported step failure details from the failed iteration, I will have a look and advise you further.
- Split the iterations into multiple tests, where each test has just part of the data. It is up to you to decide whether to split all iterations into 2, 3 or more tests. You can use the Filter for data binding to select the data to be used in each test.
I hope the information and suggestions are helpful. I remain available to continue this discussion and work together to find the best solution for you.
Regards,
Plamen Mitrev
Progress Telerik
Rank 1
Iron
Absolutely, you can contact us with any questions about the project structure and we will be happy to advise you. It is better to ask before you spend a lot of time to make changes that are not ideal for you.
I hope that you will find the way to organize the tests, so you get quicker and more stable execution of the iterative tests.