I did not find a relevant answer to our problem so I am contacting you. I'm are experiencing a few problems using the artoftest runner remotely on a virtual machine, set up through the Azure pipeline.
Tests are working locally on my pc with no problem. Our issues are:
1. It contains the base URL of our site that is running on a virtual machine from which we are running the tests. Everything works ok when we run the test list locally on the laptop. But when we are running it through the command line manually on the VM, it starts the browser and goes to the localhost address that was previously used.
2. When we run the Azure pipeline, that contains the command line "ArtOfTest.Runner.exe -xml list="C:\TelerikTemp\s\Citus.Platform.Administration.FrontEnd.Tests\TestLists\demo.aiilist""
the overall result fails, with no specific error anymore, we resolved the previous errors with the Chrome browser. Monitoring the behavior of the active processes in the VM while the pipeline is running, we saw that the artoftest runner runs, and also Google Chrome pops out in the list of processes,
but nothing happens. Chrome won't open automatically and the processes terminate with the error message "Overall result: Fail" without any specific detail about the error.
Thank you,
Domagoj Halusek
1 Answer, 1 is accepted
Hello Domagoj,
I am sorry to hear you are having any troubles with implementing Test Studio tests in the Azure pipelines and I will be happy to assist you.
In order to get additional details on the issues I will list few questions on each of the topics you listed. Please, let me know your input on these.
1. The execution issues on the VM
- Do you mean that the application under test is hosted on the VM machine and thus, the URL is accessed using the name of that machine?
- Can you, please, give me an example for the localhost address you mentioned?
- Or you are talking about the localhost address reached before the navigation step is triggered?
If this is so, this is a default address which Test Studio uses to start its components in order to communicate with the browser. The only difference for the address on different machines can be the port used. - Do you mean that the tests are not passing further after that localhost address?
- Can you, please, open the Test Runner from the system tray and enable the logging on the VM machine? (please, clear this, if it was already enabled)
- The, trigger a test run through the ArtOfTest.runner.exe (manually, do not use the Azure pipeline for this attempt).
- Let me know what is the behavior - you can take also a screenshot of the browser state.
- Collect the generated log and send it zipped via this thread.
2. The failing tests from the Azure pipeline
- Did you follow the steps listed here to setup the Azure pipeline for the Test Studio tests?
- Are you using a Self-hosted agent for this pipeline?
- Have you configured the self-hosted agent to run in interactive mode (and not as a service)?
- Do you see any other messages, (apart from "Overall result: Fail") in the agent's console in the Azure?
- Generate an application log for this test execution as well - clear the logging from the Test Runner, trigger the task to execute the test list from Azure, once it fails, collect the generated logging and send it zipped over this thread.
I will be looking forward to hearing back from you with these additional notes and continue the discussion on the occurrence. Thank you in advance for your cooperation.
Regards,
Elena
Progress Telerik