Rank 1
Im trying to run a Test Studio test as part of my Release Management.
I have a Powershell script to run the test.
try
{
artoftest.runner.exe test="E:\TestStudioTry1\TestingPrototype\logon.tstest"
if (!$?)
{
write-output "Failed"
throw "Tests Failed"
}
}
catch
{
Write-Error $_.Exception.Message
# exit 1 # to get an error flagged so it can be seen by RM
}
This script runs fine from Powershell ISE but when running as a step in RM it fails, looking at the logs i get.
"Message": "------------------------------------------------------------\r\n'09/09/2016 16:45:02' - Executing test: 'logon', path: 'logon.tstest.'\r\n'09/09/2016 16:45:02' - Using .Net Runtime version: '4.0.30319.42000' for test execution. Build version is '2016.2.630.0'.\r\n'09/09/2016 16:45:02' - Starting execution....\r\n'09/09/2016 16:45:04' - Failure detected during execution. Details:\r\n------------------------------------------------------------\r\n'09/09/2016 16:45:04' - ArtOfTest.WebAii.Exceptions.UnsupportedNonInteractiveOperationException: Dialog monitoring can not be enabled when running in non-interactive mode. Follow the link for more information: http://docs.telerik.com/teststudio/user-guide/command-line-test-execution/continuous-integration/team-foundation-server-builds.aspx\r\n at ArtOfTest.WebAii.Design.Execution.ExecutionContext.SetDialogMonitoring()\r\n at ArtOfTest.WebAii.Design.Execution.ExecutionEngine.InternalExecuteTestIteration(Object codeBehindInstance, Boolean isDataDriven)\r\n at ArtOfTest.WebAii.Design.Execution.ExecutionEngine.InternalExecuteTest(Test test, TestResult initializationResult)\r\n at ArtOfTest.WebAii.Design.Execution.TestExecuteProxy.ExecuteTest(ExecuteTestCommand command)\r\n------------------------------------------------------------\r\n'09/09/2016 16:45:04' - Test completed!\r\n",
How does one include a test as part of a release management flow?
6 Answers, 1 is accepted
There are two possible reasons for the exception listed in your query. One is that Telerik tests need to be able to interact with the desktop to successfully run. In order for them to do that the build agent needs to be run in interactive (i.e. console) mode and not as a windows service. If this requirement is met than an active session will be required on the execution machine where tests are running. Here you could read further how to successfully implement Test Studio tests in a Continuous Integration Environment.
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
My build agent is set to run interactive but it's actually Release Management that is trying to run the Powershell script to execute the Telerik Test.
Let me explain what i'm trying to achieve and you could perhaps point me in the right direction
Queue new build is triggered and puts the compiled app to the drop folder.
Release Management is executed as part of the build
Creates the environments/sets up dbs etc
Rank 1
After creating the Testing environment via powershell.
Run Telerik Tests against the fresh Test installation.
It's the running Telerik Tests using Release Management I'm struggling with, possibly because it's not how it's intended to be used. May be I should be running MTM from Release Management and having the Telerik tests executed from MTM.
Somebody must be doing this already?
Please note that we are not familiar with Release Management tool and we do not have an example to provide. As of the error you get and the experience we have with other build agents I shared my recommendation to ensure the agent runs in console mode.
Anyways, please give a try to your suggestion adding the MTM and share further observation on what the result would be. This will be useful experience for us as well. Thank you in advance for the cooperation!
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
I believe I got this working by installing a second Hosted agent on the TFS server. I created a new Agent Pool, "Hosted-Interactive" then went through the prompts of the ConfigureAgent.cmd (https://www.visualstudio.com/en-us/docs/build/admin/agents/v1-windows):
- An existing configuration file was detected. This will update the local agent settings. Do you want to also replace the server registration (default is N)? n
- Enter the name for this agent (default is Agent-[SERVER]) Agent-Interactive
- Enter the URL for the Team Foundation Server (default is ) http://[server]:8080/tfs
- Configure this agent against which agent pool? (default pool name is 'default')Hosted-Interactive
- Enter the path of the work folder for this agent (default is 'C:\Agent\_work') d:\agent\_work
- Would you like to install the agent as a Windows Service (Y/N) (default is N) n
Configuration successful.
"NO" to the final question. Now I have an interactive Agent Pool.
I then had to create another Environment, TEST, in my release to run the powershell script you originally provided above and selected the new Hosted-Interactive Agent Pool to run those tasks. See TEST_Environment.png.
I set this Environment to automatically trigger after the deployment to the DEV environment. As I said, the only task in this environment is to execute the powershell script which executes the tests I have created in Test Studio. Failures or successes in the tests themselves are accurately propagated to the deployment status and it will read Rejected upon any failure.
NOTE 1: I have not found a way to avoid requiring a command prompt open on the TFS server with the c:\agent\vsoagent.exe running. I tried starting it via a Command as the final step of the previous DEV deploy but that did not work. This might prevent this from being a viable solution for you.
NOTE 2: This only runs tests AFTER a deployment. Rolling back a deployment is not possible with this approach.
@Tim: Thanks a lot for sharing your experience with us!
@Zac: I hope this will be of further help for you.
Regards,
Elena Tsvetkova
Telerik by Progress