Configuring Test Studio project to use specific TFS credentials?

Hi Matthew,

Thank you for the thorough description of the requirements you need to cover and what you have tried so far - this is really helpful for me to understand what is still missing in the configuration in your environment. 

Basically, the Test Studio Scheduling is running as a Windows service and you have correctly identified how you need to impersonate the service and allow it to use specific credentials to access the TFS repository. There is an obstacle, though, and these are the different domains in which the user and the TFS are registered.

I searched the web for any solution on the topic and this seems to require some Active Directory adjustments between the domains to be able to perform cross domain communication. You can refer to a similar topic in a Microsoft forum post here. So, to be able to use the service across different domains, you will need to add trust relationships between these.

Please, let me know if you will need any assistance on the topic or have any concerns on the case. Thank you in advance for your cooperation. 

Regards,
Elena
Progress Telerik

Elena,

Thank you for the thorough information, I appreciate it. What I ended up doing was adding the TFS credentials to the service account's "Windows Credentials" stored in Control Panel > User Accounts > Credential Manager. As a result, whenever the runtime/service account contacts TFS, it'll use these default permissions. I verified this, and it is working fine. (If I remove the Windows Credential, then scheduling a test list throws an error claiming it cannot connect to source control.)

I'm running into another issue though: when I run a test list, it isn't getting the latest test(s) from source control even though the execution server's log claims it is. My expectation is that it will get the latest files for this test list before execution. Please see the attached file. In particular, these lines claim it is getting latest:

[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Warning] CloudRunnableGUITestBuilder.CreateLocalCacheAsync() : Started getting project from source control
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Execution] SourceControlMediator.GetProject() : Getting latest source control version of project ef216904-49f9-4f7e-a36d-28d73f52b4b2
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.ConnectToServer() : ------- Function Start -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.ConnectToServer() : ------- Function End -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.AttachToRepository() : ------- Function Start -------
[03/02 15:36:16,Telerik.TestStudio.RemoteExecutor.exe(5728:14),TeamFoundationServer] TFSServer.AttachToRepository() : ------- Function End -------
[03/02 15:36:21,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Execution] SourceControlMediator.GetProject() : The project files have been retrieved from source control for ef216904-49f9-4f7e-a36d-28d73f52b4b2
[03/02 15:36:21,Telerik.TestStudio.RemoteExecutor.exe(5728:14),Warning] CloudRunnableGUITestBuilder.CreateLocalCacheAsync() : Finished getting project from source control

Notes:

• To test whether the latest is getting pulled from source control, I am modifying one of the test (.tstest) files that the test list is calling.
• The scheduling and execution applications are running on the same server.

Here is the attached log. I forget to add it to the last post!

Note: I intentionally failed the test. I am simply trying to verify that the latest test files are being pulled down from TFS on execution.

Hello Matthew,

I am pleased to know you managed to sort out the credentials issue. 

The trouble you see now is that you check-in to the TFS repository any test change and run a test list containing this change, using the 'Get Latest from TFS', but you don't see the change during the run. Please, correct me if I have misread any of the details. 

Based on the log file it seems you have closed the executing browser before the test run started - is that correct? And if it is, how do distinguish that the downloaded files from the TFS were not the changed ones? 

Can you, please, elaborate more on the particular changes that you apply to verify if the run is correct? Did you checked-in these before you trigger the run? 

Basically, the Test Studio execution process will connect to the TFS repository and will download the whole project available there. And with these shared details I am not able to identify what the difference, you see, is and what might be causing it. Therefore I will appreciate if you take the time to describe the misbehavior in further details. 

Thank you for your understanding in advance.

Regards,
Elena
Progress Telerik

The trouble you see now is that you check-in to the TFS repository any test change and run a test list containing this change, using the 'Get Latest from TFS', but you don't see the change during the run.

Yes, this is the problem I am encountering.

Based on the log file it seems you have closed the executing browser before the test run started - is that correct? And if it is, how do distinguish that the downloaded files from the TFS were not the changed ones? 

I was under the impression that 'Get Latest From TFS' would retrieve/copy the latest files into the file directory mapped to TFS on the runtime computer. I was closing the browser so that I could check the files themselves for changes.

I let the test run and reviewed the log file. It appears my changes are being ran, but they aren't being copied to the hard drive on that computer. Is that how this feature works?

Hi Matthew,

Thank you for clarifying what your doubts are. To describe how this feature works, I will share some notes about the Test Studio Scheduling setup and how this is designed. 

The Test Studio Scheduling configuration is using the Scheduling and Storage services to control the test execution on multiple machines. The Storage service is responsible for keeping a copy of the project, which can be deployed to any execution machine, and uses the MongoDB as a storage database. 

Then, once the Scheduler prepares a job for execution, the project files, which may vary depending on the tests in the test list, get deployed from the storage database to a temp folder on the selected execution machines and the test list run is then started. The temp folder is being configured in the Execution server setup. 

This is the common case when the 'Get Latest from TFS' option is not enabled.

What is different when 'Get Latest from TFS' is enabled? 

The only difference is that the remote TFS repository replaces the storage database. Which means that the project gets deployed from TFS to the temp folder on the selected remote machine. 

So, if you have applied any changes in the project, checked these into the remote repo, and trigger a run through the Scheduling.CommandLineClient, you should see the changes within the test runs. Though, if you keep a copy of the project on the remote execution machine, this will not be affected by the test list run triggered to use the latest files from TFS. 

I hope this information helps you understand what to expect from the scheduled test list runs. Of course, if you still have any concerns or doubts, please, let me know.

Thank you for your cooperation in this discussion.

Regards,
Elena
Progress Telerik