Load test will only run locally

Hi Chad,

How do I get a load test to run on multiple execution machines?  Can this only be done through a test list?

In short, that is correct. The "Run this test" button, as shown at the bottom of this page, will only run it locally. To run it remotely on all of your execution servers requires you to use a test list along with either "Run List Remotely" or "Schedule Test List".

Regards,
Cody
Telerik

Thank you Cody.  That answers my question and that was what I was afraid of :|

Unfortunately, my test list will not run the one sample load test that is in the list.  When I try to run it I am almost immediately taken to the Results tab.  Viewing the failed results of the test yield the below error (screen shot attached).

 Unable to load test 635fcb30-a212-477d-80da-bf125e7ee7e6 LoadTest\Sample\LoadTest.tstest

 

 

Hi Chad,

Why is there a GUID in the path name? I'm referring to the part "635fcb30-a212-477d-80da-bf125e7ee7e6". It's possible this is causing a path length error. Windows has a limit of 260 characters total for the path plus filename e.g. "c:\users\user A\my documents\test studio\test studio projects\635fcb30-a212-477d-80da-bf125e7ee7e6 LoadTest\Sample\LoadTest.tstest". That total cannot exceed 260 characters.

Try shortening the path length then tell me what happens.

Thanks and I look forward to hearing back from you.

Regards,
Cody
Telerik

Hi Cody,

The GUID that is displayed is not anything that we have done.  It is displayed automatically as per the screen shot.  The physical path to that particular test is D:\ConnectedToTFS\360 View CRM\QA\Automation\TestStudio\2015SR1-RP1\LoadTests\Sample\LoadTest.tstest.

BTW just to be clear, that GUIID is the "UniqueID" that is assigned to the test by Test studio which displays in the properties of that test (screen shot attached).

 

Thanks!

Chad

Hi Chad,

Thank you for the clarification. I guess the GUID is not actually part of the path. I'm just not used to seeing it displayed like that and incorrectly assumed it was part of the path. I see your total path length is only 100 characters, so we're actually fine there.

We need to create and look at the log. Please clear and enable the Test Studio application log. Then recreate the problem then view the log. Put the log into a .zip file and attach that file for analysis.

Regards,
Cody
Telerik

Hi Cody,

 After disconnecting my project from source control (TFS) and re-saving the load test, I have been able to run the sample load test remotely without the original error message.  I'm not sure how it's going to react to larger tests but will cross that bridge when it comes.  I appreciate your help and will open a new thread for that if need be.

 

Thanks!

Chad

Hello Chad,

Ok, that's partially good news. It should not be necessary to disconnect the project from source control. One possible explanation is that it caused a change to the test file forcing it to be re-uploaded to the storage server and now it runs successfully. If that theory is correct, reconnecting your project to source control will not break it.


Regards,
Cody
Telerik

Unfortunately it is happening again.  I created a new project containing a total of 7 tests, 2 of which are load tests.  One of the load tests is perfectly happy when run remotely in a list and the other simply will not run in a test list. 

I tried using my local machine as the scheduling server and running the list remotely, choosing my machine and that fails as well.  The load test run perfectly fine when I choose the "Run this test" option when editing the test, so I don't think it's a problem with the test itself.  I also don't think it is any kind of firewall or network issue as the other load test performs flawlessly.  Zipped logs and screen shots are attached.  "Icdt" indicates the failing load test and "Ref" indicates the working load test.

IcdtSuccessLocalRun - running the test using the "Run this test" button when editing the load test (not using a test list).

IcdtRemoteToLocal - running the test list with test runner configured to use my machine as the scheduler.  Ran the test list remotely selecting my machine as the target.

IcdtFailRemoteExecMachine - running the test list while pointed to a remote scheduling server and choosing a remote execution machine.

RefSuccessRemoteExecMachine - running the other load test successfully in a list to a remote execution machine.

 

Please let me know if there is further information I can provide.

 

Thanks!

Chad

Hello Chad,

I also don't think it is any kind of firewall or network issue as the other load test performs flawlessly.

I agree with you. While I do see some errors in the log files, I can't identify a specific problem in the logs that is an obvious sign of the cause of the problem. Let's do some additional diagnostics:

What happens if you put the problem test into a test list by itself? Does that work when run remotely? Is it possible it's always the second (or first) test in the list that doesn't work as expected? I saw this once before months ago, but don't clearly remember the cause of the problem. I haven't seen this for many months. I think we found and fixed the problem... but we're dealing with a new DB technology now.

If the above tests doesn't reveal anything useful, it may be time to crack open the contents of the MongoDB and see what's in it. Download and install Mongo Management Studio (free Community Edition) on the machine acting as your storage server. See the first attached screen shot for connection settings to connect to your MongoDB from Mongo Management Studio. Now drill down and look to see if you can find a test having the ID 09a0a5d3-8396-47d8-8362-1592c2d36597. See second attached screen shot for an example where to look.

If you cannot find such a test in the DB, the next step is to capture a Fiddler trace as Test Studio is trying to schedule the test list to run remotely. We need to see if the problem test is being pushed up into storage.


Regards,
Cody
Telerik

When I first ran these two load tests, they both were contained in the same test list.  The "ref" test ran and the "icdt" test did not.  Unfortunately I don't recall the ordering of the tests in the list and I have since updated that original list.

I was able to find the test with the ID of 09a0a5d3-8396-47d8-8362-1592c2d36597 in the database (attached).

I did manage to get the failing list to run remotely (which may be why I can find it in the DB now).  I don't ​know though if it was simply a result of updating the test and saving it or if the changes that were made are what fixed the test.  The "Icdt" test was set to monitor 3 performance counters on 2 remote servers (same 3 counters per server), whereas the "Ref" test did not have any counters set up.  As soon as I removed the counters and saved the test, it would run remotely.  I added the same counters back in for the same servers one by one, testing in between, and each time the test was successful.  So I ​know it's not an issue with counters being used per se​, but maybe something didn't save correctly ​with the load test the first time (although it would run successfully locally before the counter changes).  I ​do still need to find the root cause of the problem though, as it is kind of disheartening considering how quick and simple these tests are and how little traffic they record.  When I start recording and configuring larger tests, I would like to be confident that the tests will work in and of themselves (realizing they still may fail due to errors on my end of course).  I am seeing similar failures with ​some web tests as well (in the original project), where a list ​will ​not run all of the tests contained in the list remotely, but again works fine locally​.

I do still happen to have a copy of the original test files in their failing state, if that might be helpful in troubleshooting.  Otherwise just let me know what ​I need to do next in order to figure out what might be happening.

Forgot to answer: What happens if you put the problem test into a test list by itself?

I tried that after the initial failure and the test would not run by itself either.  The previous failure screen shots are from just such a scenario.

So the same test failed when the list included another successful test, as well as when it was the only test in the list.  Failures only apply to the list when run on a remote execution machine, all local tests would run successfully.

Hi,

I do still happen to have a copy of the original test files in their failing state...

I'm willing to take a copy of them and see if I can reproduce your symptoms. Unfortunately since you're no longer experiencing the problem (and we don't really know what triggered the "fix") we don't have anything more to try and diagnose it on your end. We need it in the broken state so we can continue working on isolating why it isn't working as expected.

So how large is the broken project after being put into a .zip file? You can leave out the folders Backup and Results.

Regards,
Cody
Telerik

Hi Cody,

 

There is a password in the test that I will have to remove and will unfortunately require me to update and re-save the test, which may fix things just due to act of re-saving.  Guess we will see.  I'll update that test a bit later on today (hopefully) and either let you know that it's passing after the save or zip it up and send it on along if it's still failing.

 

Thanks!

Chad

Hi Chad,

No problem. I'll standby for your results.

Regards,
Cody
Telerik

Hi Cody,

 

Unfortunately re-saving the test seemed to correct the issue for this test.  Good for the test, but not so useful in troubleshooting.  Oh well, I'll consider this closed and will create a new post if anything starts to fail again.

 

Thanks again!

Chad

Hi,

Ok sounds good. Thank you for the update.

Regards,
Cody
Telerik

And of course now it is happening again with my new larger test that I just created ^&##@#!!!!  I am sooo sick of this, I need to get to the cause of this problem.  Why will load tests not run in a list!  I don't have time to troubleshoot this freaking software every time I create something new.  Once again when I try to run the load test locally, it works.  When I include that test as the only test in a list and try to execute it remotely, I get taken immediately to the test results screen with a failure "unable to load test..."  

There are errors in the log about SourceControlMediator.GetProject(), but this project is not connected to source control.  Please help!!

 

 

 

Hi Chad,

I managed to reproduce this exact problem yesterday on my local machine. I've passed it on to our development team for analysis. I'm waiting to hear back from what is causing this and how to overcome it. I will update you when I hear back from them.


Regards,
Cody
Telerik

Thanks Cody.  I look forward to hearing from you and getting this resolved.

Hi Chad,

Thank you for your patience. Our development team is still trying to determine the root cause of this problem.


Regards,
Cody
Telerik

Thank you for the update Cody.  I look forward to hearing their findings.

 

Chad

Hi Chad,

Good news. It appears we have fixed this particular problem in our latest internal build, 2015.3.1112. I encourage you to upgrade all machines to this version and try again please.

Regards,
Cody
Telerik

Awesome!  Does this update only apply to the remote run time execution machines (i.e. no update to the stand alone Test Studio app)?

Hello,

Sorry no, it applies to both Test Studio IDE and the Runtime. Version notes can be viewed here.

Regards,
Cody
Telerik

Good deal.  I'll get those installed tonight and try out some load tests that were failing.

 

Thanks so much!

Chad

Is the message in the attached screen shot anything I need to worry about?  The installation went ok but I got the attached message on the Configure Scheduling Server screen.  I already had the scheduling service and MongoDB set up, but when I clicked apply anyway the message appeared at the bottom.

 

-Chad

Hello Chad,

To be honest I'm not sure if we need to worry about that error message or not. I'll admit it is worrisome. I can see it's truncated. It's trying to point us to an assembly that cannot be found, but I can't tell which one due to the truncated text. Is there any way to get the whole message?

Otherwise go ahead and give things a try. Generally it's not necessary to rerun Configure as Scheduling Server when you simply upgrade Test Studio. Are things working as expected now?


Regards,
Cody
Telerik

Here is the message from the log file.

 

[11/17 16:32:34,Telerik.TestStudio.Scheduling.Setup.exe(4912:1)] First trace message from process 4912: "C:\Program Files (x86)\Telerik\Test Studio\Bin\Telerik.TestStudio.Scheduling.Setup.exe"
[11/17 16:32:34,Telerik.TestStudio.Scheduling.Setup.exe(4912:1)] First trace message from foreground unnamed thread (managed ID = 1, native ID = 44).
[11/17 16:32:34,Telerik.TestStudio.Scheduling.Setup.exe(4912:1),Execution] ServiceConfiguration.SaveAsConfig() : Config file found.
[11/17 16:32:34,Telerik.TestStudio.Scheduling.Setup.exe(4912:1),Execution] ServiceConfiguration.SaveAsConfig() : Config file found.
[11/17 16:32:43,Telerik.TestStudio.Scheduling.Setup.exe(4912:1),TestStudio] MongoDbServerConfigurationViewModel.ApplyMongoDbServerSettings() : EXCEPTION! (see below)
     Outer Exception Type: System.IO.FileNotFoundException
     Message: Could not load file or assembly 'YamlDotNet, Version=3.7.0.0, Culture=neutral, PublicKeyToken=ec19458f3c15af5e' or one of its dependencies. The system cannot find the file specified.
     HRESULT: 0x80070002 (Official ID (if app.) = 2147942402, Error Bit = FAILED, Facility = FACILITY_WIN32, Code = ERROR_FILE_NOT_FOUND)
     Call Stack:
          at Telerik.TestStudio.Scheduling.Setup.ViewModels.MongoDbServerConfigurationViewModel.SaveMongoDbConfig()
          at Telerik.TestStudio.Scheduling.Setup.ViewModels.MongoDbServerConfigurationViewModel.ApplyMongoDbServerSettings()

Hello,

OK something is not right. That file should exist in our Bin folder as shown in the attached screen shot. Is it there? If you right click on it and select Properties -> Details, what file version does it report? If it's there and file version is 3.7.0.0 (which is the correct version) how are you running Configure as Scheduling Server?


Regards,
Cody
Telerik

That file is there and is the same version (3.7.0.0).  So far everything seems to be running ok.  I haven't tried any load tests yet though but I'll let you know.

 

Thanks!

Chad

Hi Chad,

So the mystery deepens. Since things seem to be working OK getting the configurator to run is almost academic. Still it would be good to nail down why it's not working as expected because it will block you from making any necessary changes in the future.

To that end what happens if you go directly to the executable in our Bin folder and double click it to run it (see attached screen shot)? That should be the same as launching it from the start menu. Also what OS are you running?


Regards,
Cody
Telerik

Hi Cody,

I copied the YamlDotNet.dll file from another computer that has Test Studio installed and after that, no more error message when running Telerik.TestStudio.Scheduling.Setup.exe.  Not sure what caused all this but there is no more error message now when running the scheduling setup program.

Hello,

Wow that is an interesting discovery. I am so glad you found a good solution.

Now to find out if load tests run remotely. I'll standby for your results when you can try this.

Regards,
Cody
Telerik

Cody,

We are running in to the issue with running load tests remotely and having the same issue as Chad.

We have updated all runtime instances and test studio ultimate instances to the latest version of internal build.  I will say, the project i am working in and trying to run load tests remotely is a new project created under the MongoDB system.  My old project that was converted from sql to mongo seems to run fine and has no issues.

Hi Ryan,

Exactly which issue are you running into? Chad had more than one problem that we have to overcome. Please include any error messages or logs that you can supply describing the problem.

Regards,
Cody
Telerik

I will get those to you.  The errors i was seeing were almost exactly the same as Chad's.  I did not however get an error on updating the schedule or storage service.  I did however run in to issue when running a load test it was stopping my scheduling server.  I submitted a support ticket separately from here.

Hi Ryan,

I think I see your support ticket that you're referring to, 997389 Problem with Running Load Tests. I also see that Ivaylo has responded to it. He's correcting asking for the application log. That is the next step in diagnosing the problem described in the your support ticket.

Regards,
Cody
Telerik

Hi Cody,

I apologize for the long delay in response, I've had other priorities come up that had to be resolved.  I'm now back at the load tests that I'll be focusing on in the coming weeks.  I'm still experiencing the problem with load tests not running remotely.  I an get them to run when I select to run remotely and select the local machine, but when I select a remote machine the test fails.  Other functional tests will run fine on that same remote computer.  I've attached the local and remote computer logs along with the error displayed on the results screen.

Also, if you guys are able to resolve Ryan's issue, would you be able to post the resolution so I can see if it may fix the problems I'm experiencing?

Thanks!

Chad

Hello Chad,

I'm not 100% certain, but based on this one entry:

Message: No connection could be made because the target machine actively refused it 10.180.20.110:8039

I am suspicious it's a firewall problem. The key difference between running a Web test and a Load test is that two extra processes are spawned for a load test which use two new TCP ports in the range 8032 - 8050. The port numbers we use is documented near the bottom of this page.


Regards,
Cody
Telerik

Hi Cody,

The error message about refusing the connection was actually due to the performance counter services not being started on the remote machines that had counters set up in this particular test.  Once I started the services, this error went away, but the load test still would not run remotely.  After much trial and error, I surmised that the problem was somehow related to the counters that were used in the test.  I got this one to run by removing and re-adding in the same counters, however other tests using the same counter template still worked fine.  Any idea what might have caused something like this?

 

Thanks!

Chad

Hello Chad,

Good detective work! I'm unable to explain why deleting and re-adding the same performance counters would make a difference. Are you 100% certain they were exactly the same counters? For example counters would fail if you try monitor performance of CPU core #8 on a machine that does not have 8 cores. You would have the same type of counters for all cores, but trying to monitor a specific device that doesn't exist would cause this kind of symptom.

Regards,
Cody
Telerik

I deleted the counters and added them back in using the same template that I had used originally.  Not sure what went wrong but the load test is working again.

 

Thanks for your help as always!

-Chad

Hello Chad,

We are very happy to hear that.

If you need further assistance please let us know.

Regards,
Boyan Boev
Telerik