Test List 'skips over' execution code

A little bit more information for you, here is the .aiilist file from the last time I tried to execute my script through a Test List:

<TestList Type="ArtOfTest.WebAii.Design.Execution.TestList" Version="2011.2.1413.0">
  <ID Type="System.String">2ac18a53-d0af-4a41-8ed9-d9dee6f17704</ID>
  <TestListName Type="System.String">temp list</TestListName>
  <Date Type="System.DateTime">2012-03-26 12:38:16Z</Date>
  <Settings Type="ArtOfTest.WebAii.Core.Settings" Version="2011.2.1413.0">
    <WebSettings Type="ArtOfTest.WebAii.Core.Settings+WebSettings" Version="2011.2.1413.0">
      <DefaultBrowser Type="ArtOfTest.WebAii.Core.BrowserType">InternetExplorer</DefaultBrowser>
      <WebAppPhysicalPath Type="System.String">
      </WebAppPhysicalPath>
      <EnableScriptLogging Type="System.Boolean">True</EnableScriptLogging>
      <EnableUILessRequestViewing Type="System.Boolean">False</EnableUILessRequestViewing>
      <LocalWebServer Type="ArtOfTest.WebAii.Core.LocalWebServerType">None</LocalWebServer>
      <BaseUrl Type="System.String">
      </BaseUrl>
      <AspNetDevServerPort Type="System.Int32">-1</AspNetDevServerPort>
      <KillBrowserProcessOnClose Type="System.Boolean">False</KillBrowserProcessOnClose>
      <RecycleBrowser Type="System.Boolean">True</RecycleBrowser>
      <UseHttpProxy Type="System.Boolean">False</UseHttpProxy>
      <EnableSilverlight Type="System.Boolean">False</EnableSilverlight>
      <VerboseHttpProxy Type="System.Boolean">False</VerboseHttpProxy>
      <SilverlightConnectTimeout Type="System.Int32">30000</SilverlightConnectTimeout>
      <MultiBrowsersExecution />
      <UseMultiBrowsers Type="System.Boolean">False</UseMultiBrowsers>
      <IsProfilingExecution Type="System.Boolean">False</IsProfilingExecution>
      <ProfilerConfigurations />
    </WebSettings>
    <WpfSettings Type="ArtOfTest.WebAii.Core.Settings+WpfSettings" Version="2011.2.1413.0" />
    <ClientReadyTimeout Type="System.Int32">30000</ClientReadyTimeout>
    <ExecuteCommandTimeout Type="System.Int32">20000</ExecuteCommandTimeout>
    <LogLocation Type="System.String">C:\WebAiiLog\</LogLocation>
    <QueryEventLogErrorsOnExit Type="System.Boolean">False</QueryEventLogErrorsOnExit>
    <ExecutionDelay Type="System.Int32">0</ExecutionDelay>
    <AnnotateExecution Type="System.Boolean">True</AnnotateExecution>
    <SimulatedMouseMoveSpeed Type="System.Single">0.3</SimulatedMouseMoveSpeed>
    <WaitCheckInterval Type="System.Int32">500</WaitCheckInterval>
    <AnnotationMode Type="ArtOfTest.WebAii.Core.AnnotationMode">All</AnnotationMode>
    <LogAnnotations Type="System.Boolean">True</LogAnnotations>
    <IsStressRecordingMode Type="System.Boolean">False</IsStressRecordingMode>
    <IsUserInteractiveMode Type="System.Boolean">False</IsUserInteractiveMode>
    <CreateLogFile Type="System.Boolean">True</CreateLogFile>
    <XMultiMgr Type="System.Boolean">True</XMultiMgr>
    <UnexpectedDialogAction Type="ArtOfTest.WebAii.Core.UnexpectedDialogAction">HandleAndFailTest</UnexpectedDialogAction>
    <ElementWaitTimeout Type="System.Int32">10000</ElementWaitTimeout>
    <DisableDialogMonitoring Type="System.Boolean">False</DisableDialogMonitoring>
  </Settings>
  <Tests>
    <Item Type="ArtOfTest.WebAii.Design.Execution.TestInfo" Version="2011.2.1413.0">
      <ID Type="System.String">a4ece2e6-629e-4997-b575-5abf386e7ed7</ID>
      <Path Type="System.String">SystemAdmin\Users\sa_UsersSetupAllRequired.tstest</Path>
    </Item>
  </Tests>
  <Filter Type="ArtOfTest.WebAii.Design.Execution.Filter" Version="2011.2.1413.0">
    <Keys />
    <Comparisons />
    <Values />
  </Filter>
  <TestListType Type="ArtOfTest.WebAii.Design.TestListType">Automated</TestListType>
</TestList>


Thanks,
Nigel.

...also, I have created a stripped down script that also fails in exactly the same way, so it is not just one script.  Here's the Test List execution log:

[Trace] : Execute test 'g_NavigateToURL'
[Trace] : Comment: Script: g_NavigateToURL; Author: Nigel Edwards; Date: 02/03/12
[Trace] : Comment: To consistently navigate to the URL and open the LogOn screen
[Trace] : Comment: Script uses Local Data
[Trace] : Navigate to Project - DataDriven: [$(URL)]
[Trace] : NavigateTo: http://vs-demo/BaseProjectQA
[Trace] : Wait for 'InnerText' 'StartsWith' 'Base Project - Log On' on 'HtmlTag' - DataDriven: [$(LogonScreenTitle)]
[Trace] : Extract 'TextContent' on 'VersionTagDiv' into DataBindVariable $(VersionNumber)
[Trace] : *** Version Number is:   v 1.9.0.11 ***
[Trace] : Comment: * End of g_NavigateToURL test script *
[Trace] : Enter text 'user' in 'TextBoxUserIdText'
[Trace] : SetText:([Element: 'input:0' (id=textBoxUserId)]) - Text:user
[Trace] : Enter text 'password' in 'PasswordPassword'
[Trace] : SetText:([Element: 'input:1' (id=Password)]) - Text:password
[Trace] : Click 'ButtonLogOnSubmit'
[Trace] : Click:([Element: 'input:2' (id=ButtonLogOn)])
[Trace] : Click 'SystemAdminLinkLink'
[Trace] : Click:([Element: 'a:7' (id=SystemAdminLink)])
[Trace] : Click 'UsersSpan'
[Trace] : Click:([Element: 'span:34'])
[Trace] : Wait for Exists 'CreateNewSpan'
[Trace] : Enter text 'abc' in 'Text'
[Trace] : SetText:([Element: 'input:1']) - Text:
[Trace] : ScrollToVisible:([Element: 'input:1']) - ElementTopAtWindowTop
[Trace] : MouseClick:LeftDown (1083,394)
[Trace] : MouseClick:LeftUp (1083,394)
[Trace] : MouseClick:LeftClick (1083,394)
[Trace] : IF (Verify 'InnerText' 'StartsWith' 'Showing 0' on 'TableUsersInfoDiv') THEN
[Trace] : Comment: else do nothing
[Trace] : Closing Window

Hello Nigel,

Please excuse me on the delayed reply.

I have reviewed your case but I cannot find any reason of the failure. The only difference between the Quick Execution and the TestList execution are the TestList settings. The settings are taken into account when running as part of a TestList and they can lead to some unexpected behavior:
http://www.telerik.com/automated-testing-tools/support/documentation/user-guide/test-execution/test-list-settings.aspx

However, I can't think on any particular setting that will cause this type of behavior. Can you confirm that the option StopTestListOnFailure is not checked (see screenshot).  We will need more info on this, please provide us the log from test results so we can investigate further.

Looking forward to hear from you.

Greetings,
Ivaylo
the Telerik team

Hi Ivaylo,

Apologies for my own late response.  I was on vacation.
I can conform that the StopTestListOnFailure function was not checked.  As to test logs, I've already posted these above (the Test Execution log, and also the contents of the .aiilist file), as well as the actual test script as a zip.  Is there something else you need to see?

Thanks,
Nigel.

Hello Nigel,

I hope you enjoyed your vacation.
Unfortunately we never had the Log file and by looking at your test we cannot see anything that might be causing the failure. Please kindly provide us with the log so we can see the actual error. Please check this short video in order to see what we need as files. It would be also helpful to record a short video for us with execution of the test list so we can see the actual moment where the test fails. You can use Jing for creating the video for us. I also want you to test something else. Could you please turn the annotations on for the test list and try to execute it again in order to see if that would pass. Please refer to the screenshot attached to see how to enable annotations for the test list.

Looking forward to hear from you.

Kind regards,
Ivaylo
the Telerik team

Hi Ivaylo,

Okay, I'll do as you ask, though at the moment I'm firefighting another project, so it may be a while until I get back to you.

Thanks,
Nigel.

Hello Nigel,

Sure, take your time. When possible for you please send us the log, and also try the annotation thing, this might slow the execution of the test list and pass the execution.

Looking forward to hear from you.

Greetings,
Ivaylo
the Telerik team

Hi Ivaylo,

I've managed to squeeze a bit of time to follow this up.  I've attached a zip containing:

 - a screenshot of the Anotation setting for Test Lists
 - the project script
 -  the log file you requested
 - the swf file
Note that only a single file was created in the Results folder (there was no folder created, as was shown in the short video link you posted in your earlier reply).

You can also see the Jing video at this address:
     http://www.screencast.com/t/WGHWSmw2h

Hopefully this will allow you to see the issue clearly.  Let me know if you need anything else.

Thanks,
Nigel.

PS - I note that ScreenCast / Jing doesn't replay in IE9, but I checked that the video replays correctly in FireFox and Chrome.

Hello Nigel,

Thank you for providing the files, I am able to see the problem in the video you have provided, however I cannot have it reproduced on our side so we cannot investigate further. Is it possible for you to create a new simple project with data driven test and if the issue is still present please provide the entire project so we can have a local repro.

Looking forward to hear from you.

Regards,
Ivaylo
the Telerik team

Hi Ivaylo,

I fully understand about not being able to fix something you cannot reproduce.  I've seen my share of those.

I'll be in receipt of a new build to resolve a memory leak I raised a ticket against (515438) in a couple of weeks.  I'll wait until I get that build and see if the problem is still present.  If it is I'll do as you suggest and get back to you.

Thanks,
Nigel.

Hello Nigel,

Thank you. Looking forward to hear from you.

All the best,
Ivaylo
the Telerik team