DataDriven based testing best practices

Hi Brian,

I apologize for the delay getting back to you on this.

What sort of policy are we talking about here? An insurance policy? A business policy? An HR policy? An advertisement policy?

Let me reiterate the problem just to make sure I understand it correctly:
1) When you create a policy there are several hundred fields that must be filled in, then the policy is saved to the system
2) You want to test making changes to an already existing policy stored in the system
3) You want to test changing just a few (or one) fields and do not want to have to re-enter all of the fields

Do I understand correctly?

Part of the problem here is how do you identify which field needs to be changed i.e. how can the test detect that field 2 or field 202 needs to be modified? Is there something in your data source that will specify this? Another detail to work out that is part of the same question is whether or not there are multiple pages involved e.g. fields 1-50 are on page 1, fields 51-99 are on page 2, so if somehow the data tells the test to set field 79 to X we need to know we have to navigate to page 2 in order to get to field 79.

Let's start by defining what the input data will look like when changing a policy. Will this input always include all data for all input fields, or will it somehow try to tell the test "change field n to X"? The test implementation varies greatly based on this answer. We'll go to the next step once you answer this starting question.

Kind regards,
Cody
the Telerik team

The policy i am talking about here is an insurance policy. What you specified is right. I have the input data for the new policy setup but for the changes i have not decided on how to do it as there are several fields on several pages. I needed to know what the process is to compare two records in a database so that i can run the script based on that.
If there is no way to compare two records on the database, i think it would be better to have all the fields in the record included and modify just the field that needs to be changed. That way the script will run through each step and pick the same values again and modify just the field that has changed. This would take longer time for the script to run which is the reason i wanted to find out better options. Please let me know if you have any questions.

Hello Brian,

I needed to know what the process is to compare two records in a database...

Test Studio does not have any built-in functionality for direct DB access or manipulation. We do have this code sample demonstrating how you can write your own code to access a DB. It would be possible (and not all that hard) to modify this code sample to compare two records in a DB and return the differences. You just have to come up with a SQL query to do so. Or you could perform two SQL queries to get two records in memory and use code to compare the two records.

Now the part I don't understand is what two DB's or records are you going to compare to get the differences? It's easy to see you have the main DB where the new policy has been created and saved. But what exactly will you compare against?

Greetings,
Cody
the Telerik team

Hello Cody,
      Thanks for taking the time to provide a solution. We have decided to run through the complete script and let test studio pick the values even if the record hasn't changed. This could cause the script to run longer than manually performing the actions but it would help us perform the work unattended.
       An unrelated question regarding my script. I might have to provide you a demo to properly explain this but let me try. I have an OK button on a web application, which when clicked gives me a pop-up. So i have a step in the script that clicks the OK button and then performs a dialog handle but when i am running the script, i noticed that the script executes the "click the OK button" but for some reason keeps searching for the button even after it has been executed. Can you please suggest why this is happening if you have seen similar issue before or i could provide you any information you need?

Thanks

Hi Brian,

Can you send me a screen shot of this dialog, or even better a video of this happening? Jing is a good free screen recorder. My one suspicion is that this dialog is a modal dialog. Modal dialogs have a bad habit of not returning control back to the test script until the modal dialog is closed. For this reason we normally set SimulateRealUser to true on the click step that causes the modal to open. This prevents the test from hanging indefinitely inside the click step.

All the best,
Cody
the Telerik team

Hi Brian,

My apologies, it's SimulateRealClick as shown in the attached screen shot (not SimulateRealUser... however that property does exist for the Set Text test steps).

I watched your video (had to find and download the correct CAMTASIO codec to play it) and am a bit concerned by what I see. Here's what I'm observing in the middle:

1) The test clicks the OK button for the Client Access Endorsement screen
2) This causes the JavaScript Confirm dialog to show up
3) Then this dialog goes away. I am concerned what caused this dialog to close. I find it hard to believe that Test Studio closed it simply because the test step still running was the one to click the OK button for the Client Access Endorsement screen.
4) Then the app proceeds to the next screen but now Test Studio is trying to handle the Confirm dialog that's already come and gone. It's out of sequence.

I'd like to look at this problem on your computer via GoToMeeting. Let me know what time zone you are in and your availability and I'll setup the meeting. I'm in Austin, TX which is Central Time (GMT -6).

All the best,
Cody
the Telerik team

I am in Eastern Time, Indiana. I am available from 1PM - 4 PM. Please let me know the time, so that i can share my test with you.

Hi,

All right. Let's target 2pm Central (3pm Eastern):

1.  Please join my meeting, Friday, November 16, 2012 at 2:00 PM Central Standard Time.
https://www2.gotomeeting.com/join/688837290

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial +1 (805) 309-0012
Access Code: 688-837-290
Audio PIN: Shown after joining the meeting

Meeting ID: 688-837-290

GoToMeeting®
Online Meetings Made Easy™

Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.

All the best,
Cody
the Telerik team

Hi Nithin,

Just to document the outcome of our meeting:

1) We discovered you're trying to do conditional dialog handling based on the contents of the data source. You were conditionally calling a subtest. This approach doesn't work. The problem is that the dialog handler must be initialized and watching for the dialog before the dialog appears. Using the approach you had the dialog would appear, then the sub test would get control and initialize the dialog handler. By then it's too late since the dialog is already open. The subtest will ignore already open dialogs.

2) You had multiple Handle Confirm dialog handlers in your test. There is a known limitation in Test Studio... you can have only 1 dialog handler of a specific type in a test. You can get around this by using sub-tests to handle the additional dialogs of the same type as needed. We fixed your test today by splitting out the affected steps into sub-tests

3) We also created a coded step that will find a policy row based on the policy number, then find the machine Modify button and click on it.

Regards,
Cody
the Telerik team

Hi Cody,
       I tried using the steps that you created for me to find a policy row based on the policy number, then find the modify button and click on it. I had to make some changes and am getting the error "Index was out of range. Must be non-negative and less than the size of the collection."  I tried looking to see why the error was generating but could not figure it out. Any help is appreciated.

Thanks

Hi Brian,

It's impossible to say what is causing this without seeing the code involved and being able to run it and study the behavior on my machine. How about we look at this together in another GoToMeeting? On Friday I have only my 10:30 slot open. On Monday I can do 1 hour anywhere from 10:30 - Noon (Central time).

All the best,
Cody
the Telerik team

Cody,
      Friday at 10:30 am Central Time should be fine with me.

Thank you,
Nithin

Hi Nithin,

You got it:

1.  Please join my meeting, Friday, November 30, 2012 at 10:30 AM Central Standard Time.
https://www2.gotomeeting.com/join/452673762

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial +1 (510) 443-0605
Access Code: 452-673-762
Audio PIN: Shown after joining the meeting

Meeting ID: 452-673-762

GoToMeeting®
Online Meetings Made Easy™

Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.

Greetings,
Cody
the Telerik team

Hi Nithin,

To summarize our meeting today:
1) We determined the wrong code was being used in the test having the problem. You will fix this yourself.
2) We looked at how to handle a case where a search returns "no client found" message. The conclusion was that you need to click on the first "select" button whether or not a client is found.
3) I showed you how to verify that text is set to Read-Only.

I am now closing this ticket as the main issue is resolved.

All the best,
Cody
the Telerik team

The code that is used in the script is displayed above. I noticed that the script runs fine in most of the cases but when there are multiple iterations that have to run with atleast one Data("FormAction") = Modify and have to execute a subtest, I am getting an "object reference not set to an instance of an object"

Hello,

Before I can begin to diagnose this problem I need to know which line exactly is throwing the "object reference not set to an instance of an object" error message. When you get this error, it should contain a line number and a filename in the stack trace that goes along with it. For example:

System.Exception: deliberate failure
   at VBTestStudioProject.WebAiiTest1.WebAiiTest1_CodedStep2() in C:\Users\gibson\Documents\Visual Studio 2010\Projects\VBTestStudioProject\VBTestStudioProject\WebAiiTest1.tstest.vb:line 85

Please look in that file and pinpoint the line that is throwing this error. Then we can start diagnosing why that line of code is in error.

Regards,
Cody
the Telerik team

Log.WriteLine(formcell.ToString())

is the location where the error is occuring. It looks like the formcell doesn't return any value in that particular iteration. The error doesn't  occur for every iteration. If you look at the code i gave earlier, the error occurs only when the    
       Me.ExecuteTest("User Access\DeleteFormSchedule.tstest")
step is executed. "DeleteFormSchedule.tstest" has an if condition which causes it to execute only in some cases. So after it is executed, and the next iteration is being executed, the error occurs.

Please let me know if you need more information.

Hello Brian,

Agreed it means that the line before it doesn't find anything:
Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))

Try adding this in from of that line:
ActiveBrowser.RefreshDomTree()

If that doesn't help let's look at this together via another GoToMeeting.

All the best,
Cody
the Telerik team