Hi,
we are facing a problem with the jUnit export of the ArtOfTest.Runner, it happens very often that the exported jUnit file is in the wrong format.
Sometimes it looks something like this:
<testsuites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<testsuite name="XXX" timestamp="2018-06-19T09:14:10Z" hostname="XYZ" tests="8" failures="0" errors="0" time="631.2858114">
<testcase name="Test1" time="0.4940152" />
...(all of the tests)
</testsuite>
</testsuites></testsuites>>
Or like this:
<testsuites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<testsuite name="XXX" timestamp="2018-06-18T19:09:09Z" hostname="XYZ" tests="8" failures="1" errors="0" time="679.4595394">
<testcase name="Test1" time="5.6998895" />
<testcase name="TestN" time="211.6068644">
<failure>
...The correct failure message...
</failure>
</testcase>
</testsuite>
</testsuites>
</testsuites>
Thanks for checking the problem,
Greg
6 Answers, 1 is accepted
Thank you for sharing details on the observed issue.
Could you please let me know if the issue you refer to are the two closing tags for <testsuites> tag? These are listed in the samples as follows:
</testsuites></testsuites>>
and
</testsuites>
</testsuites>
If there is anything else I have missed please let me know.
In addition to the above and having in mind your other query please let me know if there is anything common between the test list runs where the ArtOfTest.Runner had stuck and the wrong format of the jUnit files?
Thanks in advance for your time and cooperation! I am looking forward to hearing back from you!
Regards,
Elena Tsvetkova
Progress Telerik
Rank 1
Hi Elena,
yes, correct, the closing tags are the issue.
I cannot confirm that this is a related issue to our ArtOfTest.Runner behavior, I've tried serveral tests with the junit-parameter and without it and other parameters -> it think it is not correlating, as the ArtOfTest.Runner in all cases gets stuck very often (while terminating).
I appreciate your help very much!
Greg
Rank 1
Hi Elena,
yes, correct, the closing tags are the issue.
I cannot confirm that this is a related issue to our ArtOfTest.Runner behavior, I've tried serveral tests with the junit-parameter and without it and other parameters -> it think it is not correlating, as the ArtOfTest.Runner in all cases gets stuck very often (while terminating).
I appreciate your help very much!
Greg
Thank you for getting back.
I tried to reproduce the same on our end but with no success. This is why I would like to kindly ask for your further assistance - it will be good if you could find any pattern when the Junit result is corrupted so that we could try to reproduce it in our environment.
I am looking forward to hearing back once you manage to gather any details. Thanks in advance!
Regards,
Elena Tsvetkova
Progress Telerik
Rank 1
Hi Elena,
as we work with "test-modules" and not one large test, the problem occures nearly everytime we have more then two tests in our test list.
It happens not that often if the settings parameter for the ArtOfTest.Runner is disabled as if the settings parameter is enabled.
The problem is within failed and passed tests.
As VSTS "Publish Test Results" Task cannot load the jUnit file correctly, please let us know if you need further assistence!
Greg
Thank you for getting back to me.
Based on the shared notes we tested on our end with larger and longer test lists but still couldn't reproduce the faulty jUnit result. Therefore I would like to kindly ask you to share a project which reproduces the misbehavior. It could be only part of your whole project which you have verified produces the wrongly formatted jUnit result. As of now I would like to inspect if there is anything specific in the project itself which might be causing the troubles so I will not need to actually execute the tests.
On the other hand if you could reproduce the error with a sample project against public accessible page that will be also beneficial to continue the investigation.
Please let me know your opinion on the above request. Thank you in advance!
Regards,
Elena Tsvetkova
Progress Telerik