Today I was setting up a self-hosted agent, that could run Robot framework tests. I wanted the agent to run robot tests and push results into Azure DevOps as a test result of release run. First my releases returned weird looking return codes like ##[error]Bash exited with code ‘8’. Robot framework documentation says that runner returns following return codes:
| RC | Explanation |
|---|---|
| 0 | All critical tests passed. |
| 1-249 | Returned number of critical tests failed. |
| 250 | 250 or more critical failures. |
| 251 | Help or version information printed. |
| 252 | Invalid test data or command line options. |
| 253 | Test execution stopped by user. |
| 255 | Unexpected internal error. |
So I had 8 tests failing (because I had some unresolved connection issues). However this caused Azure DevOps test step to fail, because return code was something else than 0. Luckily this could be resolved easily by using no status flag.
robot –nostatusrc …
After that the test run was working fine, but Azure DevOps Publish task could not find any test result
2020-04-24T10:47:03.7942382Z ##[warning]Atleast for one assembly start time was not obtained due to tag not
present or parsing issue, total run duration will now be summation of time taken by each assembly2020-04-24T10:47:03.7951616Z Obtained XUnit Test Run Start Date: 2020-04-24T10:47:03.7951299Z and Completed Date: 2020-04-24T10:47:03.7951299Z2020-04-24T10:47:03.7971207Z No Result Found to Publish ‘/output/output-xunit.xml’.
After googling a while I found bit unbelievable answer from this web site.
Azure DevOps does not find the test result if Publish task is set to XUnit test format, to get it working I had to use the JUnit format even though that I had configured XUnit format into Robot Framework.
Now I can run robot tests in hosted agent, push results to Azure DevOps and view them under release runs.
