Content:
One of the most important QA Craft features is the possibility to manage the automation tests. Thanks to that, you are allowed to connect QA Craft and Jira with chosen CI server.
Video shortcut
Overall
Test Automation is based on special objects called Test Runs. They point to all chosen Test Cases with their statuses or perform dates. The execution relies on sending specific packets into the CI server where the tests are made. The result is returned to QA Craft and saved as Test Results.
General settings
General settings of connection with CI server you can find on the plugin settings page. There can be set:
- URL to CI server
- username and password for authentication
- maximal length of URL (for security and proper working)
- test execution interval
- job cron
Additionally, you can set root for finding automation test files, available folders and limit access to files.
Test Plan Panel
On the Test Plan page, there is a panel QA Craft Automation Test Runs.
Test Run
A table contains the details of all Test Runs.
Column | Description |
---|---|
TR ID | Test Run id |
Created date | Test Run creation date |
Summary | Test Run summary |
TC Progress | Current Test Run progress. It shows graphically various Test Case statuses connected to that Test Run. |
Issue key | Key of an issue that is connected to Test Run |
Execution date | Date when the Test Run was executed. |
TR Status | Test Run execution state. Possible options: Success (test ended successfully), Failed (test ended with some errors), Running (test is currently executing), Draft (draft), Waiting (test is waiting for execution), Aborted (test execution was aborted), Start error (test didn't start due to some errors, like CI server connection error, wrong response or timeout) |
actions | Details - shows the issue tree that contains connected Test Cases with their Test Case Executions' values Edit - allows for editing Test Run's settings. You can update the Test Run or save it as a template. |
Moreover, there is an option to create a new Test Run
To create TR, you can choose:
- Linked Issue - connected issue, e.g. requirements
- Test Run Status - how Test Run will be started. Possible options: Draft, Run now and Schedule (when you can set an execution date)
- Select test run template - chosen template. More about Test Runs Templates you will find below.
- Issue structure - all automation Test Cases connected to the current Test Plan. You can choose which tests will be executed.
Test Run Templates
A table that contains Test Run Templates. They help to automate executing groups of tests. Thanks to that, you won't need to create a similar Test Run manually for every execution.
Column | Description |
---|---|
Template ID | Unique id number for template |
Test Run Template created date | Date when the Test Run Template was created. |
Issue key | Key of an issue that is connected to Test Run |
Summary | Test Run Template summary |
Active | A flag if the Template is active. If not - it won't be shown on the list on the Test Run creation window. |
actions | Details - shows the issue tree that contains connected Test Cases Edit - allows for editing Test Run Template's settings. You can update the Test Run Template or create a Run. |
Test Automation Parameters
Specific settings for current Test Plan. The filled fields will override general settings. Thanks to that, you can restrict access to only necessary spaces.
Test Result
Test Result is an isolated Jira issue that serves as only information. There you can find a tree with a tested structure with its results.
REST API communication
A POST request in JSON format with the following example structure will be sent to the selected endpoint.
Data received in the commissioning request can be freely processed, e.g. at the Jenkins Pipeline level, by your own vision and ideas for the architecture of the test environment.
QA Craft plugin uses feedback from the following endpoints:
GET, Basic Auth, example:
http://jira-cluster-ip-service:8080/rest/qacraft/1.0/TestCase/changeTestCaseResolution?testCaseIssueKey=QC-3&resolution=in%20progress&result=in%20progress&testPlanIssueKey=QC-1&testRunId=676
or:
http://jira-cluster-ip-service:8080/rest/qacraft/1.0/TestCase/changeTestCaseResolution?testCaseIssueKey=QC-3&resolution=Failed&result=Done&testPlanIssueKey=QC-1&testRunId=676
Those endpoints are used to change the status of the Test Cases.
Example of the connection
Combine Continuous Testing Toolkit
The set of predefined services that allows you to start automation tests in the Continuous Testing model. For more information about that solution, look here.
Jenkins
To use automation tests in Jenkins, we will need a free Jenkins add-on - Generic Webhook Trigger. Thanks to that, you will be able to set a trigger endpoint like jenkins-cluster-ip-service:8080/generic-webhook-trigger/invoke?token=STANDARD. Executing Test Runs cause start jobs on Jenkins and receive the responses.