Exploring Xray For Test Management In Jira

What Do We Need? 

It is common for the test(s) to exhibit the same behavior across different features. To elaborate, say, for instance, on an eCommerce website, if the ‘Add To Cart’ feature is working for the dress section, one would expect it to be working for the Shirts section too. Recently, I came across a project where the scenario was such that the same test(s) could pass for one requirement and fail for the other requirement. Hence, we wanted a solution wherein our test execution could capture this need accordingly. 

The first thought that came to my mind was to duplicate the test(s) that were common across different requirements because I hadn’t seen a Test management tool that would encompass this need. Coincidentally, I was already exploring the Xray Test Management Tool, available as a Jira plugin, as one of the Test Management tools at Axelerant. I read about this feature and learned that it was available only with the server version of Jira. 

Luckily, the project that needed this requirement used their own Jira, which was a server version and we could handle this requirement with the Separations of concerns feature available in Xray.

NOTE: This feature is not available in Jira Cloud.

If you are new to Xray and haven’t heard of it before, just to give you a quick overview of the tool - It is a plugin available for Jira, on both Cloud and Server versions. You can read more about it here. 

How to implement 'Separations of Concerns' in Xray?

The default installation doesn’t come with the feature enabled. You can do it by following the steps mentioned below:

• Log in with admin privileges
• Navigate to the Manage Apps page
• In the Xray section, click on Requirement Coverage
• Uncheck the checkbox “Separation of concerns”
   
  
• Click on the Save button
• Reindex Jira in the background

Here I am assuming that you already have some experience with the Xray tool and you know how to create tests, associate them with multiple requirements, and create test execution issues for the tests. If not, then please look at their official documentation for reference.

Let’s move forward with our sample test execution, which comprises a test that is associated with two requirements. For better understanding, we will use the naming to be Test A, Requirement 1, and 2. Please follow the steps mentioned below:

• Open the test execution issue.
• Click on the Execute button for the test you want to execute.
• Select Execute from the dropdown list.
• In the Test execution window that opens up, notice that all the related requirements are displayed under the Affected Requirements section.
• Notice the option to update status is displayed for both requirements individually.
• Assuming that the test passed, we will pass it only for Requirement 1.
• Here is a screencast demonstrating the above steps. 
   
  Video file

• Refresh the Requirement 1 ticket.
• There are two things to notice here:
  • The status is reflected in the Requirement Status field, which is OK in this case because we passed the test. 
     
    
  • The Test status is still displayed as To-Do because that field is updated by the overall execution status, which we didn’t update in this case. 
     
    
• Now, similarly, let us fail the same test for Requirement 2, considering that it is not working for the second requirement. 
   
  
• Refresh the Requirement 2 ticket and notice two things:
  • The status is reflected in the Requirement Status field, which is NOK in this case because we failed the test.
     
    
  • The status of the actual test is still ToDO because we still haven’t updated the Overall execution status field.

This is how we can use the ‘Separation of concerns’ feature provided by Xray to update different requirements having a common test(s) with different testing results. 

Please take note that this is not ideally recommended and should only be used when the situation demands. Ideally, we should use the Overall execution status field to update the test execution results to accurately reflect the test execution status of the test(s) across the entire application.