How We Enabled The Development Team By Automating Production Scenarios Migration

Introduction

Speed and precision are essential in software development. The faster and more reliably code can be tested, integrated, and deployed, the better a team can respond to changes and deliver new features. Yet, many organisations rely on manual processes, causing bottlenecks that slow progress and introduce errors.

Repetitive tasks like code deployment and testing can drain time and resources.

According to McKinsey & Company, 45% of repetitive work in software development could be automated, allowing teams to focus on more strategic initiatives. Another report by Puppet shows that organizations that fully adopt automation in their CI/CD pipelines deploy code up to 208 times more frequently than their peers.

Automation accelerates these processes, enabling teams to push updates more confidently and frequently, significantly reducing time to market.

Automating Testing

Test automation ensures that new code integrates smoothly with existing functionality, reducing the risk of introducing new bugs.

Companies implementing advanced test automation practices report up to 90% faster testing cycles.

This efficiency leads to quicker, more reliable deployments, allowing development teams to focus on innovation rather than firefighting.

For insights on how automation enhances productivity check out our blog on boosting efficiency with workload automation.

Automating Production Scenarios Migration

Migrating production scenarios—real-world use cases that software must support in live environments—ensures software behaves as expected when deployed. Automating this migration process streamlines these scenarios' setup, configuration, and validation, reducing the time and effort required.

For more on automation in project planning see our post on automating project planning.

Manually migrating production scenarios can be time-consuming and error-prone. Configuring environments, transferring data, and validating setups without automation often lead to discrepancies and delays.

A Forrester survey found that 56% of development teams experienced critical errors due to manual migrations, highlighting the need for automation.

The Importance Of Data Seeding And Migration

Data seeding and migration are foundational elements of production scenario migration. Data seeding involves populating databases with initial data, which is crucial for development and testing environments. Automating this process ensures consistency across environments, preventing issues that could lead to failures or bugs in production.

A report by Gartner indicates that organizations that automate data seeding reduce their migration times by up to 50%, significantly speeding up the overall development cycle.

Discover how automation impacts various aspects of business operations in our article on what is recruitment automation.

Axelerant's Intelligent Automation Process For Production Scenario Migration

Setting Automation Goals

• Move development work to non-production environments.
• Handle multiple scenarios or entire folders at once.
• Automatically create or update connections, data stores, and structures.
• Use Make to manage these processes seamlessly.

Choosing The Right Tools And Technologies

We aimed for a no- or low-code approach for rapid iteration and impact. Make was the ideal choice due to its comprehensive API and documentation supporting scenario revisions.

Integration into the CI/CD Pipeline

Except for Slack notifications about scenario copies, all operations now occur within Make, ensuring smooth integration into our existing processes.

Implementing The Automation Framework

A Make Walkthrough

For individual scenarios, we use AUTO/COPY/OD/Scenarios > Team; for folders, we use AUTO/COPY/OD/Folders > Team.

Clicking on "Run Once" sets scenario inputs.

Scenario inputs load as variables and are processed for the desired scenarios.

The scenario ID is then pushed to Make Run Scenario to begin the copying.

The Make scenario handles all necessary assessment, implementation, and notification steps.

AUTO/COPY/OD/Scenario > Team takes a snapshot of production and destination team environments for custom functions, data stores, data structures, folders, etc.

Afterward, the scenario details are extracted, and the information for whom to notify and where to place the copied scenario is set.

While most of Make's modules are easy to copy for use elsewhere, API keys, authenticated connections, JSON data structures, webhooks, etc., are unique to the team environment and must be recreated.

Through Make's API, we extract the identity of these unique entities to determine whether they already exist in our target environment or need to be created.

We then create the data store or update their structures as needed.

After cycling through the scenarios for connections, data stores, JSON data structures, webhooks, and keys, the scenario is saved, and stakeholders are notified via Slack of the completed copy operation.

When a connection requires authentication, a Slack notification is sent to allow the copy requester to do it during the Make operation. If the authentication isn't completed in time or fails, the scenario is still copied and can be reconnected manually later.

Ensuring Scalability And Flexibility

While our system handles API keys, connections, and data structures well, updating webhook URLs or scenario IDs manually remains necessary. Future updates will further automate these processes to minimize manual edits.

Migrating Production Scenarios To A Safe Environment

As applications grow in complexity and usage scales up, managing production scenarios in a development or testing environment becomes challenging. More users and systems depend on these scenarios, making changes or updates harder, especially with remote work and flexible schedules.

The solution is to copy production systems cleanly into secure development or testing environments. Migrating production scenarios to a safe environment allows developers to replicate real-world conditions without disrupting ongoing operations.

It also enables rapid iteration, where changes can be tested, refined, and perfected before deployment. Moreover, when the updated system is ready for production, automation ensures that the copying process is swift and minimises the risk of human error.

Key Concepts For Copying A Production Scenario To Development

1. Identity To be Copied Scenarios: Clearly distinguish between copying individual scenarios and entire folder-level operations. This ensures that only the necessary elements are replicated.
2. Configure Copier Scenario: Configure and set up all necessary inputs before executing the copy. This step ensures that the copied scenario mirrors the production environment as closely as possible.
3. Copier Scenario Execution: Trigger the copying process for the selected scenario. Automation tools can precisely handle this, reducing manual intervention and associated risks.
4. Comprehensive Management: Oversee all steps of the copying process, including assessment, implementation, and communication. Ensure that every stakeholder is informed of the process's status from start to finish.
5. Environment Assessment: Create snapshots of the production and destination environments. These snapshots serve as backups and references.
6. Source Detail Extraction and Notifications: Extract all relevant details from the source scenario and set up notifications for stakeholders.
7. Target Scenario Entity Management: Recreate unique elements specific to the targeted environment. This might involve reconfiguring certain aspects to suit the development or testing environment.
8. Completion and Communication: Finalize the migration process and inform all stakeholders of the outcome.

Keeping data lookups and temporary storage within a controlled environment minimizes the risk of data leakage. Additionally, implementing global purging regularly for temporary storage ensures that data is not accessible after the migration is complete.

A Success Story: Impact Of A Successful Migration At Axelerant

At Axelerant, automating production scenario migration has significantly impacted development efficiency and productivity. With the right tools and processes in place, what used to take hours of preparation can now be accomplished in minutes.

Developers can quickly shift into a productive mindset, knowing that the environment is secure and ready for them to work on.

By automating security checks and authentication processes, our Intelligent Automation team ensured that data integrity remains intact at every migration stage. Notifications keep the team informed, allowing them to start the migration process, take a break, and return later to continue without any issues.

Automation reduces the chances of accidental errors, data deletion, and system failures, leading to more reliable outcomes.

Lessons Learned

• Encouraging Development Copies: Making it easy to set up testing and development copies ensures that engineers are more likely to use them than work directly on live systems. This reduces the risk of accidental errors and system failures.
• Separation of Environments: Multiple functionalities can be built and tested safely before live deployment by clearly separating production and development environments. Axelerant's literal copy-and-paste capabilities between scenarios ensure that different teams can work on various features simultaneously without risk.

Conclusion

Looking forward, Axelerant plans to refine the automation process further. Breaking down the scenario copying process into smaller, more manageable chunks will enhance team collaboration and reduce manual editing requirements. Converting this system into a custom application could help other organizations adopt better software lifecycle management practices.

Want to know if automation is the way for your organization? Book a call with us now!