Here at Tintri, we field a lot of questions from customers who know that automation is the solution to a problem that they have, but who may not be quite sure where to start. There’s plenty of information out there on tools and technologies, but for someone new to the automation game, it may not be clear how to use a given set of tools to solve a problem.
In this article, we’ll look at an approach for solving problems with automation. We’ll even build an example as we go.
We’re going through this process to solve a real problem. It always helps to define what that problem is so that we can keep referring back to it. Many automation tasks have many components to them and it’s surprisingly easy to lose sight of the task at hand. It could be that you’re using automation to streamline the onboarding of new employees or customers. Perhaps it’s the automated deployment of test/dev labs as clusters of virtual machines. In any case, there’s some kind of business need that needs to be fulfilled.
In our example, we have a number of production SQL Server instances where some nightly reporting jobs are run. These reporting jobs are quite heavyweight and impact other users during their large processing window. What we want to do is to have the reporting jobs run on non-production VMs with that day’s production data.
Break It Down
Obviously, this problem involves quite a few moving parts and there are often many ways to get the job done. What we want to do is to break down the problem into smaller sub-problems to make it easier to develop. Using our example:
- The reporting jobs themselves may need some amount of work to be able to run on another VM. The application owners have the expertise to be able to manage that and we can help things along by performing a one-off clone of each of the production VMs.
- We need this to happen each night, so this will likely be some kind of script run as an automatic job, rather than having someone manually run the script.
- We need this operation to be performed across a number of our production VMs, so we’re probably going to find ourselves with a loop.
- We need to make the production data available to the reporting VMs with the least amount of impact as possible:
- We could perform a SQL dump and restore to copy the database contents daily from production to reporting, but that’s quite an expensive operation.
- Instead, knowing that the VMs are configured to have a system/application drive, a database drive and a database logs drive, we can use Tintri’s SyncVM feature to make the data and log volumes available to the reporting VMs using zero-copy snapshots.
- We need to handle unexpected errors. If something goes wrong overnight, we need to make sure that we have enough information available to us in the morning to resolve the issue. In some cases, there may also be things that we can do to handle these issues in the automation. In our example, we’re going to keep the code simple and simply log any issues.
As you can see, we’ve taken this fairly large business need and broken it down into manageable pieces and the design has pretty much fallen out of that process.
As we start to write code to do this work for us, try running it. If you write a few lines of automation and run it and it fails, that’s going to be easier to rectify than if you write dozens of lines and it fails in a few places. Make a few small changes and verify that they work. Then make more changes and do the same.
You’ll see that as we start looking at actual lines of code, we’ll start with a particular operation (Sync-TintriVDisk) and work our way back. We’ll make small iterative changes at a time and execute them after each small set of changes just to make sure that we didn’t break anything.
I would strongly recommend testing this against non-production systems to begin with too. Once we know it works, you’ll be able to let it loose on the real world.
Next, we’ll look at actually implementing this in PowerShell to run as a nightly job.