When you are using a combination of on-premises and cloud for your workloads it is important to have visibility into and control over what you are doing.
As a Microsoft specialist, Operations Management Suite (OMS) is what I use to monitor, automate and protect workloads across the hybrid cloud whether they are in Azure, AWS, or on premise. New features are added automatically to OMS and while you can read the blurb on the Microsoft site I always think it is much better to look at real time/real world use to gain a better understanding.
In this, the first in three blogs about the benefits of OMS, I’d like to take you through the steps required to ingest data into an OMS workspace.
Back in June Microsoft released the new View Designer feature in OMS. View Designer allows you to create a personalised overview of the data you most need to monitor and analyse. It gives you the capability to use log searches to create bespoke visualisations of the data in the OMS repository. This is a very powerful capability, enabling you to create a single pane of glass dashboards across data sets for any technology for which you are collecting data. Good times!
Part One – Collect the Data
For this blog I’m going to demonstrate how easy it is to create a custom Squid Proxy OMS view using Visual Designer – important if you want insight into the speed of your web servers. This is a good example because the same process can be used for any application or technology you want to gather log data for.
So, let’s configure OMS to collect the Squid access.log
You can collect this data from a Syslog server, however, for this example we will be reading straight from the Squid log location.
Using a Syslog server will allow you to consolidate several Linux based logs for the OMS agent to read data from.
There are a few good blogs that detail different ways to collect and store Linux data in OMS. For this demo, I will demonstrate how to manually import a squid log for collection in OMS.
Before we start, let’s make a few assumptions:
- The OMS agent for Linux has been installed as the target computer node running the Squid proxy service. (See here)
- The /var/log/squid/access.log file has the permission of 644 set (Otherwise you will receive an “access denied”).
- You are collecting Linux performance data.
When we are finished, the dashboard could look something like this:
Not very busy at this stage, however, it is a good base to build on.
Let’s go get some data!
Log into the OMS workspace then open the Settings interface:
Go to Preview features> Enable Custom Logs:
Go to Custom Logs > ‘Add+‘:
From the Squid proxy, copy the access.log file to your computer. From the dialogue, select as below:
Select New line as the delimiter:
Enter the location on the Linux server of the Squid access.log file and click the + button:
Give your new custom log a name and add any information you want. Make a note of the name you give it.
Note: It can take up to an hour for the data to start being collected, so go and have a coffee or work on something else for a while.
To test that the log data is being collected go back to the log search. In the search window type:
Type=Squid_CL (Or whatever you called your custom log)
If the data is being ingested into OMS, this will return the raw data
So in this first blog, we have successfully connected to the Squid access.log on the Linux server and the data is being imported into OMS.
In the next two blogs we will address:
Part Two – Prepping the Data
- We will create the custom fields required to manipulate the data and set up the required queries to make it all look good and, more importantly, be relevant.
Part Three – Putting it all together
- We will look at how to configure a custom solution to visualise the log data, creating a dashboard and demonstrating the flexibility that OMS gives you to display the data terms.
By Nathan O’Sullivan, Solutions Architect, SystemsUp
SystemsUp has a full-time team of solutions architects and cloud consultants who are highly certified across all the major public cloud platforms as identified by Gartner and Forrester Research such as, Azure, Office 365, AWS and Google Cloud. To find out more about how SystemsUp can help design and build the cloud you need please contact us.