Azure performance data retrieval - step by step guide
by INVOKE Team
Posted on July 25, 2019 at 12:00 AM
Azure gaining cloud market sharing and lot of companies are migrating their applications to Azure Cloud. After migrating applications, first thing users look for is monitoring. How to monitor the resources being used?
Azure Portal has good charting capabilities to monitor resource usage in different dimensions (example CPU, Disk etc., for Azure VMs). But, users look for more ways to retrieve the usage data and customize it for their own needs, could be for integrating with their cloud automation tools or running it through analytics to optimize cloud costs and so on. This blog post is our experience in retrieving and integrating Azure VM metrics with a third party client for one of our clients.
Retrieving Azure monitoring data involves 2 steps:
- Authentication
- Data request and response processing
Each of these steps can be performed using different tools. The tool you use determine how easy/complex this process is going to be. In this blog post, we used a combination of tools to get things done (we had few frustrating moments to get a few configurations done via Azure Portal, same configurations worked like a charm when executed through Azure PowerShell).
Authentication
Back to our topic, to retrieve any data from the Azure Monitor service, the request should be first authenticated by AD. Once authentication is successful, a token will be granted. Clients should use this token in every request to retrieve the data. Now, how to authenticate and get this token?
There are 2 ways authentication setup can be done:
- MSI (Managed Identity)
- Service principle
Which approach to choose depends on the use case. What we found in our experience and/or research is, if your application is a “native client” application and running outside “service principle” is the ONLY approach will work. In all other scenarios better go with MSI approach which is newer way Microsoft promoting to avoid managing passwords.
MSI approach:
This approach is straightforward to configure and is the recommended approach if your client is running in Azure Cloud. Login to Portal and pick the Azure VM where your client would be running. Click on “identify” from left menu then select “system assigned” on the right side pane. Set “status” to “ON”.
Once you assign “Managed Identity” to VM, go to the resource group or subscription where the resources to be monitored are running and add “Monitoring Reader” role and select the VM for which you enabled “Managed Identity”. Learn more about Azure RBAC Role assignment in portal.
Service Principle approach:
To make this approach work, first we need to create "service principle" and then use those details to authenticate. If your client is running outside Azure cloud, this is the approach we need to use.
Service Principle creation
This is the most confusing and tough configuration you need to go through if you are a programmer with limited knowledge on system administration end. Why the complexity? Based on Azure REST API documentation :
All the tasks executed against the Azure Monitor API use the Azure Resource Manager authentication model. Therefore, all requests must be authenticated with Azure Active Directory (Azure AD).
In other words, Azure AD is the gatekeeper which determines whether a request to Azure Monitor API resource is authorized or not. What is Azure AD? That is a discussion for different blog posts, but here is a quick definition:
Azure Active Directory (Azure AD) is Microsoft’s cloud-based identity and access management service, which helps your employee's sign in and access resources in External resources, such as Microsoft Office 365, the Azure portal, and thousands of other SaaS applications.
Service Principle can be created either using Azure Portal or Azure PowerShell (Our experience is, PowerShell is a better tool to achieve these things, their portal is little messed up).
Here is the PowerShell script: $subscriptionId = "{azure-subscription-id}" $resourceGroupName = "{resource-group-name}" # Authenticate to a specific Azure subscription. Connect-AzAccount -SubscriptionId $subscriptionId (If this is the first time you are using PowerShell to access Azure resources, you will be prompted with a code and devicelogin url for authentication. Just follow the instructions and complete the authentication.) # Password for the service principal $pwd = "{service-principal-password}" $secureStringPassword = ConvertTo-SecureString -String $pwd -AsPlainText -Force # Create a new Azure AD application $azureAdApplication = New-AzADApplication ` -DisplayName "My Azure Monitor" -HomePage "https://localhost/azure-monitor" -IdentifierUris "https://localhost/azure-monitor" -Password $secureStringPassword # Create a new service principal associated with the designated application New-AzADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId # Assign Reader role to the newly created service principal New-AzRoleAssignment -RoleDefinitionName Reader ` -ServicePrincipalName $azureAdApplication.ApplicationId.Guid $subscription = Get-AzSubscription -SubscriptionId $subscriptionId $subscription.TenantId
The statement “$azureAdApplication.ApplicationId.Guid” prints Client ID. $subscription.TenantId prints “Tenant ID” (and the password you entered in above script too). Take note of these which are needed for authentication and authorization.
Authentication
Authentication step helped us to create a service principle which we could use in our program while requesting data.
Get access token from Azure AD by using authentication details obtained above: This can be done using pure HTTP REST calls or Azure provided libraries for specific programming language. For example, using ADAL library for java, we can obtain the token using
AuthenticationContext authContext = new AuthenticationContext(AUTHORITY, false, service);
ClientCredential clientCred = new ClientCredential(CLIENT_ID, "service-principal-password entered in above script");
Future
CLIENT_ID should be the value you noted from Powershell script output AUTHORITY should be https://login.microsoftonline.com/TenantID Replace TenantID with the value noted from Powershell script output
Data request and response processing
Use the token retrieved from above to initiate subsequent metric request calls. Using Java Apache HTTP library, the code would be something like:
URIBuilder uriBuilder = new URIBuilder(); uriBuilder.setScheme("https").setHost("management.azure.com") .setPath("/subscriptions/{SUBSCRIPTION}/resourceGroups/{RESOURCE_GROUP}/providers/Microsoft.Sql/servers/{VM_NAME}/providers/microsoft.insights/metrics") .addParameter("timespan", "2019-06-20T00:26:00Z/2019-06-20T00:56:00Z") .addParameter("interval", "PT1M") .addParameter("metricnames", "THE_METRIC_YOU_WANT") .addParameter("aggregation", "Average") .addParameter("api-version", "2018-01-01"); URI uri = uriBuilder.build(); HttpClient client = HttpClientBuilder.create().build(); HttpGet request = new HttpGet(uri); request.setHeader(HttpHeaders.CONTENT_TYPE, "application/json"); request.setHeader(HttpHeaders.AUTHORIZATION, " Bearer " + token); request.setHeader(HttpHeaders.ACCEPT, "application/json"); HttpResponse response = client.execute(request); // Get the response BufferedReader rd = new BufferedReader(new InputStreamReader( response.getEntity().getContent())); StringBuilder jsonData = new StringBuilder(); String line = ""; while ((line = rd.readLine()) != null) { jsonData.append(line); }
Pay special attention to HttpHeaders.AUTHORIZATION header. The access token should be embedded in header as Authorization token and should be prefixed with Bearer and space then toke.
If everything is configured well, this should return the results for metric you passed as value for parameter “metricnames”. Whether you use MSI approach or service principle, access token need to always obtained by client program and should be embedded as Authorization header while submitting request for data.
If the request is successful, response will be in JSON format with metric data for the duration requested for.
Have more questions? Talk to us, we could help you.
