Cost optimisation in Azure Log Analytics and Azure Application Insights

Disclaimer

This case study is based on a real cost optimisation analysis for Azure Log Analytics and Azure Application Insights. For privacy reasons, all client names, resource names and specific identifiers have been anonymised. The data, analyses and optimisation measures presented nevertheless reflect the project’s actual circumstances and results.

Contents

• Analysis of the existing environments
  • DEV environment
  • QA environment
• Findings from the DEV and QA environments
  • Optimise data ingestion
  • Data retention and recovery strategies
  • Transformations with KQL
  • Cost management
• Recommended cost optimisation actions
  • Optimise data ingestion
  • Analyse API endpoints and apply targeted filters
  • Evaluate and change table plans
  • Daily caps only as a last resort
  • Cost management and transparency
  • Further optimisations
• Conclusion

Analysis of the existing environments

DEV environment

Log Analytics

• Resource: dev-demo-analytics-log
• Location: France Central
• Price per GB: EUR 2.358 (as of 23 July 2025)

App Insights

• Resource: dev-demo
• Location: West Europe

Visual overview of a 90-day period

Figure: The last 90 days for the 5 most important App Insights tables. The original Azure portal screenshots in this case study retain their German interface and labels.

Total number of events over 90 days:

Usage sorted in ascending order:

Figure: Percentage distribution of the App Insights tables.

Tables that need optimisation:

1. AppPerformanceCounters
2. AppTraces
3. AppDependencies
4. AppEvents

Diagnostic settings

No settings present.

Assumption about when data reduction started

Figure: Timing of a suspected cost optimisation measure.

From 12 June, measures were taken to store less App Insights data. The Log Analytics daily cap may have been set to 1 GB/day to reduce data volume immediately.

Querying the Dependencies table over 90 days

// KQL query for the Dependencies table
dependencies
| where timestamp >= startofday(ago(90d))
| summarize sum(_BilledSize) by operation_Name
| render barchart

Figure: Extract of 1000 entries that are billed and incur costs.

Figure: Grouped data extract of 1000 entries, with GB rounded to 2 decimal places.

Summary of the telemetry data

1. Cost analysis of the API endpoints responsible for the highest data volumes and costs.
2. Optimisation potential because frequent data collection for API endpoints such as POST Container/GetContainerToken is not always necessary and could be reduced. For development teams, however, it remains interesting and useful for analysing application behavior.
3. A period limited to 90 days to inspect current data and identify trends in which API endpoints are called particularly often.
4. Billing transparency because the KQL analysis aggregates with the _BilledSize filter. This is important for cost control and forecasting.

This detailed data analysis can support dashboards for continuous monitoring. Alerts can be configured when individual API endpoints produce unusually high traffic in a short period and therefore incur costs. It can also prompt code reviews to improve expensive API endpoints.

QA environment

Log Analytics

• Resource: dev-demo-analytics-log
• Location: France Central
• Price per GB: EUR 2.358 (as of 23 July 2025)

App Insights

• Resource: qa-demoapp1 and qa-demoapp2
• Location: West Europe and Germany West Central

Visualisation over a 90-day period

Figure: The last 90 days for the 5 most important App Insights tables in the QA environment.

Dashboard

Figure: Dashboard of billable App Insights tables sorted by GB.

Data extracted from the dashboard

Cost calculation (price: EUR 2.358 per GB, as of 23 July 2025):

Note: 962.88 MB = 0.963 GB (1 GB = 1024 MB).

Calculation: cost per table = volume (GB) multiplied by EUR 2.358.

Total cost = sum of all individual table costs.

Telemetry data over 90 days

Summary of the 90-day telemetry data

1. A general “Down” trend is visible from beginning to end, indicating an overall decline in daily data ingestion.
   1. Less activity or application usage.
   2. Cost optimisation measures such as enabled sampling or SDK configuration.
2. AppDependencies is notably high and regularly shows large data volumes that incur costs.
3. Other tables are unremarkable and store relatively little data.

Findings from the DEV and QA environments

Optimise data ingestion

1. Filter at the source using Data Collection Rules and table transformations to collect only relevant metrics and logs from App Insights and Log Analytics, including only necessary columns where appropriate. This reduces data volume, minimises stored data and lowers costs.
   1. Recommended action: yes.
2. Enable sampling for App Insights to prevent unnecessarily large amounts of telemetry data being collected: Requests, Dependencies, Traces, PerformanceCounters and Metrics.
   1. Recommended actions:
      1. Enable sampling.
      2. Advanced sampling:
         1. Enable adaptive sampling for spikes through the SDK, not in the portal.
      3. Rule out duplicate data collection in code as an active measure.
      4. Adjust the host.json file when using Azure Functions:
         1. logging.applicationInsights.samplingSettings
         2. logging.logLevel
      5. Use the SDK and disable automatic data collection:
         1. Rename the InstrumentationKey or do not set it as a variable.
         2. Install and configure the SDK and control collection of genuinely business-critical data. This has the highest savings potential.
3. In general, regularly check which applications or services store particularly large amounts of data and whether filtering or adjustments can reduce that volume further.
4. Limit time intervals for different services such as dashboards, workbooks, Power BI reports, KQL queries in the Basic plan, DCRs and availability tests in Azure Monitor.
   1. No action required because there is no access to inspect these services, and no availability tests that could incur costs are active.

Log Analytics daily cap

Daily cap: This measure is currently active in DEV and QA at 1 GB/day.

Problem: As soon as Log Analytics reaches the storage limit of 1 GB/day, for example at 13:00, no further logs from any other services are stored. This includes services I currently cannot see due to permissions, such as Container Insights and Activity Logs. These data do not appear anywhere and are lost. All these services compete to store their data.

Consequence: Data loss can cause monitoring blind spots. Dashboard visualisations show data only up to a certain limit. Missing data in a measurement period can prevent alerts from firing correctly, even when the trend actually shows a critical increase and an alert would have been necessary.

Recommendation: Treat this only as a last option when data is sent to Log Analytics without control. A better option is enabling sampling, plus adaptive sampling through the SDK, in the individual services so that only a fraction of the data reaches Log Analytics. Where possible, use Data Collection Rules and table transformations as well, or instead, to filter data before it is stored in Log Analytics.

Data retention and recovery strategies

No change is currently necessary because the standard retention periods of 30 or 90 days are already configured for App Insights tables, and no longer retention period is required for compliance reasons.

Nevertheless, choosing the right table plan offers an opportunity to save costs:

• Recommended action:
  • Identify tables that can move from “Analytics” to “Basic”. This offers moderate to high savings potential.

Table plan settings

The original table plan documentation describes the available plans.

The following table adds Azure Log Analytics prices for the France Central region.

Summary of table plans

• Analytics: For critical logs that need long-term, repeated analysis through complex queries.
• Basic: For high log volumes with a short query window, meaning 30 days of interactive retention, and little or no need for cross-table analysis.

Identified App Insights tables

Important App Insights tables

In summary:

These tables represent the different telemetry data App Insights collects to monitor and analyse application performance, reliability and usage in the DEV and QA environments.

Transformations with KQL

Moving individual table plans from “Analytics” to “Basic” can introduce additional costs for the data being analysed. It is therefore important to aggregate only the data necessary for each visualisation and analysis.

Cost management

1. Maintain an ongoing view of both ingestion volume and the period covered by costs in Azure Cost Management to identify optimisation opportunities early.
2. Configure budget limits and notifications to avoid cost overruns and react early.

Recommended cost optimisation actions

Optimise data ingestion

Goal: Reduce data volume at the source.

Recommendations:

• Use Data Collection Rules and table transformations to filter relevant telemetry data.
• Enable sampling in App Insights, including adaptive sampling through the SDK.
• Avoid repeated data collection, for example duplicate SDK initialisation.
• Optimise LogLevel in AppTraces: less Debug, more Warn/Error.

Analyse API endpoints and apply targeted filters

Goal: Reduce data costs by analysing high-frequency API endpoints.

Recommendations:

• Automate KQL analyses, grouping by _BilledSize.
• Review endpoints such as POST /GetContainerToken.
• Provide regular feedback to development teams for targeted optimisation.

Evaluate and change table plans

Goal: Reduce storage costs through appropriate table plans.

Recommendations:

• Move tables to the Basic plan.
  • Lower costs: EUR 0.697/GB instead of EUR 2.358/GB.
• Keep the Analytics plan when cross-table queries and alerts are needed.

Daily caps only as a last resort

Goal: Avoid monitoring gaps.

Recommendations:

• A daily cap of 1 GB/day is currently active, but should not be a permanent solution.
• Risk: once the cap is reached, all further logs are discarded, which can cause blind spots.
• A better solution combines sampling, adaptive SDK sampling and DCRs.

Cost management and transparency

Goal: Identify and control costs early.

Recommendations:

• Set up Azure Cost Management alerts and budgets.
• Create dashboards to monitor costs and volume.

Further optimisations

• Disable the InstrumentationKey or fully configure the SDK.
• Adjust time intervals, for example for dashboards.
• Collect less data.
• Store only relevant data.
• Raise awareness within teams.
• Adjust host.json in Azure Functions:
  • logging.applicationInsights.samplingSettings
  • logging.logLevel

Conclusion

The cost optimisation analysis for Azure Log Analytics and Azure Application Insights in the DEV and QA environments showed that targeted measures can deliver substantial cost savings.

Implementing Data Collection Rules and table transformations, enabling sampling methods and evaluating table plans significantly reduced ingestion and storage costs, resulting in estimated cost savings of around 70% for the client.