The \u201cspreadsheet Olympics\u201d worked when the bill was small. Once it crossed a few hundred thousand a month? Forget it. We needed to zoom in faster and with way more detail.<\/p>\n
The Turning Point<\/strong><\/h3>\nThist came during one particularly painful review. Our EC2 line item was up by about 40% week-over-week. Everyone stared at the spreadsheet. Someone guessed \u201cmaybe it\u2019s new QA servers?\u201d Another thought it was \u201cprobably dev workloads.\u201d Nobody was sure.
\nWe pulled up the raw CUR (Cost and Usage Report) in Athena later that day, ran a quick SQL query, and boom: we found the culprit. A set of m5.2xlarge instances running 24\/7 in a forgotten test environment. Cost Explorer alone never would have shown that clearly.
\nThat\u2019s when it clicked: if CUR + Athena could give us the answers instantly, why not layer QuickSight on top and automate the whole thing?<\/p>\n
The Solution: Automated Cost Intelligence via QuickSight<\/strong><\/p>\nHere\u2019s what our architecture looks like now:<\/p>\n
\n- CUR in S3<\/strong>: This is the source of truth, delivered daily.<\/li>\n
- Athena<\/strong>: SQL queries over the CUR. We built views to simplify common breakdowns (by service, by usage type, by resource tags).<\/li>\n
- QuickSight<\/strong>: Dashboards that visualize those views. Drill-down enabled, interactive filters, the whole deal.<\/li>\n<\/ol>\n
The beauty is it updates itself. No more exporting CSVs or chasing data freshness.<\/p>\n
How the Dashboard Works (Three Layers)<\/h3>\n
We structured the dashboard as three sheets, each one drilling deeper.<\/p>\n
1. Service-Level Cost Comparison:<\/h4>\n
High-level, quick and dirty. Compares costs across AWS services:<\/p>\n
\n- Weekly Comparison<\/strong>: Displays the costs for the last two completed weeks for each AWS service, along with the absolute cost difference.<\/li>\n<\/ul>\n
![\"abc\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2025\/06\/Screenshot-2025-06-18-at-12.34.34\u202fPM-1024x429.png\")
Weekly Service Based Billing<\/p><\/div>\n
\n- Monthly Comparison<\/strong>: Similar layout and logic, comparing the last two completed months instead of weeks.<\/li>\n<\/ul>\n
![\"abc\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2025\/06\/Screenshot-2025-06-18-at-12.37.14\u202fPM-1024x348.png\")
Monthly Service Based Billing<\/p><\/div>\n
If EC2 or S3 suddenly spikes, this view makes it obvious. It\u2019s basically our \u201csmoke alarm.\u201d<\/p>\n
2. Usage-Based Cost Analysis for a Selected Service:<\/h4>\n
This sheet allows a focused analysis of cost breakdown based on usage types within a selected AWS service. It dynamically updates based on the selected service from the first sheet.<\/p>\n
Capabilities:<\/strong><\/p>\n\n- Pick a service and see which usage types are driving the increase. For EC2, that might be On-Demand instance hours (BoxUsage). For S3, it might be data transfer or API requests.<\/li>\n
- This was the game-changer. Instead of just saying \u201cEC2 is up,\u201d we could say \u201cOn-Demand usage hours doubled.\u201d<\/li>\n<\/ul>\n
![\"abc\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2025\/06\/Screenshot-2025-06-18-at-12.38.34\u202fPM-1024x392.png\")
Usage Based Billing<\/p><\/div>\n
This level of insight helps narrow down specific areas causing the increase in spend, offering better control over high-cost operations<\/span><\/p>\n3. Resource-Level Cost Analysis for Selected Usage Type<\/h4>\n
This is where the detective work gets easy. We pull in:<\/p>\n
\n- Resource IDs.<\/li>\n
- Tags (Account, Environment, Channel, Owner).<\/li>\n
- Exact cost + usage numbers.<\/li>\n<\/ul>\n
![\"abc\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2025\/06\/Screenshot-2025-06-18-at-12.42.18\u202fPM-1024x525.png\")
Resource Based Billing<\/p><\/div>\n
Without this, anomalies were fuzzy. With it, they\u2019re crystal clear.<\/p>\n
Pivot Table at the Account Level With AWS Tags<\/strong><\/h3>\nA pivot table that displays the price of each service according to usage type has been made in order to mimic the visibility provided by the AWS Billing console. Although it offers more customisation and granularity, this view is very similar to the “Bills” section of the AWS console.<\/p>\n
The Reason for Developing the Account-Level Pivot Table<\/h3>\n
This pivot table was created to support projects that enable content owners to:<\/p>\n
\n- Establish, plan, and automate news, FAST, and live video streaming channels.<\/li>\n
- Distribute those channels among various endpoints and platforms.<\/li>\n
- Learn about performance, cost-effectiveness, and channel-level content delivery.<\/li>\n<\/ul>\n
Because of this architecture, every channel frequently functions within the system as a separate client or unit. As a result, content owners must:<\/p>\n
\n- Recognize the prices and savings offered by each channel.<\/li>\n
- Attribute AWS usage accurately to individual accounts and channels.<\/li>\n
- Make informed decisions around resource optimization and pricing strategy.<\/li>\n<\/ul>\n
The purpose of this pivot table is to give an organized and understandable breakdown of AWS prices and discounts at the Account ID and Account Name levels. The use of metadata extracted from resource tags, which enables clients to view cost and discount attribution based on their unique account and channel details, is what makes this approach unique.<\/p>\n
![\"abc\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2025\/06\/Screenshot-2025-06-18-at-12.43.43\u202fPM-1024x567.png\")
Account Level Table<\/p><\/div>\n
Tag-Based Identification<\/strong><\/h3>\nThe values for Account ID<\/strong>, Account Name<\/strong>, Channel ID<\/strong>, and Channel Name<\/strong> are extracted from the resource tags associated with the respective AWS resources. These tags are parsed and joined with the billing line items to produce a complete view of the cost structure, mapped accurately to client-specific identifiers.<\/p>\nAttribution of Discounts<\/h3>\n
The table has distinct columns to show:<\/p>\n
\n- Usage-based cost<\/li>\n
- Total effective cost<\/li>\n
- Discounts applied, including:<\/li>\n<\/ul>\n
Enterprise Discount Program (EDP)<\/p>\n
Private Rate Card Discounts<\/p>\n
Bundled Discounts<\/p>\n
Value Proposition<\/strong><\/h3>\nThis structured and tag-enriched view helps clients understand:<\/p>\n
\n- Which services are contributing to their AWS bills.<\/li>\n
- What level of discount is being applied.<\/li>\n
- How usage is distributed across their own internal account and channel identifiers.<\/li>\n<\/ul>\n
More transparent chargeback and showback reporting between organisational units is made possible by this format.<\/p>\n
Why Not Just Use AWS Cost Explorer?<\/strong><\/h3>\nWhile AWS Cost Explorer provides a native interface for cost analysis, it comes with certain limitations:<\/p>\n
\n\n\nFeature<\/strong><\/td>\n AWS Cost Explorer<\/strong><\/td>\n Custom Pivot Table View<\/strong><\/td>\n<\/tr>\n\nTag-based account\/channel mapping<\/strong><\/td>\nLimited flexibility<\/td>\n Full customization via tags<\/td>\n<\/tr>\n \n Custom column hierarchy<\/strong><\/td>\nNot supported<\/td>\n Fully configurable<\/td>\n<\/tr>\n \n Discounts breakdown<\/strong><\/td>\nAggregated view<\/td>\n Separated discount attribution<\/td>\n<\/tr>\n \nOffline export and manipulation<\/strong><\/td>\nManual CSV download<\/td>\n Structured data, easier automation<\/td>\n<\/tr>\n \n Resource-level traceability<\/strong><\/td>\nPartially supported<\/td>\n Fully supported<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nThis pivot table approach is particularly beneficial when there is a need to provide cost visibility per client<\/strong>, per channel<\/strong>, or per project<\/strong>, in a way that aligns with internal organizational tagging strategies.<\/p>\nConclusion<\/strong><\/h3>\nSwitching from manual reviews to an automated, tag-driven QuickSight dashboard has been a game-changer for us.<\/p>\n
We\u2019re no longer stuck in spreadsheet hell. We can see anomalies almost in real time. Finance finally has the breakdowns they wanted. Engineers can trace spend back to actual resources. And cost reviews have gone from dreaded chores to genuinely useful conversations.<\/p>\n
If you\u2019re still leaning only on Cost Explorer and CSV exports, take it from me: it\u2019s worth building something better. Start small, keep tags clean, and let QuickSight + Athena do the heavy lifting.<\/p>\n
Because at the end of the day, it\u2019s not just about saving money. It\u2019s about clarity, accountability, and being able to answer the question we all hate: \u201cWhy did the AWS bill go up again?\u201d<\/p>\n
\nWe pulled up the raw CUR (Cost and Usage Report) in Athena later that day, ran a quick SQL query, and boom: we found the culprit. A set of m5.2xlarge instances running 24\/7 in a forgotten test environment. Cost Explorer alone never would have shown that clearly.
\nThat\u2019s when it clicked: if CUR + Athena could give us the answers instantly, why not layer QuickSight on top and automate the whole thing?<\/p>\n
Here\u2019s what our architecture looks like now:<\/p>\n
- \n
- CUR in S3<\/strong>: This is the source of truth, delivered daily.<\/li>\n
- Athena<\/strong>: SQL queries over the CUR. We built views to simplify common breakdowns (by service, by usage type, by resource tags).<\/li>\n
- QuickSight<\/strong>: Dashboards that visualize those views. Drill-down enabled, interactive filters, the whole deal.<\/li>\n<\/ol>\n
The beauty is it updates itself. No more exporting CSVs or chasing data freshness.<\/p>\n
How the Dashboard Works (Three Layers)<\/h3>\n
We structured the dashboard as three sheets, each one drilling deeper.<\/p>\n
1. Service-Level Cost Comparison:<\/h4>\n
High-level, quick and dirty. Compares costs across AWS services:<\/p>\n
- \n
- Weekly Comparison<\/strong>: Displays the costs for the last two completed weeks for each AWS service, along with the absolute cost difference.<\/li>\n<\/ul>\n
Weekly Service Based Billing<\/p><\/div>\n
- \n
- Monthly Comparison<\/strong>: Similar layout and logic, comparing the last two completed months instead of weeks.<\/li>\n<\/ul>\n
Monthly Service Based Billing<\/p><\/div>\n
If EC2 or S3 suddenly spikes, this view makes it obvious. It\u2019s basically our \u201csmoke alarm.\u201d<\/p>\n
2. Usage-Based Cost Analysis for a Selected Service:<\/h4>\n
This sheet allows a focused analysis of cost breakdown based on usage types within a selected AWS service. It dynamically updates based on the selected service from the first sheet.<\/p>\n
Capabilities:<\/strong><\/p>\n
- \n
- Pick a service and see which usage types are driving the increase. For EC2, that might be On-Demand instance hours (BoxUsage). For S3, it might be data transfer or API requests.<\/li>\n
- This was the game-changer. Instead of just saying \u201cEC2 is up,\u201d we could say \u201cOn-Demand usage hours doubled.\u201d<\/li>\n<\/ul>\n
Usage Based Billing<\/p><\/div>\n
This level of insight helps narrow down specific areas causing the increase in spend, offering better control over high-cost operations<\/span><\/p>\n
3. Resource-Level Cost Analysis for Selected Usage Type<\/h4>\n
This is where the detective work gets easy. We pull in:<\/p>\n
- \n
- Resource IDs.<\/li>\n
- Tags (Account, Environment, Channel, Owner).<\/li>\n
- Exact cost + usage numbers.<\/li>\n<\/ul>\n
Resource Based Billing<\/p><\/div>\n
Without this, anomalies were fuzzy. With it, they\u2019re crystal clear.<\/p>\n
Pivot Table at the Account Level With AWS Tags<\/strong><\/h3>\n
A pivot table that displays the price of each service according to usage type has been made in order to mimic the visibility provided by the AWS Billing console. Although it offers more customisation and granularity, this view is very similar to the “Bills” section of the AWS console.<\/p>\n
The Reason for Developing the Account-Level Pivot Table<\/h3>\n
This pivot table was created to support projects that enable content owners to:<\/p>\n
- \n
- Establish, plan, and automate news, FAST, and live video streaming channels.<\/li>\n
- Distribute those channels among various endpoints and platforms.<\/li>\n
- Learn about performance, cost-effectiveness, and channel-level content delivery.<\/li>\n<\/ul>\n
Because of this architecture, every channel frequently functions within the system as a separate client or unit. As a result, content owners must:<\/p>\n
- \n
- Recognize the prices and savings offered by each channel.<\/li>\n
- Attribute AWS usage accurately to individual accounts and channels.<\/li>\n
- Make informed decisions around resource optimization and pricing strategy.<\/li>\n<\/ul>\n
The purpose of this pivot table is to give an organized and understandable breakdown of AWS prices and discounts at the Account ID and Account Name levels. The use of metadata extracted from resource tags, which enables clients to view cost and discount attribution based on their unique account and channel details, is what makes this approach unique.<\/p>\n
Account Level Table<\/p><\/div>\n
Tag-Based Identification<\/strong><\/h3>\n
The values for Account ID<\/strong>, Account Name<\/strong>, Channel ID<\/strong>, and Channel Name<\/strong> are extracted from the resource tags associated with the respective AWS resources. These tags are parsed and joined with the billing line items to produce a complete view of the cost structure, mapped accurately to client-specific identifiers.<\/p>\n
Attribution of Discounts<\/h3>\n
The table has distinct columns to show:<\/p>\n
- \n
- Usage-based cost<\/li>\n
- Total effective cost<\/li>\n
- Discounts applied, including:<\/li>\n<\/ul>\n
Enterprise Discount Program (EDP)<\/p>\n
Private Rate Card Discounts<\/p>\n
Bundled Discounts<\/p>\n
Value Proposition<\/strong><\/h3>\n
This structured and tag-enriched view helps clients understand:<\/p>\n
- \n
- Which services are contributing to their AWS bills.<\/li>\n
- What level of discount is being applied.<\/li>\n
- How usage is distributed across their own internal account and channel identifiers.<\/li>\n<\/ul>\n
More transparent chargeback and showback reporting between organisational units is made possible by this format.<\/p>\n
Why Not Just Use AWS Cost Explorer?<\/strong><\/h3>\n
While AWS Cost Explorer provides a native interface for cost analysis, it comes with certain limitations:<\/p>\n
\n\n
\n Feature<\/strong><\/td>\n AWS Cost Explorer<\/strong><\/td>\n Custom Pivot Table View<\/strong><\/td>\n<\/tr>\n \n Tag-based account\/channel mapping<\/strong><\/td>\n Limited flexibility<\/td>\n Full customization via tags<\/td>\n<\/tr>\n \n Custom column hierarchy<\/strong><\/td>\n Not supported<\/td>\n Fully configurable<\/td>\n<\/tr>\n \n Discounts breakdown<\/strong><\/td>\n Aggregated view<\/td>\n Separated discount attribution<\/td>\n<\/tr>\n \n Offline export and manipulation<\/strong><\/td>\n Manual CSV download<\/td>\n Structured data, easier automation<\/td>\n<\/tr>\n \n Resource-level traceability<\/strong><\/td>\n Partially supported<\/td>\n Fully supported<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n This pivot table approach is particularly beneficial when there is a need to provide cost visibility per client<\/strong>, per channel<\/strong>, or per project<\/strong>, in a way that aligns with internal organizational tagging strategies.<\/p>\n
Conclusion<\/strong><\/h3>\n
Switching from manual reviews to an automated, tag-driven QuickSight dashboard has been a game-changer for us.<\/p>\n
We\u2019re no longer stuck in spreadsheet hell. We can see anomalies almost in real time. Finance finally has the breakdowns they wanted. Engineers can trace spend back to actual resources. And cost reviews have gone from dreaded chores to genuinely useful conversations.<\/p>\n
If you\u2019re still leaning only on Cost Explorer and CSV exports, take it from me: it\u2019s worth building something better. Start small, keep tags clean, and let QuickSight + Athena do the heavy lifting.<\/p>\n
Because at the end of the day, it\u2019s not just about saving money. It\u2019s about clarity, accountability, and being able to answer the question we all hate: \u201cWhy did the AWS bill go up again?\u201d<\/p>\n
- Monthly Comparison<\/strong>: Similar layout and logic, comparing the last two completed months instead of weeks.<\/li>\n<\/ul>\n
- Athena<\/strong>: SQL queries over the CUR. We built views to simplify common breakdowns (by service, by usage type, by resource tags).<\/li>\n