Disclaimer: The views and opinions expressed in this article belong solely to the author and do not necessarily reflect those of ISC2.
As organizations increasingly rely on external cloud platforms such as Amazon Web Services (AWS), maintaining a clear understanding of what’s going on with external services is essential. Based on his experience, Jatin Mannepalli CISSP, CCSP, argues that effective logging is one of the most critical aspects of securing any cloud environment. He considers why having the right logs and knowing how to use them can make or break your security posture.
When talking about logging, I can't help but think of some real-world incidents that have had a lasting impact. In 2022, Pegasus Airlines experienced unauthorized access to its AWS environment, leading to significant operational disruptions. The 2019 Capital One breach, in which a simple AWS misconfiguration compromised over 100 million records, showing how devastating small mistakes can be. In June 2021, Turkish beauty brand Cosmolog Kozmetik suffered a data leak due to a misconfigured Amazon S3 bucket. Then there’s the Verkada ransomware attack, in which inadequate logging delayed threat detection and response, causing both financial and reputational harm.
These incidents highlight a critical truth: that, without robust logging and monitoring, it’s not just your infrastructure at risk, but your organization’s reputation, privacy, and future. I’ve often emphasized to stakeholders how effective logging not only meets compliance requirements like GDPR, SOX or HIPAA, but also inspires confidence among security teams and reduces the likelihood of costly breaches.
Centralizing logs in a security information and event management (SIEM) system has been a game-changer for teams I’ve worked with. They provide real-time analysis and detection capabilities, enabling swift incident response and proactive defenses. However, it all starts with understanding which AWS log sources are most important and how to integrate them into your SIEM effectively.
Critical AWS Logs and Why They Matter
Integrating AWS logs with a SIEM system is not just about checking a box; it’s about creating actionable visibility into your environment. It isn’t easy to secure what you can't see. Below are the log sources I’ve found to be the most impactful in real-world scenarios and how they’ve helped me and my teams secure and track AWS environments effectively and efficiently.
| AWS Service | How to Forward | Why They Matter |
| AWS CloudTrail Logs | Enable CloudTrail logging to an S3 bucket or CloudWatch Logs. Use an S3-triggered Lambda function, CloudWatch Logs subscription filter or Kinesis Firehose to stream logs directly to the SIEM. Configure your SIEM to parse and monitor for suspicious activities like unauthorized API calls or IAM role changes. |
Whether it’s unauthorized access or suspicious admin changes, they provided the visibility needed to act swiftly while providing non-repudiation. These logs have covered my bases countless times by tracking every action in the AWS accounts. |
| AWS Config Logs | Use Amazon Kinesis Data Firehose or AWS Lambda to stream logs from an S3 bucket to your SIEM. Ensure secure data transfer with encryption and appropriate IAM permissions. Configure the SIEM for log ingestion, parsing and monitoring of configuration changes, to detect non-compliant states and enforce security standards. | These logs have been invaluable for spotting configuration drifts and policy violations before they caused trouble. They’re essential for tracking resource changes and catching misconfigurations, like open security groups or overly permissive IAM roles. |
| VPC Flow Logs | Enable VPC Flow Logs to CloudWatch or S3. Use Lambda or a CloudWatch subscription filter to forward logs to your SIEM or leverage Kinesis or SQS for scalable, real-time or batch processing. Ensure logs are parsed and monitored for security events such as non-standard ports, malicious domains or unauthorized DNS server traffic. | When analyzing network behavior or investigating anomalies, these logs provide a detailed history of traffic patterns. They’ve been particularly useful for detecting unauthorized access attempts, especially when you are investigating a breach. |
| AWS GuardDuty Logs | Export GuardDuty findings to CloudWatch Events or an S3 bucket. Use a Lambda function triggered by these exports to transfer logs to your SIEM for analysis and continuous security monitoring. | GuardDuty logs are indispensable for identifying and responding to security threats in AWS. They’ve flagged suspicious activity, like unauthorized API calls, brute-force attempts and traffic from known malicious IPs. GuardDuty's actionable insights have helped me quickly prioritize and address potential threats. |
| AWS S3 Access Logs | Enable Server Access Logging in the S3 console and configure a target bucket to store logs. Use an S3 event-triggered Lambda function or AWS Kinesis Firehose to process and send logs to the SIEM. Configure the SIEM to parse and monitor for suspicious activities like unauthorized access or data exfiltration. | Once, during an investigation of a suspected unauthorized data access, these logs helped to pinpoint exactly who accessed an S3 bucket and when, saving critical time. |
| AWS WAF Logs | Enable WAF logging in the WebACL settings and choose either Kinesis Data Firehose or an S3 bucket as the log destination. Use Kinesis Firehose to stream logs directly to your SIEM or configure a Lambda function triggered by S3 to forward logs. Monitor web-based threats effectively through comprehensive log analysis. | These logs have been instrumental in improving my approach towards building secure architecture around web-based attacks. They have often provided me with the details (http traffic logs) to block malicious activity and fine-tune WAF rules. |
| AWS Lambda Function Logs | Enable logging to Amazon CloudWatch and set up a CloudWatch Logs subscription filter or use Kinesis Data Firehose to stream logs to your SIEM. Alternatively, install and configure a Lambda Extension from your SIEM provider to forward logs directly from the Lambda function to the SIEM, bypassing CloudWatch for reduced latency and greater flexibility. | Debugging Lambda issues or detecting unauthorized function executions would have been nearly impossible without these logs, given they are server-less and leave no trace except for the log files. They’ve also proven to be critical for auditing and forensic analysis. |
| AWS CloudWatch Logs | Ensure necessary logs are captured in relevant Log Groups. Create a Subscription Filter in CloudWatch Logs and choose a Lambda function, Kinesis Data Firehose or Kinesis Stream as the destination to process and send logs to your SIEM. Alternatively, export logs to an S3 bucket and use S3 event notifications to trigger a Lambda function for forwarding. | AWS CloudWatch Logs have been very useful for detecting security issues, like unusual login attempts or multiple failed API calls. They’ve helped me in monitoring application behavior, identify suspicious behavior and troubleshoot incidents on the go. |
| AWS Elastic Load Balancer (ELB) Logs | Enable ELB access logging in the ELB settings and configure a target S3 bucket or CloudWatch Logs as the log destination. Set up log forwarding using an S3-triggered Lambda function, CloudWatch Logs subscription filter, or Kinesis Data Firehose to stream logs directly to your SIEM. Analyze client requests and backend access for deeper security incident insights. | These logs have been key to understanding traffic patterns, identifying anomalies and preparing for DDoS attacks before they escalate and cause any outage. |
| AWS Route 53 and Route 53 Resolver Logs | Enable DNS query logging in the Route 53 console for hosted zones and configure Route 53 Resolver query logging for VPCs. Set log destinations to CloudWatch Logs or an S3 bucket. Create a Subscription Filter in CloudWatch Logs or use an S3-triggered Lambda function or Kinesis Data Firehose to stream logs to your SIEM. Monitor for suspicious DNS activities like tunneling or reconnaissance. | DNS management and DNS security is often overlooked, but Route 53 logs have often assisted me in uncovering unauthorized DNS queries, misconfigured DNS records, unusual traffic patterns and prevent potential tunneling attacks during an active incident investigation. |
Quick Tip: For most AWS services, AWS Kinesis Firehose is ideal for high-throughput log forwarding, while Lambda excels in custom log processing. Use Firehose for simplicity and Lambda for flexibility or combine as per your specific needs.
Best Practices For AWS Log Integration
To maximize the value of AWS logs, also consider these best practices:
- Comprehensive Coverage: Ensure all critical AWS services are logged to avoid security blind spots
- Log Normalization: Standardize log formats for easier correlation and actionable insights
- Real-Time Ingestion: Avoid delays in log forwarding to enable timely incident responses
- Access Control: Encrypt logs and enforce role-based access to protect sensitive data and prevent unauthorized modifications
- Regular Audits: Periodic reviews of logging setups help close gaps, reduce costs, and improve efficiency
The Criticality of Logs
The breach examples referenced at the start of this article (at Pegasus Airlines, Capital One and Verkada) highlighted the critical role of comprehensive AWS logging. While integrating key AWS log sources into SIEMs enables proactive threat detection and swift response, I’ve also encountered challenges like alert fatigue. Irrelevant log sources can generate excessive false positives, making it harder to focus on real threats. Carefully selecting log sources is key to ensuring meaningful insights without getting overwhelmed. Therefore, it’s important to understand the balance.
Artificial intelligence (AI) and machine learning (ML) are not perfect, yet. Nonetheless, they have transformed how I approach logging. By learning from past event logs, AI-powered SIEMs can predict threats (such as DDoS or ransomware) and trigger automatic responses – something traditional systems struggle with. They have often helped me create dynamic baselines which can be a foreign concept for a traditional SIEM. This evolution has pushed SIEM solutions beyond static rules, enabling proactive defense against increasingly complex threats.
Effective log management, SIEM integration, and AI/ML have proven to empower security teams to make informed decisions, mitigate risks and keep critical data secure. A holistic logging approach not only protects infrastructure but also supports compliance and business continuity. Make reviewing and optimizing your logging strategy a priority and repetitive – it’s an essential step in staying resilient against evolving threats.
JatinMannepalli CISSP, CCSP, has over 10 years of experience in cybersecurity and risk management across IT, finance, management consulting and high-frequency trading sectors. He has held security engineering, architecture, management and consulting roles, with responsibility for designing secure systems, mitigating risks and aligning cybersecurity strategies with business goals.
Related Insights
- CCSP – Certified Cloud Security Professional
- CCSP Versus AWS Cloud Certs
- Cloud Exit Strategies: Why and How to Avoid Vendor Lock-in