VPC Flow Logs: A Practical Guide for Security & Compliance
A lot of teams only realize they need VPC Flow Logs after an incident has already gone sideways.
A workload starts behaving oddly. An analyst sees suspicious outbound connections. Someone asks the most basic question in cloud incident response: what else did this instance talk to, when, and was that traffic allowed or blocked? If you don't have a network record already flowing into your monitoring stack, you're left reconstructing events from fragments. Security groups, application logs, endpoint alerts, and cloud control-plane events help, but they don't replace a direct record of network conversations.
That's why VPC Flow Logs matter. They give you a durable, agentless record of Layer 3 and Layer 4 traffic across your cloud network. In practice, that means better incident response, cleaner troubleshooting, more defensible compliance evidence, and a way to spot traffic patterns that are hurting both security posture and cloud spend.
Table of Contents
- Why Your Cloud Needs Network Visibility
- What Are VPC Flow Logs Explained
- Decoding the VPC Flow Log Schema
- Enabling and Managing Flow Logs
- Flow Logs Across AWS Azure and GCP
- Using Flow Logs for Threat Detection in a SIEM
- Best Practices and Avoiding Common Pitfalls
Why Your Cloud Needs Network Visibility
The ugly part of cloud incidents isn't usually the first alert. It's the uncertainty that follows.
A server gets flagged for suspicious behavior, but the team can't quickly answer whether it reached storage services, another subnet, or an external address. That gap slows triage. It also creates risk, because responders start making assumptions instead of working from evidence.
VPC Flow Logs solve that specific problem. They provide the network ground truth for accepted and rejected IP traffic, which makes them useful for threat hunting, root cause analysis, and validating whether controls behaved the way you expected.
What teams miss without them
Without flow data, three common workflows become much harder:
- Incident response: Analysts can see a host alert, but not the surrounding network context.
- Security policy tuning: Engineers know a connection failed, but can't tell whether the pattern is isolated or systemic.
- Audit readiness: Compliance teams need evidence of network access controls, not just screenshots of configuration.
That last point matters more than many teams expect. In regulated environments, network evidence often has to stand up after the fact, not just during a live incident. If you're building a broader cloud security monitoring practice, flow logs are one of the first telemetry sources worth standardizing.
Practical rule: If an analyst can't answer "who talked to whom, when, and was it allowed?" within minutes, the cloud visibility stack is incomplete.
The operational payoff
Flow logs also help outside of pure security. Operations teams use them to investigate noisy endpoints, unexpected traffic paths, and connectivity problems that don't show up clearly in application logs.
The key benefit is that this data exists below the application layer but above packet capture complexity. You don't need to deploy agents on every instance just to understand whether communication happened. You need a reliable network record and a way to search it fast.
That's the core "so what" of VPC Flow Logs. They turn cloud networking from guesswork into evidence.
What Are VPC Flow Logs Explained
A common cloud incident starts the same way. An alert fires on a workload, the team checks security groups and route tables, and the first real question is still unanswered: did this system communicate with anything suspicious?
VPC Flow Logs answer that question with network metadata. They record who talked to whom, over which port and protocol, during what time window, how much traffic moved, and whether AWS accepted or rejected the connection. For security teams, that turns a vague host alert into something you can investigate. For operations teams, it separates a broken application from a blocked path. For compliance teams, it creates a searchable record of network access that can be retained and reviewed later.
What they capture
AWS VPC Flow Logs are an agentless Layer 3 and Layer 4 telemetry source. They capture metadata such as source and destination IP addresses, source and destination ports, protocol, byte counts, packet counts, timestamps, and the final action such as ACCEPT or REJECT.
That design matters in production. Teams get network evidence without installing agents on every instance, and collection happens outside the application stack. In practice, that makes flow logs one of the easiest telemetry sources to standardize across accounts and VPCs. It also makes them a strong fit for centralized analysis in a SIEM such as UTMStack, where analysts can correlate network activity with IAM events, endpoint alerts, DNS logs, and cloud control plane changes.
AWS also aggregates these records over a capture window rather than logging every packet as a separate event. That reduces collection overhead, but it changes how analysts should use the data. Flow logs are well suited for identifying patterns, denied connections, scanning behavior, unexpected east-west traffic, and data transfer anomalies. They are less suited for packet-level forensics.
What they do not capture
Flow logs are metadata only. They do not include packet payloads, HTTP headers, SQL statements, API bodies, TLS certificates exchanged in session content, or the command that ran on the host.
They also do not cover every traffic path in AWS. Some traffic types are excluded, which means teams that treat flow logs as full network ground truth create blind spots. That matters during investigations. If an analyst expects to see every DNS request, every internal service exchange, or every platform-managed connection in one dataset, they can draw the wrong conclusion from missing records.
The practical rule is simple. Use flow logs to confirm that communication occurred and how it was handled. Use packet capture, application logs, DNS logs, WAF telemetry, and endpoint data when you need to know what happened inside the session.
Flow logs answer whether a connection occurred, where it went, and whether it was allowed. They do not explain the application transaction inside that connection.
Why that boundary matters
Security outcomes improve when teams assign the right job to the right data source.
Use VPC Flow Logs to:
- Confirm communication paths: Identify the source, destination, port, protocol, and allow or deny result for a suspected connection.
- Spot control failures: Find repeated
REJECTevents, unexpected allowed outbound traffic, or traffic that bypasses expected inspection paths. - Investigate exposure: Verify whether an internet-facing asset received connections it should never have accepted.
- Build historical context: Reconstruct connection patterns during the period around an alert, outage, or audit request.
- Feed SIEM detections: Correlate flow activity in UTMStack with threat intelligence, asset context, and other cloud logs to surface scanning, lateral movement, and data transfer anomalies.
Do not use them as a substitute for full packet inspection or application-layer monitoring. A rejected SSH attempt, a successful database connection, and an outbound call to a suspicious IP can all appear in flow logs. The query sent over that database session or the command executed after login will not.
That trade-off is exactly why flow logs are so useful. They are broad, cheap to collect compared with packet capture, and easy to centralize, but they only become operationally valuable when the team understands their limits and parses them in context.
Decoding the VPC Flow Log Schema
A flow log record looks simple until an analyst has to answer a hard question fast: Was this failed connection attempt harmless noise, a scan, or the first sign of lateral movement? The schema determines how quickly that answer comes together.
AWS began with a smaller core record and later added fields that make investigations far more precise. As noted in Astuto's field-by-field analysis, newer versions introduced fields such as pkt-dst-aws-service, flow-direction, and traffic-path. Those additions matter because they turn a basic network record into something a SIEM can classify, correlate, and alert on with far less guesswork.
Identity fields
Identity fields answer the first investigation question: which asset was involved?
The fields analysts use first are usually:
account-idto confirm ownership in multi-account estatesinterface-idto tie the record to a specific ENIsrcaddranddstaddrto identify the endpoints
These fields are what let a team move from an IOC to scope. In UTMStack, that usually means pivoting from a flagged IP or ENI to every related connection, then enriching the result with asset tags, cloud account context, and prior alerts. Without that mapping step, flow logs stay isolated records instead of investigation evidence.
Traffic fields
Traffic fields answer the next question: what happened on the wire?
The daily workhorses are:
srcportanddstportfor service contextprotocolfor transport identificationpacketsandbytesfor session size and shapeactionfor the allow or deny decision
Practical security value is evident in scenarios like these: Repeated REJECT records from one source to many destination ports often support a scan hypothesis. Allowed outbound connections with unusually high bytes values can justify an exfiltration review. A spike in denied RDP or SSH traffic against a management subnet points to exposure or poor filtering. The fields are simple, but the outcome is not. They help analysts decide whether to escalate, suppress, or correlate with endpoint and identity telemetry.
Metadata fields
The newer metadata fields are often the difference between a quick answer and a slow investigation.
flow-direction separates ingress from egress, which matters immediately when triaging suspicious outbound traffic. traffic-path helps explain whether traffic stayed inside expected paths or moved in a way that deserves review. pkt-dst-aws-service adds useful context for cloud-native activity. If a workload is sending data to an AWS service, the team can tell the difference between expected S3 use and an unexpected transfer pattern that needs scrutiny.
Focus on the fields that answer security questions fast.
That is also where many teams create blind spots. If the parser in your SIEM only keeps the older default fields, newer context never reaches detections. UTMStack should normalize these fields at ingest, not leave them buried in raw text, or analysts will miss the distinction between internal east-west traffic, internet egress, and service-bound connections.
A practical way to parse the record
A useful triage model is to read each flow record in three passes:
| Question | Useful fields | Why it matters |
|---|---|---|
| Who talked | srcaddr, dstaddr, interface-id, account-id |
Establishes ownership and the communication pair |
| What happened | srcport, dstport, protocol, action |
Shows service path and the allow or deny result |
| How much and when | packets, bytes, start, end |
Adds timing, duration, and transfer context |
That structure works well in a SIEM because it maps cleanly to detections and dashboards. Analysts can build one view for denied connection patterns, another for unusual outbound volume, and another for service-specific traffic such as flows headed to S3 or DynamoDB. The schema only becomes useful when the fields are parsed with a security question in mind.
Enabling and Managing Flow Logs
The first design choice isn't technical. It's operational. Where do you need visibility first?
AWS lets you enable flow logs at the VPC, subnet, or Elastic Network Interface level. The right scope depends on whether you're trying to establish broad baseline visibility, watch a sensitive segment, or investigate a specific workload.
Choosing the right scope
A practical model looks like this:
- VPC level: Best when you need foundational visibility fast across an environment.
- Subnet level: Useful for isolating sensitive application tiers, management zones, or regulated segments.
- ENI level: Best for targeted troubleshooting or high-value assets that justify tighter observation.
A lot of teams start too narrow and then regret the blind spots. Starting broad is usually easier for investigations, then narrowing or filtering based on what you use.
Choosing the destination
AWS classifies VPC Flow Logs as vended logs, and cost is driven by the volume captured plus the storage destination you choose, such as CloudWatch, S3, or Firehose, according to the Vantage handbook entry on VPC Flow Logs pricing.
That pricing model makes destination choice part of your architecture, not just a logging preference.
| Destination | Best use | Trade-off |
|---|---|---|
| CloudWatch Logs | Fast search and native AWS workflows | Can become expensive for large sustained volumes |
| Amazon S3 | Long-term retention and archival analysis | Less immediate for active detection unless paired with downstream analytics |
| Data Firehose | Streaming into analytics and SIEM pipelines | Requires more pipeline design and operational ownership |
What works in practice
For many teams, the split is straightforward. Use CloudWatch when operators need quick native access. Use S3 when retention and cost control matter more. Use Firehose when the primary goal is to stream telemetry into a central detection platform.
The same Vantage reference notes that organizations also use this data to identify high-latency traffic between regions or zones and re-place workloads to reduce unnecessary transfer costs. That's one of the more overlooked benefits of flow logs. They aren't just a security artifact. They can expose expensive architecture habits.
A short walkthrough helps if you're standardizing this across teams:
Management habits that prevent pain
Once enabled, treat flow logs like any other production telemetry source.
- Verify permissions early: Log delivery failures are operational failures, not minor misconfigurations.
- Set retention deliberately: Compliance, investigation, and cost goals rarely point to the same retention period.
- Review log volume regularly: Flow data can expand quickly in busy environments, especially when teams enable broad collection and forget it.
Good flow logging isn't just "turned on." It's scoped, routed, retained, and checked.
Flow Logs Across AWS Azure and GCP
Multi-cloud teams don't get to think about flow logs as an AWS-only topic. They need a normalized view across providers, and each platform packages network telemetry a little differently.
AWS centers flow logs around the VPC, subnet, and ENI. Azure teams often work with NSG-focused logging, which tends to emphasize rule decisions. GCP also provides VPC-level network telemetry, but the operational feel is different again. For security architects and MSSPs, the challenge isn't just collection. It's normalization.
Cloud Flow Log Comparison
| Feature | AWS VPC Flow Logs | Azure NSG Flow Logs | GCP VPC Flow Logs |
|---|---|---|---|
| Primary scope | VPC, subnet, ENI | NSG-focused network visibility | VPC network visibility |
| Typical security question | Who talked to whom, and was traffic allowed | Which NSG rule allowed or denied the connection | What traffic patterns exist across the virtual network |
| Format style | Commonly space-delimited, customizable | Commonly structured records | Commonly structured records |
| Operational strength | Broad AWS network context | Policy-centric visibility | Strong fit for GCP network observability |
| Common normalization need | Map ENIs, ports, actions, and timing | Map rule outcomes and network context | Map flow metadata into shared fields |
| SIEM challenge | Parsing custom schemas consistently | Preserving NSG rule context | Aligning provider fields to common detection logic |
What matters more than provider differences
The useful question isn't which provider has the "best" flow logs. It's whether your team can answer the same security questions regardless of cloud:
- Can analysts trace east-west and outbound traffic consistently
- Can engineers distinguish rejected traffic from missing traffic
- Can auditors see evidence of network control behavior across environments
In a hybrid estate, the hard part isn't collecting network telemetry. It's making AWS, Azure, and GCP tell a coherent story.
A practical normalization model
When centralizing cloud flow logs, normalize around a few stable concepts:
- Endpoint identity such as source, destination, interface, subscription, or account.
- Transport context such as ports, protocol, and action.
- Timing and volume such as start, end, packets, and bytes.
- Provider-specific enrichment such as AWS service destination context or Azure rule metadata.
If you get those mappings right, detections become portable. If you don't, every cloud ends up with its own rule set, its own dashboard logic, and its own blind spots.
Using Flow Logs for Threat Detection in a SIEM
VPC Flow Logs become much more valuable once they leave raw storage and enter a SIEM pipeline that can parse, normalize, correlate, and alert.
On their own, they already support important investigations. They help teams tighten security group rules, identify a noisy endpoint generating unexpected load, and determine the volume of data exfiltration during an incident. They also work best when integrated with SIEM systems for real-time monitoring and alerting, as described in Query's discussion of flow log applicability and limitations.
What raw logs won't do well
A raw flow record doesn't tell you whether a destination is expected for that workload. It doesn't know whether the source host had an endpoint alert an hour earlier. It doesn't know whether repeated rejections from the same source are normal noise or reconnaissance.
That's why centralization matters. A centralized log management approach gives analysts one place to combine flow telemetry with identity events, endpoint alerts, firewall records, and cloud control-plane activity.
Detection patterns that work
The best flow-log detections are simple, explainable, and tied to actual operations.
- Port scanning: One source produces repeated
REJECToutcomes across multiple destination ports or multiple hosts in a short period. - Possible exfiltration: A workload that normally talks internally starts sending unusually large outbound byte volumes to an uncommon destination.
- Suspicious beaconing: A host establishes repeated small connections at regular intervals to a destination your team doesn't recognize.
- Security group tuning candidates: Repeated rejected flows between systems that are supposed to communicate indicate broken policy or drift.
Here's a practical way to think about query logic:
| Detection goal | Core fields | Analyst interpretation |
|---|---|---|
| Scan activity | srcaddr, dstport, action |
Many rejects from one source across many targets or ports |
| Exfiltration review | srcaddr, dstaddr, bytes, flow-direction |
Outbound transfer pattern that doesn't match workload behavior |
| Lateral movement triage | srcaddr, dstaddr, dstport, action |
New east-west communications between unrelated systems |
| Policy validation | interface-id, action, dstport |
Repeated denies show where rules are too broad or too tight |
Parsing rules matter more than dashboards
A lot of SIEM projects fail at the ingestion layer. The team forwards logs but doesn't normalize key fields consistently, so detection rules become brittle.
For AWS flow data, parse and preserve at least:
- Network identity: source, destination, interface, account
- Transport details: ports, protocol, action
- Session context: bytes, packets, timestamps
- Cloud enrichment: service destination and traffic direction when available
Then correlate those records with:
- Threat intelligence hits
- Asset criticality
- Cloud ownership tags
- Endpoint or identity alerts from the same time window
Analyst habit: Treat flow logs as relationship data. Their value jumps when you connect them to asset, identity, and alert context.
That correlation step is what turns a pile of network metadata into an actual investigation queue.
Best Practices and Avoiding Common Pitfalls
Teams get in trouble with VPC Flow Logs when they treat them as proof of visibility instead of one network telemetry source among several. That mistake shows up later during an incident, when the SOC expects to answer an application-layer question and only has IPs, ports, accepts, rejects, byte counts, and timestamps.
AWS flow logs are useful because they show who talked to whom, over what port, and whether the traffic was allowed or denied. They do not include packet payloads. That means they can support detections for scanning, lateral movement, denied egress, and unusual transfer volume, but they cannot confirm SQL injection, stolen session tokens, or malicious API parameters on their own.
The blind spots that matter in practice are usually operational, not theoretical:
- Treating flow logs as full-fidelity network evidence. They are metadata. Use them to narrow scope, validate policy behavior, and pivot to better sources for content-level analysis.
- Assuming enabled means complete. Logging configuration, cloud service behavior, and architectural choices can all reduce what you capture.
- Sending logs to a SIEM without parser checks. If
action,srcaddr,dstaddr,bytes, or direction fields arrive inconsistently, correlation rules break unnoticed. - Keeping every record forever without a plan. Storage costs rise fast, and analysts still struggle if the retained data is noisy and unprioritized.
AWS also documents limits that affect coverage in specific designs, including cases involving VPC Block Public Access on its VPC Flow Logs limitations page. The practical lesson is simple. Validate what your environment does and does not log before you build detections or compliance attestations around it.
A sound operating model is less complicated than many teams make it.
- Collect with intent. Pick fields, retention, and destinations based on concrete use cases such as threat hunting, segmentation validation, PCI evidence, or egress monitoring.
- Test for absence. Generate known traffic, blocked traffic, east-west traffic, and internet-bound traffic, then confirm each case appears as expected.
- Tune for cost and value. High-volume subnets, chatty services, and long retention windows can create expensive noise. Filter and tier retention based on investigative value.
- Correlate in the SIEM. Flow logs become far more useful after enrichment with asset tags, identity context, threat intelligence, and endpoint alerts. A dedicated network traffic analysis workflow helps analysts move from raw metadata to decisions.
- Document the gaps. Compliance teams and incident responders need to know where flow logs are authoritative, where they are supportive, and where another control owns visibility.
One warning is worth repeating. A busy dashboard can hide a weak logging design.
UTMStack helps centralize flow telemetry with endpoint, cloud, and compliance data so detections, investigations, and evidence collection happen in one place instead of across disconnected tools.
