AWS VPC Flow Logs

All articles in

Network Security

AWS VPC Flow Logs

VPC flow logs as the name suggests capture records of network traffic data including details about the source and destination, traffic flow, network protocol, packet and the context of the connection for any inbound and outbound network activity in the AWS VPC.

Network logs are relevant for Security because they provide visibility into the traffic allowing us to detect suspicious patterns. These patterns could be known threats like a notorious IP address with a history of DDOS attacks, or sometimes could even be a zero-day in action. The network logs are also useful after security incidents to identify the perpetrators and respond to the attack by isolating resources from the compromised host. If this does not help bring the hammer down on enabling Network logs for your team, then consider that there might be compliance and legal regulations to enable these in your organization.

What are the levels at which VPC Flow logs operate?

You can create a flow log for a VPC, a subnet, or a network interface. If you create a flow log for a subnet or VPC, each network interface in that subnet or VPC is monitored.

Regardless of the type of network interface, you can create flow logs for all network interfaces even those that are created by other AWS services, such as:

  • Elastic Load Balancing
  • Amazon RDS
  • Amazon ElastiCache
  • Amazon Redshift
  • Amazon WorkSpaces
  • NAT gateways
  • Transit gateways

Is VPC Flow logs the only Network log in AWS ?

No, there are many more depending on the type of AWS services you may be using. Let’s take a look at some of the most common scenarios where you might encounter other network logs in AWS apart from VPC Flow Logs – 

  • If you are using Load balancers in front of your application, then AWS provides Access Logs that capture detailed information about requests sent to the load balancer. Taking the example of a Network Load Balancer, access logs may contain important TLS handshake like TLS cipher and TLS protocol. 
  • For AWS web services that might be offering web content directly to users e.g. AWS CloudFront and AWS S3, there are also Access Logs which contain detailed information about every user request that the service receives. 

These were just a couple of use cases, but there might be more. So it would be best to do a deep dive on Network logs if you are looking to create a comprehensive Network logs strategy on AWS.

What are the fields that can show up in a VPC Flow Log?

As of the writing of this article, there are 29 fields that can show up in VPC flow logs. Let’s categorize them into different buckets based on their functionality, most of these fields are self-explanatory once you put them in the right context so I won’t go into a deep dive explanation of each. 

  • Fields to identify the source of the request
    • srcaddr
    • srcport
    • pkt-src-aws-service
    • pkt-srcaddr
  • Fields to identity the destination of request
    • dstaddr
    • dstport
    • pkt-dst-aws-service
    • pkt-dstaddr
  • Fields with metadata about the AWS resource for which the flow is captured
    • vpc-id
    • subnet-id
    • account-id
    • instance-id
    • interface-id
    • region
    • az-id
    • sublocation-type
    • sublocation-id
  • Fields to describe the traffic and the packet within it
    • bytes
    • packets
    • flow-direction
    • traffic-path
    • protocol
    • tcp-flags
    • type
  • Fields with metadata about the context of request
    • start
    • end
    • action
    • log-status
    • version

How can we analyze Flow Logs?

Data will only be costing you $$ in storage until you analyze it and create some value addition from that analysis. So let’s look at some ways in which we can analyze the VPC Flow Logs: 

  • If you are looking for a quick troubleshooting from the AWS console, then storing the Flow Logs in CloudWatch Log groups might be an easy solution. AWS provides a feature of Logs Insight which enables you to interactively search and analyze your log data in Amazon CloudWatch Logs. You can perform queries to help you more efficiently and effectively respond to operational issues. Here is a screenshot of the same in action:
  • If you are looking for an alternate solution in AWS which provides more complex capabilities like joins or other enrichments in data, then you could send the VPC Flow Logs data to other managed AWS Data Analysis services like Athena, OpenSearch or Redshift. All three of these services have their own pros and cons, but are outside the scope of this article to describe.
  • You could use a third-party SIEM (Security Incident and Event Monitoring) tool to parse, query and alert on the VPC Flow Logs. Splunk is the first one that comes to mind when we think about SIEM, but there are a number of new and innovative startups that have come up recently in the SIEM space, so please do your due diligence before you select one.

Demo

In this security lab, we would demo the creation of VPC flow logs via 3 mechanisms or 3 levels – for VPC, for subnet and for ENIs. We would then send a network connection request to an EC2 instance and analyze the different fields that show up in the VPC Flow Logs.

Follow us on: