AWS Cloud Operations Blog

Announcing AWS CloudTrail network activity events for VPC Endpoints

Today, we are excited to announce AWS CloudTrail network activity for VPC endpoints, a new event type that captures actions transmitted through a Virtual Private Cloud Endpoint. In this preview, this new event type captures network activity events from VPC endpoints for Amazon Elastic Compute Cloud (EC2), AWS Key Management Service (KMS), Amazon S3, and AWS CloudTrail. This feature will also expand to a wide array of AWS services in the future.

AWS customers use Amazon Virtual Private Cloud (VPC) to create a secure and isolated private cloud within the AWS public cloud. Since VPCs frequently access other AWS services, VPC customers are also setting up VPC endpoints, which create a direct connection between resources inside their VPC and these other AWS services. VPC endpoints reduce security risks and data transfer costs by removing connections between a VPC and the public Internet. With network activity events for VPC endpoints, you can view details of who is accessing resources within your network. For example, you can view logs of actions that were denied due to VPC endpoints policies or use these events to validate the impact of updating existing policies. VPC endpoint Owners previously have no way to record denied actions due to VPC endpoint policies. With this launch, AWS CloudTrail network activity events will enable greater ability to identify and respond to malicious or unauthorized actions in your data perimeter.

CloudTrail logs all API calls made by IAM principals in your account. This service now includes an opt-in feature to log actions through your VPC endpoints. IAM principals include authenticated users and applications that can make AWS requests. These logs are redacted to maintain consumer privacy but identify when there is activity from an account accessing supported services through your VPC endpoint.

In this blog post, we show you how the following steps to show you how to get started with Network Activity events. We will also cover an example scenario where an unauthorized user tries to access your S3 buckets via VPC endpoints. With this scenario, we will go through common steps on how to investigate and run queries to find the logs associated with the access denial event:

  • Prerequisite and setup will cover the steps on creating the resources we will use in this blog such as VPC, VPC endpoints, EC2, CloudTrail and IAM role.
  • Opt-in on Network Activity events in CloudTrail and CloudTrail Lake Event Data Store.
  • Simulate an authorized and unauthorized user trying to access Amazon S3 via VPC endpoints.
  • Run CloudTrail Lake query to show denial event calls into Amazon S3 due to a VPC endpoint policy.
  • Visualize query results with CloudTrail widgets.

Prerequisites

  1. Create an IAM role and name it role-ec2-s3. Attach AmazonsS3FullAccess permissions policy to the IAM role. This IAM role will be attached to the EC2 instance that will be used to access a bucket in Amazon S3.
  2. Create a VPC with a private subnet.
  3. Create a VPC Endpoint for Amazon S3, apply the following policy in your VPC endpoint. This policy only allows the role named role-ec2-s3.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::(account_id):role/role-s3"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket"
            ],
            "Resource": "*"
        }
    ]
}

4. Create an S3 bucket.

5. Create one running EC2 instance in the private subnet.

6. Attach the IAM role role-ec2-s3 into your EC2 instance.

7. Create an EC2 Instance Connect Endpoint. This endpoint will be used to conveniently SSH your EC2 instance via the management console.

8. Open the CloudTrail console and select Create trail

Figure 1. CloudTrail Dashboard

9. On the Create Trail page, for Trail name, type a name for your trail. For more information, see Naming requirements for CloudTrail resources, S3 buckets, and KMS keys.

10. For Storage location, choose Create new S3 bucket to create a bucket. When you create a bucket, CloudTrail creates and applies the required bucket policies.

11. For Log file SSE-KMS encryption, choose Enabled and name the KMS alias, for example ‘AWS-VPCE-ALIAS’

12. Leave all other fields default and click

Figure 2. Selecting Attributes for the trail

13. For Events, Disable Management Events and Enable Network activity Events.

14. For Network activity events, select amazonaws.com in the drop-down menu.

15. Select ‘Log network activity access denied events from the Log selector template drop down. This will allow you to track any ‘access denied’ events from the VPC endpoint to Amazon S3.

16. Leave all other fields default and click

Figure 3. Network activity event source

17. Review the steps previously configured and select Create Trail.

Figure 4. Successfully created network trail

18. You have successfully configured a trail to monitor the API calls made to your VPC endpoints.

Create a CloudTrail Lake Event Data Store to ingest Network activity events.

  1. From the CloudTrail console page, select Lake to open the drop-down menu, & choose Event data stores.
  2. Choose Create event data store.
  3. On the Configure event data store page, in General details, and enter a name for your EDS, for example “VPCE-Activity-EDS”
  4. Specify the one-year extendable retention pricing or the seven-year retention period options.
  5. Leave all other fields default and click next.

Figure 5. Configuring the event data store

  1. On the Choose events page, choose AWS events, and then choose CloudTrail events.
  2. For CloudTrail events, choose only Network activity events.
  3. For Network activity events, select amazonaws.com from the drop-down menu for Network activity event source.
  4. Select Log network activity access denied events from the Log selector template drop down.
  5. Choose Next.

Figure 6. Selecting network activity events for EDS

11. On the Review and create page, review your choices. Choose Edit to make changes to a section. When you’re ready to create the event data store, choose Create event data store.

Figure 7. Successfully created EDS

  1. You have successfully created an event data store.

Simulate an authorized role trying to access Amazon S3 using VPC endpoints

  1. Connect to your EC2 instance using the EC2 Instance Connect Endpoint.
  1. Run the Amazon S3 putObject command to create a folder in s3 buckets: Example command below.
    aws s3api put-object --bucket bucket name --key folder name

The result of the command should show the list of S3 buckets.

Figure 8. Results of the command should show the list of S3 buckets.

3. Confirm in the creation of the folder by accessing the S3 console a viewing the contents of the bucket where the folder was created.

Figure 9. S3 confirmation out resource creation

Simulate an unauthorized role trying to access Amazon S3 using VPC endpoints

  1. Update the VPC Endpoint Policy for Amazon S3 and change Allow to Deny.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": {
                "AWS": "arn:aws:iam::(account_id):role/role-s3"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket"
            ],
            "Resource": "*"
        }
    ]
}

2. Connect to your EC2 instance using the EC2 Instance Connect Endpoint.

3. Run the Amazon S3 putObject command to create a folder in s3 buckets: Example command below.

aws s3api put-object --bucket bucket name --key folder name

The result of the command should be an AccessDeniedException because no VPC endpoint policy allows the action.Figure 10. Results of the command should be an AccessDeniedException because no VPC endpoint policy allows the action.

Investigate VpceAccessDenied events caused by unauthorized users using CloudTrail Lake

  1. Go to CloudTrail Lake in your console and in the navigation pane choose Query under the drop down list for Lake and select the newly created event data store.
  2. Run the query below. Note to update the <Event data storeId>, (filter_start_event_time) and (filter_end_event_time) before running the query.
SELECT count(*) as VPCAccessDenied, userIdentity.arn,
    userIdentity.accountid,
    eventName,
    errorMessage,
    vpcEndpointId,
    vpcEndpointAccountId,
    sourceIPAddress
FROM <Event data store Id>
WHERE eventCategory = 'NetworkActivity'
    AND eventSource = 's3.amazonaws.com' 
    AND errorCode = 'VpceAccessDenied' 
    AND eventtime >= 'filter_start_event_time’
    AND eventtime <= 'filter_end_event_time’
    GROUP BY 
    userIdentity.arn,
    userIdentity.accountid,
    eventName,
    errorMessage,
    vpcEndpointId,
    vpcEndpointAccountId,
    sourceIPAddress
    ORDER BY eventName DESC;

The sample CloudTrail Lake query displays Network activity events with error code of VpceAccessDenied. The result set of the query includes vpcEndpointId to where the activity traversed from.

Figure 11. Sample CloudTrail Lake query to display network activities

Visualizing CloudTrail queries through widgets

  1. In the CloudTrail Query console, select Visualizer, then add to dashboard.

Figure 12. Visualizing queries through a CloudTrail dashboard

2. Enter a widget title, for example ‘VPCE-DenyEvents-Widgets’ and select create dashboard.


Figure 13. Naming CloudTrail widget

3. Enter a name for the dashboard. Confirm that the event data store is the same one created in previous steps and select create dashboard.

4. You have now visualized a query of VPC endpoint access denied actions

Figure 14. CloudTrail dashboard with widget output

Cleaning up

To avoid incurring future charges, delete the resources you created in this blog.

Conclusion

AWS CloudTrail network activity event logging enhances cloud security and network visibility. The feature enables comprehensive monitoring and auditing within Virtual Private Cloud (VPC) environments. VPC endpoint logging helps organizations detect and respond to unauthorized access attempts. As an opt-in feature, it preserves consumer privacy while eliminating the need for custom monitoring systems. Customers can easily track when external credentials access their VPC endpoints, preventing unauthorized data movement. CloudTrail Network Activity events strengthen security compliance and streamline operations by providing detailed insights into VPC endpoint interactions. To learn more about CloudTrail network activity for VPC endpoints and its capabilities, review the documentation.

Diwakar Balakrishnan

Diwakar Balakrishnan is a Solutions Architect and a member of the technical field community for Cloud Operations focusing on configuration, compliance and auditing. When he is way from his laptop, he enjoys cooking and driving muscle cars.

John Agustin

John is a Sr Technical Account Manager at AWS and an active member of Cloud Operations technical field community. He partners with enterprise customers to drive business transformation through cloud optimization and operational excellence. His expertise spans Cloud Governance, FinOps, and Data Analytics, enabling organizations to achieve their strategic cloud initiatives.

Angel Tolson

Angel is a Solutions Architect at AWS, partnered with enterprise customers providing them with technical guidance and helping them achieve their business goals.