Continuous configuration and compliance monitoring with AWS Config

Config splash

I wrote previously about inventorying your assets in AWS using an external open source tool.  An alternative to this approach is to use AWS Config.

This AWS service certainly has its imperfections (e.g. it doesn’t support all AWS resources) but it is easy to set up and can be quite useful too. When you first enable it, Config will analyse the resources in your account and make the summary available to you in a dashboard (example below). It’s a regional service, so you might want to enable it in all active regions.


Config, however, doesn’t stop there. You can now use this snapshot as a reference point and track all changes to your resources on a timeline. It can be useful when you need to analyse historical records, demonstrate compliance or gain visibility in your change management practices. It can also notify you of any configuration changes if you set up SNS notifications.

Config rules allow you to continuously track compliance with various baselines. AWS provide quite a few out of the box and you can create your own to tailor to the specific environment you operate in. You have to pay separately for rules, so I encourage you to check out pricing first.

As with some other AWS services, you can aggregate the data in a single account. I recommend using the account used for security operations as a master. You will then need to establish a two-way handshake, inviting member accounts and authorising the master account to be able to consolidate the results.



How to detect threats in AWS with GuardDuty


Once some basic asset management, identity and access management and logging capabilities in AWS have been established, it’s time to move to the threat detection phase of your security programme.

There are several ways to implement threat detection in AWS but by far the easiest (and perhaps cheapest) set up is to use Amazon’s native GuardDuty. It detects root user logins, policy changes, compromised keys, instances, users and more. As an added benefit, Amazon keep adding new rules as they continue evolving the service.

To detect threats in your AWS environment, GuardDuty ingests CloudTrail, VPC FlowLogs and VPC DNS logs. You don’t need to configure these separately for GuardDuty to be able to access them, simplifying the set up. The price of the service depends on the number of events analysed but it comes with a free 30-day trial which allows you to understand the scope, utility and potential costs.

It’s a regional service, so it should be enabled in all regions, even the ones you currently don’t have any resources. You might start using new regions in the future and, perhaps more importantly, the attackers might do it on your behalf. It doesn’t cost extra in the region with no activity, so there is really no excuse to switch it on everywhere.

To streamline the management, I recommend following the AWS guidance on channelling the findings to a single account, where they can be analysed by the security operations team.


It requires establishing master-member relationship between accounts, where the master account will be the one monitored by the security operations team. You will then need to enable GuardDuty in every member account and accept the invite from the master.

You don’t have to rely on the AWS console to access GuardDuty findings, as they can be streamed using CloudWatch Events and Kinesis to centralise the analysis. You can also write custom rules specific to your environment and mute existing ones customising the implementation. These, however, require a bit more practice, so I will cover them in future blogs.

How to set up CloudTrail logging in AWS

I bet you already know that you should enable CloudTrail on your AWS accounts, if you haven’t yet. This service captures all the API activity taking place in your AWS account and stores it in an S3 bucket for that account by default. This means you would have to configure the logging and storage permissions for every AWS account your company has. If you are tasked with securing your cloud infrastructure, you will first need to establish how many accounts your organisation owns and how CloudTrail is configured for them. Additionally, you would want to have access to S3 buckets storing these logs in every account to be able to analyse them.

If this doesn’t sound complicated already, think of a potential error in permissions where logs can be deleted by an account administrator. Or situations where new accounts are created without your awareness and therefore not part of the overall logging pipeline. Luckily, these scenarios can be avoided if you are using the Organization Trail.

Create Trail

Your accounts have to be part of the same AWS Organization, of course. You also would need to have a separate account for security operations. Hopefully, this has been done already. If not, feel free to refer to my previous blogs on inventorying your assets and IAM fundamentals for further guidance on setting it up.

Establishing an Organization Trail not only allows you to collect, store and analyse logs centrally, it also ensures all new accounts created will have CloudTrail enabled and configured by default (and it cannot be turned off by child accounts).


Switch on Insights while you’re at it. This will simply the analysis down the line, alerting unusual API activity. Logging data events (for both S3 and Lambda) and integrating with CloudWatch Logs is also a good idea.

Where can all these logs be stored? The best destination (before archiving) is the S3 bucket in your account used for security operations, so that’s where it should be created.

S3 bucket

Enabling encryption and Object Lock is always a good idea. While encryption will help with confidentiality of your log data, Object Lock will ensure redundancy and prevent objects from accidental deletion. It requires versioning to be enabled and is best configured on bucket creation. Don’t forget to block public access!

S3 block public access

You will then must to use your organisational root account to set up Organization Trail, selecting the bucket you created in your operational security account as a destination (rather than creating a new bucket in your master account).

For this to work, you will need to set up appropriate permissions on that bucket. It is also advisable to set up access for child accounts to be able to read their own logs.

If you had other trails in your accounts previously, feel free to turn them off to avoid unnecessary duplication and save money. It’s best to give it a day for these trails to run in parallel though to ensure nothing is lost in transition. Keep your old S3 buckets used for collection in your accounts previously; you will need these logs too. You can transfer them to Glacier later on.

And that’s how you set up CloudTrail for centralised collection, storage and analysis..

AWS security fundamentals: IAM


Here I am going to build on my previous blog of inventorying AWS accounts and talk about identity and access management. By now you have probably realised that your organisation, depending on its size, has more accounts with a lot of associated resources than you initially thought. The way users are created and access is managed in these accounts has a direct impact on the overall security of your infrastructure.

What accounts should your company have? Well it really depends on the nature of your organisation but I tend to see the following pattern for software development driven companies:

1. Organisation root. Your organisation root account should be used to create other accounts (and some other limited amount of operations) and otherwise shouldn’t be touched. Secure the credentials and leave it alone. It should not have any resources associated with it.

2. Identity. Not strictly necessary to have a separate account for this but isn’t it great to be able to manage all your users in a single account?

3. Operations. This account should be used for log collection and analysis. Your security team will be happy.

4, 5 and 6. A separate account for your development, staging and production environments. It’s a good idea to separate them for the ease of managing permissions and pleasing auditors.

Users and services that are managed within an AWS account, should only get access to what they need.

Security specialists are spending a great deal of their time reviewing firewall rules when working on their on-premise infrastructure to ensure they are not too permissive. When we move to the cloud, these rules look somewhat different but their importance has only increased.

To demonstrate the relationship between accounts, users, groups, roles and permissions, let’s walk through an example scenario of a developer in your company requiring read only access to the staging environment.

No automation or anything even remotely advanced is going to be discussed here as we are just covering the basics in this blog. It is no less important, however, to get these right. The principles discussed here will lay the foundation for more advanced concepts. Again, the terminology here is specific to AWS but overarching principles can be applied to any cloud environment.

To start with this scenario, let’s create a custom role CompanyReadOnly and attach an AWS managed ReadOnlyAccess policy in the Permissions tab.

Role Policies
CompanyReadOnly ReadOnlyAccess

This role allows a trusted entity (an account in this case) to access this account. When you access this account you will get the permissions defined in the policy.

Let’s say we have an account where all users are managed (the Identity account in point 2 in the list above). In this account, create a custom policy CompanyAssumeRoleStagingReadOnly allowing assuming the right role, where 123456789012 is Staging account ID which is the trusted entity for the Identity account:

    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::123456789012:role/CompanyReadOnly"

Now let’s create a custom StagingReadOnly group and attach the above policy in the Permission tab.

Group Permissions
StagingReadOnly CompanyAssumeRoleStagingReadOnly

Finally add a user to that group:

User Group Permissions
Developer StagingReadOnly CompanyAssumeRoleStagingReadOnly

In this group additional permissions can be added, e.g. AWS managed enforce-mfa policy for mandatory multi-factor authentication.

Of course, granular policies specifying access to particular services rather than blanket ReadOnly is preferred. Remember the aim here is to demonstrate IAM fundamental principles rather than recommend specific approaches you should use. The policies will depend on the AWS resources your organisation actually uses.

How to inventory your AWS assets


Securing your cloud infrastructure starts with establishing visibility of your assets. I’ll be using Amazon Web Services (AWS) as an example here but principles discussed in this blog can be applied to any IaaS provider.

Speaking about securing your AWS environment specifically, a good place to start is the AWS Security Maturity Roadmap by Scott Piper. He suggests identifying all AWS accounts in your organisation as a first step in your cloud security programme. 

Following Scott’s guidance, it’s a good idea to check in with your DevOps team and/or Finance to establish what accounts are being used in your company. Capture this information in a spreadsheet, documenting account name, ID, description and an owner at a minimum. You can expand on this in the future to track compliance with baseline requirements (e.g. enabling CloudTrail logs).

Once we have a comprehensive view of the accounts used in the organisation, we need to find out what resources these accounts use and how they are configured. We can get metadata about the accounts using CloudMapper’s collect command. CloudMapper is a great open source tool and can do much more than that. It deserves a separate blog, but for now just check out setup instructions on its GitHub page and Scott’s detailed instructions on using the collect command.

The CloudMapper report will reveal the resources you use in all the regions (the image at the top of this blog is from the demo data). This can be useful in scenarios where employees in your company might test out new services and forget to switch them off or nobody knows what these services are used for to begin with. In either case, the company ends up paying for these, so it makes economic sense to investigate, and disabling them will also reduce the attack surface.

In addition to that, the report includes a section on security findings and will alert of potential misconfigurations on the account. It also provides recommendations on how to address them. Below is an example report based on the demo data. 


As we are just establishing the view of our assets in AWS at this stage, we are not going to discuss remediation activities in this blog. We will, however, use this report to understand how much work is ahead of us and prioritise accordingly.

Of course, it is always a good idea to tackle high criticality issues like publicly exposed S3 buckets with sensitive information but don’t get discouraged by a potentially large number of security findings. Instead, focus on strategic improvements that will prevent these issues from happening in the future.

To lay the foundation for a security improvements programme at this point, I suggest adding all the identified accounts to an AWS Organisation if you haven’t already. This will simplify account management and billing and allow you to apply organisation-wide service control policies.