From a canary alert to a verdict: How Daylight investigates Tracebit alerts

Canaries can be an important part of an organization’s defense-in-depth strategy. They generate an alert when someone uses a resource nobody has a reason to use: a decoy AWS credential, a fake Kubernetes secret, an S3 bucket that serves no workload. As such, canaries provide a high-fidelity signal that something is awry.

Historically, however, managed detection and response and other security service providers have struggled with canaries because the alerts they generate don’t indicate clear intent, and you need context to determine whether a canary alert is malicious, suspicious, or benign. 

They tell you that someone or something interacted with a canary, but you still have to investigate who or how they interacted with the canary in order to determine whether or not the activity is malicious. In other words, a canary alert is a strong lead but one that might lead nowhere if you don’t have access to the necessary context..

Of course, this is an incredibly useful lead for an internal security team because they, by definition, have access to all the context and all the tribal knowledge required to adjudicate canary alerts quickly and confidently. Third-party security services, on the other hand, are limited by the visibility they have into the environment where the canary trigger fired. For example, if an adversary interacts with a canary credential in an organization’s Github repository, then your service provider almost certainly needs access to Github and identity logs to investigate the alert. If they don’t have that access, then the alert probably isn’t actionable for them, even though it could be a signal of compromise. 

Daylight was built to have that context. That's why we've integrated with Tracebit.

How the integration works

Tracebit alerts reach Daylight through the customer's existing pipeline. In today's deployments, that means Panther: Tracebit sends a webhook to Panther, and Daylight ingests the alert along with the severity the customer has assigned.

We don't re-score those alerts. A canary trigger is already a meaningful signal, so Daylight carries the customer's severity straight into a case and puts the effort where it matters: the investigation.

What the investigation looks like

A canary alert answers "was it triggered?" Daylight's job is to answer "by whom, how, and should we be worried?" For every Tracebit alert, Daylight:

• Enriches the indicators in the alert (the acting identity, the source IP, and the canary itself) against threat intelligence;
• Resolves the identity behind the activity and builds a profile of that user;
• Pulls device context from EDR and MDM for the associated machine, including host-level process activity, to pin down what actually made the call;
• Reviews identity provider activity around the time of the alert, by both IP and identity;
• Pulls the activity native to the canary's resource type, such as CloudTrail for an AWS canary or Kubernetes audit logs for a Kubernetes canary; and
• When a person can be identified, asks them directly to explain the activity, and adds their answer to the evidence.

Throughout, Daylight treats canary alerts as suspicious until something explains them.

One nuance matters a great deal. Tracebit canary resources are named to blend in with the real resources around them, and many are designed to look like attractive targets, such as a customer-data bucket or a production database token. So Daylight never treats the name as evidence of anything. The verdict depends on the access pattern: a direct, targeted grab at a specific secret is very different from a broad listing that happened to sweep up a decoy alongside everything else.

Two real-world examples 

In one case, a customer planted temporary AWS canary credentials on a developer server. The credentials were then used to run the `aws s3 ls` command against a staging bucket, presumably in an attempt to list the objects stored in that bucket. The request was denied, and it came from the NAT gateway serving that same development environment. The canary alert alone can't say whether this was an attacker who had landed on the box or a script picking up whatever credentials it found. Daylight answers that by looking at who was on the host, which process issued the command, and what else that identity did around the same time.

In another, a Kubernetes secret canary was triggered when an engineer, authenticated through an admin SSO role, ran a `kubectl` command from their laptop that listed secrets in a namespace. This is exactly where the access pattern matters. Listing every secret in a namespace is a broad sweep that will pick up the canary incidentally. Retrieving that specific secret by name is a targeted action. Daylight weighs those very differently, and the difference, combined with the engineer's own explanation, drives the verdict.

A new pattern: AI agents tripping canaries

We're increasingly seeing canaries tripped not by attackers but by AI agents that users have pointed at their own infrastructure. An engineer asks a coding agent to clean up secrets, debug an issue, or pull some data, and the agent enumerates resources, tripping a canary in the process.

While this kind of activity is often benign, there’s no way of determining that without investigating the alert. If the agent's actual behavior, such as broad enumeration or cross-account access, goes beyond what the user's explanation covers, Daylight flags the case as unresolved rather than closing it out. As agents take on more hands-on work in production environments, canaries are becoming one of the clearest ways to see when an agent goes somewhere it shouldn't, as long as someone has the context to tell the difference.

Better together

Tracebit deploys canaries across cloud, on-prem, endpoints and CI/CD. Deployment takes a few lines of Terraform or through existing automation tooling, with no agents to run. Each canary is generated so it matches the environment it sits in providing a source of high-fidelity signals. . Daylight supplies the context to turn each alert into a confident verdict: Who triggered  the canary, how, and whether it matters. Together, they let organizations get the full value of deception without adding triage work to their own team.