Tag: data

10 Questions to Ask Your Blockchain Analytics Provider about Data Quality

Blockchain analytics tools provide critical intelligence to compliance teams, regulators, and investigators. These professionals use that intelligence to uncover illicit activity, prioritize investigations, support enforcement actions, and ultimately hold bad actors accountable. But those outcomes depend on one thing: the quality of the underlying blockchain data.

If the data is wrong, investigators will waste time and resources chasing a false lead, and compliance analysts could miss sanctions exposure. The downstream effects can be even worse: a single incorrect attribution can discredit hundreds of related insights, undermine entire investigations and lead to wrongful customer terminations.

Selecting the right blockchain analytics provider is therefore a mission-critical decision. Evaluating the quality of the underlying data requires more than comparing feature lists or coverage claims. Any provider performing rigorous analytical work should be able to explain the methodology behind its conclusions, the evidence supporting them, the safeguards used to prioritize accuracy, and show that the claims hold up under scrutiny and independent testing. The following questions are designed to assess the rigor, transparency, and evidentiary standards behind a provider’s methodology as part of your due diligence.

How addresses get grouped

  1. How do you determine that multiple addresses belong to the same entity? 

Some methodologies establish common ownership deterministically. Others infer ownership  probabilistically. Both approaches can be useful, but it is important to understand which method is being used and when.

  1. What happens when your grouping methods get it wrong? 

Every technique has blind spots. For example, CoinJoin transactions need to be identified and excluded from UTXO co-spending heuristics. A good provider has mapped out these edge cases and built protections against them, and not just assumed errors are rare.

  1. Do you use different techniques for different blockchains? 

Different blockchains, like Bitcoin and Ethereum, operate in fundamentally different ways with distinct architectures, transaction models, and behavioral patterns. As a result, the techniques used to group addresses together should differ as well. If a provider uses the same general terminology across blockchains, ask what methodology is being applied.

How entities get labeled

  1. What evidence supports your labels, and how reliable is it? 

A label confirmed by a reliable source — for example a dataset seized by law enforcement — is very different from one based on a single uncorroborated report, e.g an anonymous tip.

  1. If you changed the label, would the address grouping still hold up? 

The grouping and the label should be independent. If removing a label causes the grouping to fall apart, neither claim stands on its own.

  1. Do you distinguish between who runs a wallet and who uses it? 

When you deposit crypto at an exchange, your deposit address is linked to you, but the exchange controls it. Failing to distinguish between users and service providers can lead to incorrect ownership claims and attribution errors. A provider should be able to explain how it differentiates between who uses an address and who ultimately controls it. The same challenge exists with nested entities, where one company relies on another company’s custodial or wallet infrastructure. Robust attribution requires understanding not just who interacts with an address, but who ultimately controls it.

How methodology gets tested

  1. Has your methodology been challenged in court? 

Legal proceedings test whether the methodology behind clustering and attribution can be submitted as evidence. A methodology that has satisfied the Daubert standard is fundamentally different from one that has never faced it, even if they seem similar on the surface. If a provider’s methods are relied upon in investigations, compliance decisions, or enforcement actions, understanding how those methods have performed under legal scrutiny can provide valuable insight into their rigor, methodology, and reliability.

  1. Have you participated in independent accuracy studies? 

Opportunities to verify the accuracy of blockchain analytics are rare and valuable. But if law enforcement seizes wallet infrastructure, then outside parties can compare empirical evidence with attribution data. These moments provide a unique opportunity to validate whether a provider’s methodologies produce accurate results in the real world. Has your provider welcomed that kind of external testing, or avoided it?

  1. Where do you draw the line with machine learning? 

ML is great for spotting patterns. But if ML outputs automatically get treated as confirmed facts, errors can quickly multiply. Understanding where a provider relies on machine learning can help you distinguish between evidence-based conclusions and probabilistic assessments that may require further validation. Ask how your provider leverages ML and if those outputs are not grouped with other methodologies and are clearly labelled as such.

  1. Can you explain how any given cluster was built? 

For any specific cluster, a provider should be able to walk you through how it was constructed and what evidence supports it. If they can’t trace how a cluster came together, then their cluster might not be right.

Any blockchain analytics company should be able to provide clear and specific answers to these questions. They’re the product of transparency, quality control, and strong evidentiary standards — the same standards your investigations depend on.

To learn more about Chainalysis’s data standards, read Defining the Cluster, our formal ontology for blockchain address analysis and intelligence claims, and why we published it

An Ontology for Accountability: Defining What Data Quality Means in Blockchain Analytics

We built this to be questioned

Before I came to blockchain analytics, I spent years in academia studying the formal correctness of distributed systems. My work was proving, mathematically, that code and protocols behave correctly under every possible execution variant. Not most of them. All of them. In that world, there isn’t a “good enough.” There’s proof, or there’s no proof.

I left academia to apply that thinking to real-world problems, first in financial infrastructure where failure is enormously expensive, and eventually to blockchain analytics, where failure directly impacts peoples’ livelihoods. I’ve now spent nearly a decade as Chainalysis’s Chief Scientist, and I’m ultimately responsible for the quality of our data. That responsibility is something I carry into every decision about how our systems work and the claims they make.

I mention this background because it explains something about how I approach this field, and why the following moment that happened a few years back changed everything.

The moment

A customer came to me with a problem. They had two blockchain analytics tools giving them two very different answers about the same deposit address. Ours called it a gambling service. The other tool called it CSAM.

This was not a disagreement about whether an address belongs to Exchange A or Exchange B. Rather, it was a  disagreement where one answer means a person placed a bet, and the other means they may have purchased child sexual abuse material.

I looked at it and quickly understood what had happened. Small, regular payments of similar size and frequency can look alike if all you’re doing is matching statistical patterns. A small-time gambler and something far darker can produce similar transaction footprints when viewed through a narrow enough lens. That’s a dangerous conclusion, and a reckless foundation to build it on – no evidence, just appearance.

That kind of conclusion cannot easily be independently proven by the people who act on it. People rely on this data and may treat it as fact. We have proven that it can be relied on, if done right. This, however, was exploitation of that trust. And it could have terrible consequences.

Building research-grade systems for a field that didn’t exist

When I joined this space, blockchain analytics wasn’t an established discipline. There were no textbooks, no accreditation bodies, no precedent. The problems were novel. It was research. And we were building systems to be used by people who didn’t fully technically comprehend blockchains, people who relied on the correctness of our work and trusted it. Law enforcement agents building cases. Compliance teams making decisions that affect real people and their access to money. Prosecutors presenting evidence in court.

So I did what came naturally: applied academic rigor to everything we built. If it could withstand peer review, it was good enough. If it couldn’t, discard. We drew hard lines between what could be proven through deterministic, reproducible on-chain analysis, and what required intelligence tradecraft. We enumerated failure modes and safeguarded against them. We treated the structural layer of our analysis as a scientific discipline, because that’s what it is.

When you’ve spent years proving systems correct under all conditions, you don’t ship something and hope it holds up. You build it so you can prove it holds up.

Rigor is a philosophy, not a phase

That standard wasn’t something we outgrew as the company scaled or the blockchain ecosystem evolved. As new chain architectures emerged, as the complexity of on-chain activity grew exponentially, we kept the bar exactly where it was and improved our operational processes.

When our methodology was subjected to full Daubert scrutiny in federal court in United States v. Sterlingov, it was found admissible across every criterion. I never doubted that outcome, because I knew our methods could stand up to peer review.

When independent researchers at Delft University in collaboration with law enforcement conducted the only empirical validation study of attribution accuracy against ground truth from seized infrastructure, we welcomed it. We let the results be published. We treated it as what it was: the kind of external scrutiny that any legitimate discipline should invite. Another provider tried to suppress that same study through legal threats. That tells you something. No. It tells you everything.

As the field grew, new entrants arrived. And with them came a gradual erosion that I’ve watched with growing concern. Definitions got looser. Machine learning outputs started being treated as forensic facts. I started getting questions from customers about data quality issues that shouldn’t exist if the underlying methodology were sound. The CSAM-versus-gambling moment was the most visceral, but it wasn’t the only one, and they haven’t stopped coming.

The vocabulary gap

The field is still young and the technology is genuinely new. But the deeper problem isn’t that the investigators, regulators, and courts who depend on this data don’t fully understand the underlying mechanics. It’s that they shouldn’t have to. You can’t expect everyone investigating crime in this space to appreciate every nuance and technical aspect. That’s what science gives you. Blockchain analytics hasn’t formalized that yet since the space is new. We’re building toward the kind of standards that mature scientific disciplines have established.

That’s why we wrote the ontology. Neither as a product specification nor a marketing document, but formal paper that outlines the standards we have upheld since our founding that breaks down the work we do into its constituent parts, assigns each an evidentiary standard, and gives the industry a vocabulary for accountability.

It defines two tiers. The layer that determines whether addresses share common control, must meet a standard of structural soundness: deterministic, reproducible, auditable, with known and documented failure modes. The attribution layer, linking those addresses to a named entity, follows a structured and recognizable confidence framework: source characterization with documented reasoning requirements. Both are rigorous. But they’re different kinds of rigor, and conflating them is how you confuse people, and failing to appreciate them is how you end up labeling a gambler as a CSAM purchaser.

Why this makes me proud

This paper formalizes the principles that have guided our methodology since our founding, while acknowledging that our implementation continues to evolve. It required us to name what we’ve been doing all along. The standards it defines are the standards we already hold ourselves to. The bright lines it draws are bright lines we drew years ago, because nothing less would withstand peer review.

The consumers of blockchain analytics data don’t always have the information to perform their own peer review. What makes me proud is working for a company that does not violate the trust those people place in us. Every system we designed, every heuristic we deployed, every analytical claim we made was built with the assumption that it would one day be examined by someone adversarial, skeptical, and technically capable, and that it needed to hold up. And it has held up. Because we built it that way.

But pride in our own work isn’t enough. The people affected by blockchain analytics outputs, the subjects of investigations, the users flagged by compliance systems, the defendants in criminal cases, deserve an industry that holds itself to this standard collectively. This paper is a first step, not a final word. We’re publishing our definitions because someone needs to take the lead in responsible transparency. Someone has to establish the boundaries. We believe we have. Now we’re inviting the industry to build on them with us.