Records from CRM, ERP, billing, spreadsheets and support
Normalize, null out generic keys and match in a cascade
One row per person or company and the source map
Per-customer metrics, campaigns, a clean CRM and AI agent
What it solves
- Each customer is counted once, even with records in the CRM, ERP, billing and spreadsheets.
- Different people who share a phone or email stop being merged into one.
- Every merge says which key it used and can be reviewed.
Business questions it answers
Ask your agent these questions once the case is live.
What you need
- The systems that hold customer records connected to Nekt
- At least one strong key in part of the base: CPF (individual tax ID), CNPJ (company tax ID), email, or one system's id stored in another
- An owner to review doubtful cases
What you get
- A table of unique entities with the best value for each field
- The map from each source record to its entity, with method and confidence
- A queue of doubtful pairs for human review
- An AI agent that finds the customer by any document, email or phone
Where to activate
LTV, cohorts, churn and overdue payments read the entity, not the source record.
Reactivation and cross-sell segments come from the single base, with opt-out honored per person.
Confirmed merges go back to the CRM through reverse ETL, with the entity id stored on the contact.
Exact rules in a cascade resolve most of the base. Probabilistic matching only comes in when documents are missing, and then the work becomes calibrating and reviewing. The detailed view has the business rules, the pitfalls and the acceptance test. Or send everything straight to your agent with the Nekt MCP.
The same customer shows up with a different name, phone and document in each system, and several times inside the same CRM. In one real base, not a single customer on the accounts receivable list matched a CRM contact by name. Matching by name finds nothing, and matching only by phone or email merges relatives, coworkers and the front desk number into one customer. The result is LTV, overdue payments and campaigns computed on the wrong identities.
How it is usually done, and the rule
Each row is a mistake that already happened in practice.
CPF and CNPJ digits only, left-padded with zeros to 11 (CPF) or 14 (CNPJ): a document read as a number loses its leading zero and does not match. Email lowercased, trimmed and validated by format. Phone digits only, dropping the 55 country code when 12 or 13 digits remain, keeping 10 or 11. Name uppercased, without accents and without company suffixes (ltda, me, epp, eireli, s/a).
CPF with all identical digits (00000000000, 11111111111), an invalid check digit or scientific notation, empty phones, '0' or '55', and filler emails (naotem@, sememail@) become null. Validate the CPF and CNPJ check digits. A placeholder used as a key merges hundreds of people into one customer.
The front desk phone, the finance email, a broker's or an internal salesperson's email show up on many records. Keep a blocklist of internal addresses and apply a frequency cap: a phone or email present on more than 5 distinct records stops being a key. A name that appears more than 20 times is not used to group candidates either.
1) One system's id stored in another (a custom field with the ERP or old CRM id). 2) A valid CPF or CNPJ. 3) Email. 4) Phone. Collect the matches from every rule and keep, for each record, the highest-priority rule. Store on each link which rule matched.
Matching email and phone are used to find candidates and to corroborate. The name decides: if the first name is very different, the pair is not the same person, even with identical email and phone, because relatives and coworkers share contacts. In one real base, phone alone produced dozens of times more pairs than name plus phone. Only an identical CPF or CNPJ confirms without looking at the name.
When linking an event, ticket, message or sale to a record by phone or email, use only the keys that point to a single record. A key that points to two or more stays unlinked and goes to the queue, instead of picking one at random.
Pick the main record by completeness (how many of CPF, email, phone, name and birth date are filled), breaking ties by the most recent modification date and then by id. Empty fields on the main record are filled from the secondary ones. The tie-break must be deterministic, or each run picks a different winner.
The same person can be a lead, customer, owner, tenant, guarantor, investor or partner. Create one entity per person and one link row per role, instead of one record per role. When a lead and a customer coexist, the customer prevails.
For companies, the entity is the full 14-digit CNPJ. The CNPJ root (first 8 digits) groups headquarters and branches: store it as a group column, never as a merge key. An 11-digit document is an individual; markers in the name (ltda, me, s/a) help separate individuals from companies when the document is missing.
Keep a map from each source id to the entity, updated by upsert, with method, confidence and date. Low-confidence pairs (name plus phone, no document) go to a human review queue, and manual decisions live in a table with who decided and why. Merges already made at the source, such as HubSpot's hs_merged_object_ids, go into the map so old ids still find the entity.
What each key can do
From strongest to weakest. Store on each link which one was used.
Definitions
Pitfalls that already happened
An ERP spreadsheet kept a single company record for several people. The first round of contact creation in the CRM wrote one person's phone, email and CPF onto another, and more than a hundred contacts had to be fixed.
Separate person from company before creating or merging, and never inherit contact data from a company record.
The source load wrote every record twice, with identical content. Every count taken straight from the raw table came out doubled.
Remove exact duplicates on the way in, before any match, and compare rows with distinct ids.
ERP tables kept rows flagged as deleted (deleted, or D_E_L_E_T_ in TOTVS). They matched live records and duplicated customers and receivables.
Filter out deleted records before matching.
With no CPF or email in common, the same customer was spread across four records, one as a customer in one system and three as leads in another CRM, all with the same phone.
Use name plus phone as a signal for human review, not as an automatic merge.
A matched B by phone, B matched C by email, and A and C, who had nothing in common, became the same person.
In the probabilistic model, generate pairs with a permissive threshold and form groups with a strict one (for example, 0.85 and 0.95).
Two records tied on the selection criteria were resolved with no tie-break, and the main record changed from one run to the next.
A fixed tie-break by modification date and, last, by id.
Acceptance test
The case is done when every item passes.
- Each source record (system + id) appears in exactly one entity in the map.
- No entity has two distinct CPFs or two distinct full CNPJs.
- No merge used a placeholder (CPF with identical digits, filler email, short phone) or a key above the frequency cap.
- Purchases or revenue summed by entity equal the sum by source record, with no difference.
- Running the process twice on the same data produces the same entities and the same main records.
Agent package
Paste into Claude, ChatGPT or any client with the Nekt MCP connected.
Identity of people and companies - Customer = entity in the identity table (1 row per person or company). Count and sum by it, never by the source record. - To find a customer, normalize the search: CPF and CNPJ digits only (11 and 14 positions), email lowercased, phone digits only and without 55. - Link confidence order: manual decision > id stored in another system > CPF or CNPJ > email > phone. A name alone never confirms. - A phone or email shared by several entities is an ambiguous key: do not link events or sales through it. - Companies: the entity is the full CNPJ. The CNPJ root is the economic group, in a separate column. - A person can have several roles (lead, customer, owner, tenant, guarantor, partner). When lead and customer coexist, customer wins. - Internal accounts, test accounts and the company's own emails are not customers. - If an answer relies on a link by phone or email, say so. If the record is not in the map, say it was not resolved instead of guessing. Client exceptions: - <client>: internal emails and phones on the blocklist: <list>. - <client>: CRM field <field> stores the ERP id and is the strongest key. - <client>: frequency cap of <n> records per phone or email.
Build the "Deduplication and entity resolution" case in my Nekt workspace following the rules at https://academy.nekt.com/en/casos/deduplicacao-de-cadastros.md. 1) Find the source tables and confirm the columns the rules use. 2) Create a transformation at the stated grain applying every rule. 3) Run the acceptance test and show me the result of each item. 4) Create the case's context document in the workspace, leaving my clients' exceptions open for me to fill in.
Variations
Without documents, a Fellegi-Sunter model (for example, with Splink) groups on clean keys and respects a negative rule: no group can hold two distinct CPFs.
The same logic finds properties listed twice across agencies by normalized address: street number without leading zeros and unit number extracted from the address complement.
For companies, the public CNPJ registry from Receita Federal confirms the document and fills in legal name, activity code (CNAE) and registration status.