Duplicate Detection

When families submit multiple applications or when data is imported from different sources, duplicate records can occur. The Leads Management module automatically detects potential duplicates to help you maintain a clean and accurate applicant database.

How Detection Works

When a new lead is created -- whether through a form submission, manual entry, or data import -- the system checks for existing records that might be duplicates. The detection runs automatically and flags matches for your review.

Detection Methods

The system uses the following methods to identify potential duplicates:

1. Email matching -- the primary detection rule. If a new applicant shares a parent/guardian email address with an existing applicant, they are flagged as potential duplicates.

2. Name and date of birth matching -- if the child's name and birthdate match an existing record, it is flagged.

3. Name-only matching -- a less strict rule that matches on child name alone (produces more potential false positives).

4. Parent name and phone matching -- matches records where the parent name and phone number are identical.

5. Parent name and address matching -- matches records where the parent name and address fields match.

TIP

Email matching is the most reliable detection rule and is the default. The other rules can be configured depending on your school's needs and the quality of your data.

Duplicate Indicators

When a potential duplicate is detected, several visual indicators appear:

• Amber flag icon -- appears next to the applicant's name in the table view.
• Duplicate badge -- shown on the applicant's detail page.
• Dashboard count -- the pipeline statistics include a "Potential Duplicates" count.
• is_potential_duplicate flag -- set to true on the lead record.

Viewing Duplicate Details

To review a potential duplicate:

1. Open the applicant's detail page by clicking their name in the table or Kanban board.
2. If the applicant is flagged as a potential duplicate, you will see a duplicate notification panel.
3. The panel shows the linked duplicate record, including:
   • The other applicant's name and key details
   • Guardian email that triggered the match
   • The stage and status of the other record

Duplicate Detection at Creation

When a new applicant is created (via form submission or API), the system:

1. Searches for an existing lead with the same guardian email within the same school.
2. If found, sets the duplicate_of_lead_id field to reference the existing record.
3. Marks the new lead as is_potential_duplicate = true.
4. Creates an audit log entry of type potential_duplicate.

WARNING

Duplicate detection does not block the creation of new records. Both the original and the duplicate are kept until you review and decide to merge or dismiss the flag.

Dismissing a Duplicate Flag

If a flagged duplicate is actually a separate, legitimate applicant (e.g., siblings with the same parent email), you can dismiss the duplicate flag:

1. Open the applicant's detail page.
2. In the duplicate notification panel, click Dismiss.
3. The is_potential_duplicate flag is cleared and the duplicate_of_lead_id reference is removed.
4. The amber flag icon disappears from the table view.

Family Detection Service

The system includes a more advanced Family Detection Service that groups leads into potential family clusters. This service uses the parents table and family_members junction table to identify:

• Families sharing a parent -- when the same parent record (identified by email) is linked to multiple families, those families are grouped.
• Orphan leads -- leads without a family record that share a parent email with an existing family are identified and flagged for linking.

The detection rules available are:

Rule	What It Matches
parent_email	Leads sharing a parent with the same email address
child_name_dob	Leads with identical child name and date of birth
child_name_only	Leads with identical child name (less strict)
parent_name_phone	Leads with identical parent name and phone number
parent_name_address	Leads with identical parent name and address

Was this helpful?