TensorTonicTensorTonic
Problems
Study PlansProjectsNewInterviewPricingFeedback
Problems
Loading...
1 / 1

ETL Deduplication

MLOps
Easy

Deduplicate records using the values in key_columns as a composite key. Select one record per key according to strategy:

  • "first": select the first occurrence
  • "last": select the last occurrence
  • "most_complete": select the record with the fewest None values, breaking ties by first occurrence

Regardless of strategy, order the output by the first appearance of each unique key. Return the selected records as a list of dictionaries without changing the input records.

Loading visualization...

Examples

Input: records = [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}], key_columns = ["id"], strategy = "first"

Output: [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}]

Explanation: The two records have different keys, so both remain.

Input: records = [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}, {"id": 1, "name": "Alice Smith", "email": "alice.s@test.com"}], key_columns = ["id"], strategy = "first"

Output: [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}]

Hint 1

Use tuple(record[column] for column in key_columns) as the composite key.

Hint 2

Store key order separately from grouped records, then select from each group.

Requirements

  • Build composite keys in key_columns order
  • Select records according to the chosen strategy
  • Break completeness ties by first occurrence
  • Preserve first-key appearance order

Constraints

  • strategy is first, last, or most_complete
  • Every record contains every key column
  • Key values are hashable
Try Similar Problems
Etl Schema ValidationEtl Dependency OrchestrationFeature Store LookupImpute MissingModel Versioning Basics

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: string

You must run your code first.
PrevNext

ETL Deduplication

MLOps
Easy

Deduplicate records using the values in key_columns as a composite key. Select one record per key according to strategy:

  • "first": select the first occurrence
  • "last": select the last occurrence
  • "most_complete": select the record with the fewest None values, breaking ties by first occurrence

Regardless of strategy, order the output by the first appearance of each unique key. Return the selected records as a list of dictionaries without changing the input records.

Loading visualization...

Examples

Input: records = [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}], key_columns = ["id"], strategy = "first"

Output: [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}]

Explanation: The two records have different keys, so both remain.

Input: records = [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}, {"id": 1, "name": "Alice Smith", "email": "alice.s@test.com"}], key_columns = ["id"], strategy = "first"

Output: [{"id": 1, "name": "Alice", "email": "alice@test.com"}, {"id": 2, "name": "Bob", "email": "bob@test.com"}]

Hint 1

Use tuple(record[column] for column in key_columns) as the composite key.

Hint 2

Store key order separately from grouped records, then select from each group.

Requirements

  • Build composite keys in key_columns order
  • Select records according to the chosen strategy
  • Break completeness ties by first occurrence
  • Preserve first-key appearance order

Constraints

  • strategy is first, last, or most_complete
  • Every record contains every key column
  • Key values are hashable
Try Similar Problems
Etl Schema ValidationEtl Dependency OrchestrationFeature Store LookupImpute MissingModel Versioning Basics

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: string

You must run your code first.