Skip to main content

Filter Node

The Filter Node is a versatile server-side tool designed to filter datasets based on a set of user-defined predicates. This node allows you to apply complex filtering logic to your data, enabling precise control over which records are included in the output. By leveraging predicates, operators, and logical conditions, the Filter Node provides a powerful mechanism for refining datasets to meet specific requirements.

Overview​

The Filter Node operates entirely on the server, ensuring efficient processing of large datasets. It requires an array of predicates, where each predicate specifies a filter name, values, operators, and logical operators. This structure allows for flexible and dynamic filtering, making it suitable for a wide range of use cases.

Key Features​

The Filter Node offers the following features:

  • Predicate-based filtering: Define filters using predicates that include filter names, values, operators, and logical conditions.
  • Server-side processing: All filtering operations are performed on the server for high performance and scalability.
  • Linearization by value: Optionally explode filters by data values for granular control, provided the dataset is not excessively large.
  • Customizable logic: Combine predicates using logical operators (e.g., AND, OR) to create complex filtering rules.
  • Seamless integration: Works effortlessly with other nodes in your workflow to streamline data preparation.

How it works​

Using the Filter Node involves the following steps:

  1. Add the Filter Node to your data flow.
  2. Define an array of predicates, where each predicate includes:
    • Filter name: The name of the filter to apply (e.g., "age", "status").
    • Values: The values to filter by (e.g., [25, 30, 35], ["active", "inactive"]).
    • Operators: The comparison operators to use (e.g., =, !=, <, >, IN).
    • Logical operator: The logical condition to combine predicates (e.g., AND, OR).
  3. Connect your dataset to the node.
  4. The node processes the data on the server, applying the defined predicates to filter the dataset.
  5. Retrieve the filtered dataset for further analysis or visualization.

Use cases​

The Filter Node is ideal for a variety of scenarios, including:

  • Data exploration: Quickly filter datasets to focus on specific subsets of data for analysis.
  • Data preparation: Refine datasets by removing unwanted records or focusing on relevant data points.
  • Dynamic filtering: Apply user-defined filters based on real-time input or external conditions.
  • Complex queries: Implement advanced filtering logic that combines multiple conditions and operators.

With the Filter Node, you can efficiently manage and manipulate your datasets, ensuring that only the relevant records are retained for analysis or reporting.