Models

Models is the main concept you need to be aware of when using Quary. A model is simply the definition of a transformation.

Although seemingly, it's actually rather simple. A model is a way to define how to transform data from one shape into another. Those transformations can be simple (like renaming a column) or more complex (joining, filtering, summing). That's it, some examples are:

• Renaming a column in a table
• Joining two tables together on a key
• Filtering a table to only include rows that match a certain criteria

At its core, Quary is a way to define these transformations and concatenate them together to build complex transformations, which sometimes may be called a pipeline.

A model definition

In Quary, a model is made up of two parts:

• The transformation itself
• Documentation for that model

SQL File

The transformation itself is a .sql that is placed inside the models folder. The name of the file defines the name of the model. For example, if you have a file called models/total_orders.sql then the model name is total_orders. The contents of the file is the SQL that defines the transformation. For example:

SELECT COUNT(*) AS total_orders FROM q.orders

The above SQL is a simple transformation that counts the number of rows in the orders model. Note the q. prefix, this is the prefix that is used to reference other models in the project and a key part of how Quary works. Models should not reference tables directly, they should always reference other models or sources.

Documentation

The documentation is a .yaml file that is placed inside the models folder. The path of the file for the above example could be models/total_orders.yaml. The contents of the file is the documentation for the model. For example:

models:
  - name: total_orders
    description: The total number of orders
    tags:
      - metric
    columns:
      - name: total_orders
        description: The total number of orders
        tests:
          - type: not_null
          - type: gte
            info:
              value: 0

Note in the above example, the name field matches the name of the model file. This is how Quary knows which documentation file to use for which model. In addition, the documentation file outlines

• a description field which is used to describe and document the model
• a tags field which is used to categorize the model with various tags
• a columns field which is used to describe the columns that are returned by the model

The columns field is an array of objects that describe each column. In the above example, there is only one column called total_orders which has a description and a set of tests. Tests are a key part of Quary. The provide confidence and help to ensure that the data is in fact correct.