dbt Integration

Deep dive into connecting your dbt semantic layer to Metricly. Learn about MetricFlow, manifest generation, and CI/CD automation.

Metricly uses the same MetricFlow engine that powers dbt Cloud's Semantic Layer. This means you get enterprise-grade metric consistency without the enterprise price tag. This guide covers everything you need to know about the integration.

Understanding MetricFlow

MetricFlow is dbt Labs' open-source metric layer that sits between your dbt models and your BI tools. It provides:

Consistent Metrics

Define metrics once, use them everywhere. No more conflicting definitions across tools.

Optimized SQL

MetricFlow generates efficient SQL with proper joins, aggregations, and filters.

Version Controlled

Metric definitions live in your dbt project and follow your Git workflow.

Query-Time Access

Data is queried directly from your warehouse. Nothing is stored or cached externally.

dbt Cloud vs. dbt Core
Metricly works with both dbt Core and dbt Cloud. If you use dbt Core, you get the full semantic layer without paying for dbt Cloud Enterprise.

Semantic Models & Metrics

The semantic layer is built from semantic models and metrics defined in your dbt project. Here is an example structure:

Semantic Model Example

models/semantic/orders.ymlyaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# models/semantic/orders.yml
semantic_models:
  - name: orders
    description: "Order transactions"
    model: ref('fct_orders')
    defaults:
      agg_time_dimension: order_date
 
    entities:
      - name: order_id
        type: primary
      - name: customer_id
        type: foreign
 
    dimensions:
      - name: order_date
        type: time
        type_params:
          time_granularity: day
      - name: order_status
        type: categorical
      - name: country
        type: categorical
 
    measures:
      - name: order_total
        agg: sum
        expr: order_amount
      - name: order_count
        agg: count
        expr: order_id

Metric Definition Example

models/semantic/metrics.ymlyaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# models/semantic/metrics.yml
metrics:
  - name: revenue
    description: "Total revenue from orders"
    type: simple
    type_params:
      measure: order_total
    filter: |
      {{ Dimension('order_status') }} = 'completed'
 
  - name: average_order_value
    description: "Average value per order"
    type: derived
    type_params:
      expr: revenue / order_count
      metrics:
        - revenue
        - order_count
Best practices for metrics
  • Use descriptive names that business users will understand
  • Add descriptions to all metrics and dimensions
  • Define a default time dimension for time-series analysis
  • Use filters to exclude invalid or test data

Generating the Semantic Manifest

The semantic manifest is a JSON file that contains all your metrics, dimensions, and entities. Metricly uses this file to understand your semantic layer.

Install dbt-metricflow

terminalbash
# Install the MetricFlow CLI
pip install dbt-metricflow

# Or with your dbt adapter
pip install dbt-metricflow[bigquery]
pip install dbt-metricflow[snowflake]
pip install dbt-metricflow[databricks]

Generate the manifest

terminalbash
# Navigate to your dbt project
cd your-dbt-project

# Parse the project and generate the manifest
dbt parse

# The semantic manifest is created at:
# target/semantic_manifest.json

Validate your metrics (optional)

terminalbash
# Test a metric query locally
mf query --metrics revenue --dimensions order_date__month

# List all available metrics
mf list metrics

# List all available dimensions
mf list dimensions
Keep your manifest in sync
Remember to regenerate and re-upload your manifest whenever you change metric definitions. See the CI/CD section below for automation.

Warehouse Configuration

Metricly needs read access to your data warehouse to execute MetricFlow queries. Here are example configurations for supported warehouses:

BigQuery

bigquery-configyaml
# BigQuery connection settings
project_id: your-gcp-project
dataset: your_dataset
keyfile: /path/to/service-account.json
# Or use GOOGLE_APPLICATION_CREDENTIALS env var

Snowflake

snowflake-configyaml
# Snowflake connection settings
account: your-account.region
user: your_user
password: ${SNOWFLAKE_PASSWORD}
database: YOUR_DATABASE
warehouse: YOUR_WAREHOUSE
schema: YOUR_SCHEMA

Databricks

databricks-configyaml
# Databricks connection settings
host: your-workspace.cloud.databricks.com
http_path: /sql/1.0/warehouses/abc123
token: ${DATABRICKS_TOKEN}
catalog: your_catalog
schema: your_schema
Security best practices
  • Create a dedicated service account for Metricly
  • Grant only SELECT permissions on required schemas
  • Use environment variables for sensitive credentials
  • Rotate credentials regularly

CI/CD Automation

Automate manifest uploads to keep your Metricly dashboards in sync with your dbt project. Here is an example GitHub Actions workflow:

.github/workflows/deploy-manifest.ymlyaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# .github/workflows/deploy-manifest.yml
name: Deploy Semantic Manifest
 
on:
  push:
    branches: [main]
    paths:
      - 'models/semantic/**'
 
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
 
      - name: Install dependencies
        run: |
          pip install dbt-metricflow[bigquery]
 
      - name: Generate manifest
        run: |
          dbt parse
        env:
          DBT_PROFILES_DIR: .
 
      - name: Upload to Metricly
        run: |
          curl -X POST https://metricly.xyz/api/manifest \
            -H "Authorization: Bearer ${{ secrets.METRICLY_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d @target/semantic_manifest.json

API Upload

You can also upload manifests directly via the API:

terminalbash
# Upload manifest via API
curl -X POST https://metricly.xyz/api/manifest \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d @target/semantic_manifest.json

# Response
{
  "success": true,
  "metrics_count": 12,
  "dimensions_count": 24,
  "entities_count": 5
}

Troubleshooting

Manifest not found

If dbt parse does not create a semantic_manifest.json:

  • Ensure you have dbt-metricflow installed
  • Check that your dbt version is 1.6 or later
  • Verify you have semantic models defined in your project

Metric queries failing

If queries return errors after uploading the manifest:

  • Verify warehouse credentials are correct
  • Ensure the service account has access to the required schemas
  • Test queries locally with mf query
  • Check that model references in semantic models are valid

Missing dimensions

If some dimensions are not appearing in Metricly:

  • Ensure dimensions are defined in semantic models, not just dbt models
  • Check that dimension names do not conflict across semantic models
  • Regenerate and re-upload the manifest

Next Steps

Dashboard Building Guide

Learn about widget types, controls, and building production dashboards.