Mobile Quickstart
👉 Take me there! 👈
Requirements​
In addition to dbt being installed and a mobile events dataset being available in your database:
- Snowplow Android or iOS mobile tracker version 1.1.0 or later implemented.
- Mobile session context enabled (ios or android).
- Screen view events enabled (ios or android).
In addition to the standard privileges required by dbt, our packages by default write to additional schemas beyond just your profile schema. If your connected user does not have create schema privileges, you will need to ensure that the following schemas exist in your warehouse and the user can create tables in them:
<profile_schema>_derived<profile_schema>_scratch<profile_schema>_snowplow_manifest
Alternatively, you can override the output schemas our models write to, see the relevant package configuration page for how to do this.
Installation​
Check dbt Hub for the latest installation instructions, or read the dbt docs for more information on installing packages. If you are using multiple packages you may need to up/downgrade a specific package to ensure compatibility.
Setup​
1. Override the dispatch order in your project​
To take advantage of the optimized upsert that the Snowplow packages offer you need to ensure that certain macros are called from snowplow_utils first before dbt-core. This can be achieved by adding the following to the top level of your dbt_project.yml file:
# dbt_project.yml
...
dispatch:
- macro_namespace: dbt
search_order: ['snowplow_utils', 'dbt']
If you do not do this the package will still work, but the incremental upserts will become more costly over time.
2. Adding the selector.yml file​
Within the packages we have provided a suite of suggested selectors to run and test the models within the package together with the mobile model. This leverages dbt's selector flag. You can find out more about each selector in the YAML Selectors section.
These are defined in the selectors.yml file (source) within the package, however in order to use these selections you will need to copy this file into your own dbt project directory. This is a top-level file and therefore should sit alongside your dbt_project.yml file. If you are using multiple packages in your project you will need to combine the contents of these into a single file.
3. Check source data​
This package will by default assume your Snowplow events data is contained in the atomic schema of your target.database, in the table labeled events. In order to change this, please add the following to your dbt_project.yml file:
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__atomic_schema: schema_with_snowplow_events
snowplow__database: database_with_snowplow_events
snowplow__events_table: table_of_snowplow_events
Please note that your target.database is NULL if using Databricks. In Databricks, schemas and databases are used interchangeably and in the dbt implementation of Databricks therefore we always use the schema value, so adjust your snowplow__atomic_schema value if you need to.
4. Enabled desired contexts​
The mobile package has the option to join in data from the following 4 Snowplow contexts:
- Mobile context -- Device type, OS, etc.
- Geolocation context -- Device latitude, longitude, bearing, etc.
- Application context -- App version and build
- Screen context -- Screen details associated with mobile event
By default these are all disabled in the mobile package. Assuming you have the enrichments turned on in your Snowplow pipeline, to enable the contexts within the package please modify the following in your dbt_project.yml file:
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__enable_mobile_context: true
snowplow__enable_geolocation_context: true
snowplow__enable_application_context: true
snowplow__enable_screen_context: true
5. Enable desired modules​
The mobile package has the option to join in data from the following 1 Snowplow module:
- App Errors module -- Details relating to app errors that occur during sessions
By default this module is disabled in the mobile package. Assuming you have the enrichments turned on in your Snowplow pipeline, to enable the module within the package please modify the following in your dbt_project.yml file:
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__enable_app_errors_module: true
6. Filter your data set​
You can specify both start_date at which to start processing events and the app_id's to filter for. By default the start_date is set to 2020-01-01 and all app_id's are selected. To change this please add/modify the following in your dbt_project.yml file:
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__start_date: 'yyyy-mm-dd'
snowplow__app_id: ['my_app_1','my_app_2']
7. Additional vendor specific configuration​
Verify which column your events table is partitioned on. It will likely be partitioned on collector_tstamp or derived_tstamp. If it is partitioned on collector_tstamp you should set snowplow__derived_tstamp_partitioned to false. This will ensure only the collector_tstamp column is used for partition pruning when querying the events table:
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__derived_tstamp_partitioned: false
Add the following variable to your dbt project's dbt_project.yml file
# dbt_project.yml
...
vars:
snowplow_mobile:
snowplow__databricks_catalog: 'hive_metastore'
Depending on the use case it should either be the catalog (for Unity Catalog users from databricks connector 1.1.1 onwards, defaulted to 'hive_metastore') or the same value as your snowplow__atomic_schema (unless changed it should be 'atomic'). This is needed to handle the database property within models/base/src_base.yml.
A more detailed explanation for how to set up your Databricks configuration properly can be found in Unity Catalog support.
8. Run your model​
You can now run your models for the first time by running the below command (see the operation page for more information on operation of the package):
dbt run --selector snowplow_mobile